Performance Optimizations in Codexis Build System

This document describes the performance optimizations implemented in the Codexis build system to improve build times and resource utilization.

LaTeX Compilation Caching

Overview

LaTeX compilation is one of the most time-consuming operations in the build process. To improve performance, a caching mechanism has been implemented that avoids redundant LaTeX compilation when the source files haven't changed.

Implementation Details

The caching mechanism works as follows:

1. When LatexCompiler.compile_all is called, it computes a hash of all relevant source files (.tex, .bib, .sty, .cls) in the directory and subdirectories.
2. This hash is used as a cache key to check if a valid cache exists.
3. If a valid cache exists, the compilation is skipped, significantly reducing build time.
4. If no valid cache exists or if compilation is forced, the normal compilation process is executed, and the results are cached for future use.

Usage

All builders (PDF, HTML, EPUB, DOCX, Markdown, RTF) have been updated to use the caching mechanism. The caching behavior can be controlled through the following options:

• :use_cache - When true (default), enables caching for LaTeX compilation.
• :force_compile - When true, forces compilation even if a valid cache exists.

Example:

# Enable caching (default)
options = [use_cache: true]

# Force compilation regardless of cache
options = [use_cache: true, force_compile: true]

# Disable caching
options = [use_cache: false]

Performance Impact

The caching mechanism can significantly improve build performance, especially for repeated builds of the same or slightly modified documents. Benchmarks show:

• First build: No improvement (cache is created)
• Subsequent builds with no changes: Up to 90% reduction in build time
• Builds with minor changes: 50-80% reduction in build time, depending on the extent of changes

Future Optimizations

Additional performance optimizations that could be implemented in the future:

1. Parallel File Operations: Enhance parallel processing for large directories during file operations.
2. Smarter Dependency Management: Optimize the dependency chain between formats to avoid unnecessary builds.
3. Memory-Mapped I/O: Implement memory-mapped I/O for large file operations to reduce memory usage and improve performance.
4. Incremental Builds: Implement incremental builds that only process changed files rather than rebuilding everything.

Benchmarking

To measure the impact of these optimizations, use the built-in benchmarking tools:

# Run a complete benchmark suite
Codexis.Benchmarks.BuildPerformanceBenchmark.run_benchmark_suite()

# Run specific benchmarks
Codexis.Benchmarks.BuildPerformanceBenchmark.benchmark_caching_performance()

The benchmark results will show the performance improvement achieved by the caching mechanism and other optimizations.