NixCon 2025

Embarrassingly parallel evaluations
2025-09-06 , Lecture Hall

tl;dr: How we reduced evaluation time for 300+ packages from 5 mintues to 5 seconds.


Nix offers an unparalleled collection of readily packaged C/C++ libraries, with the added benefits of cross-compilation support and a declarative configuration language. That's why we chose it to manage third-party dependencies for our Bazel-built monorepo.
This not-too-common use of Nix as a polyglot build system for external libraries and tools led us to approach Nix in a different way, both in how we think about it and how we used it. Our usage scenarios seemed rarely exercised an we found several opportunities for optimization in Nix code itself.

At the core of the problem, we had 300+ packages that we needed to evaluate and build on demand. These evaluation times quickly added up, and we had to rethink the design to get some optimizations.
This task set is embarrassingly parallel as each package evaluation is independent. But we realized along the way that full parallelism wasn’t the best we could do.

We will explain how we leveraged nix evaluation caches (plural!), flakes, manually crafted store paths and parallel execution to reduce evaluation time from 5 minutes to 5 seconds, and bring other less quantifiable improvements.
The work led to a few interesting scripts and two main PRs to nix itself (see below), one of which is already merged and released while the second is subject to discussion because of its impact on other cached operations. We will use that discussion to illustrate the tradeoffs of caching parallel evaluation.

See also:

Guillaume Maudoux (a.k.a @layus on GitHub and elsewhere) is an engineer and researcher specialized in build systems.

Based in Belgium, Guillaume blends a background in computer science, engineering, and applied mathematics with strong interests in correctness, reliability, and efficiency in software systems.

Guillaume works at Tweag’s Scalable Build Systems group. A team of engineers and researchers focused on developer productivity, robust tooling, and enterprise-scale build performance. His work focusses on integrating Nix into Bazel and in particular supporting remote execution. For some reason his skills are most appreciated in the robotics industry.

You may find him contributing to random open source projects because he never resists fixing a bug he found, and digging to uncover the root cause of problems. He will happily share his findings in blogs, conferences or around a coffeeC-w tea.

Guillaume’s reputation stems from his ability to bridge academic rigor with systems engineering pragmatism. His work helps teams build complex software more reliably, efficiently, and at scale.