When More Threads Make Things Slower

Johnny Kao — Mon, 12 Oct 2026 13:00:00 GMT

Why matching the number of workers to the available compression blocks can make small requests much faster.

More threads usually help compression when there is enough data to process in parallel. For a small request, however, the cost of preparing those threads can exceed the useful work. While working on C-Blosc2 PR #805, I found a case where a 16 KiB compression request with sixteen configured threads took about 710 µs with a fresh context. After the change, the same measured operation took about 18 µs, almost 39× faster. This is a result for that particular request and context lifecycle, not a 39× increase in the compression algorithm's throughput.

The improvement came from two decisions that are easy to distinguish: skip parallel startup when a request contains only one block, and avoid scheduling workers that have no blocks to process when a request contains only a few blocks. Neither requires changing how the data is compressed. To understand the results, we first need to look at what each decision avoids, and then at the tests used to evaluate them.

Blosc Home Page (Posts by Johnny Kao)

When More Threads Make Things Slower