Blosc, an extremely fast, multi-threaded, meta-compressor library
What Can Blosc Do For You?
Plain and simple: it allows you to compress – but especially – decompress your data, fast.
What Makes Blosc Different?
High performance compressor, optimized for binary data (but text is fine too).
Designed to transmit data to the processor cache faster than a memcpy() OS call. Achieving that will obviously depend on the dataset, but that is the goal.
Leverages SIMD (SSE2, AVX2) and multi-threading capabilities present in modern multi-core processors.
APIs for C, Python, Julia and more.
It can use different, very fast compressors. Just write your code once and get access to an amazing range of compressors, like BloscLZ, LZ4, LZ4HC, Snappy, Zlib or Zstd.
Want to contribute to the Blosc ecosystem?
We are eager to hear about ideas and code contributions (via Pull Requests preferably)! If you don't have time to contribute code and you would like us to do the job, you can contract us.
Enjoy data!
Why different compressors?
There is a reason why such a huge variety of compressors exist: each one has its own strengths for different scenarios. Blosc is all about speed, so it has integrated some of the fastest ones available, as well as some achieving an excellent speed/compression ratio:
Compression pre-filters
In some situations, using filters before doing the actual compression may improve both performance and/or compression ratio. Blosc comes with a couple of filters (also called pre-conditioners) called shuffle and bitshuffle which rearrange bytes and bits in a clever way for the compression stage to work more efficiently and specially, faster.
Professional services
NEW: We are providing professional services for Blosc and its ecosystem.