I am an independent consultant and developer with more than 15 years of experience in Python and C programming languages, and almost 25 years leveraging compression techniques for accelerating I/O. With customer needs in mind I can apply my experience to help determining the best options for handling and analyzing big datasets in the most efficient way.
As you may know, I am the main author of PyTables, bcolz as well as the ultra-fast Blosc compressor that has become the basis for some novel and innovative libraries in the I/O space. In particular, the bcolz project provides a highly efficient way to perform out-of-core queries and computations on column stored datasets that has become particularly interesting for the analysis of medium/large-sized time-series datasets.
What I offer:
- Development services on existing or new packages in the Blosc ecosystem (including PyTables and bcolz).
- Development services for custom software that is for doing optimal I/O (either using Blosc/PyTables/bcolz or not).
- Consulting services on better leveraging the Blosc ecosystem in your own setups. Services like implementing new functionality, providing efficient binaries or optimizing parameters for large streams of chunks are typical.
For fees or any other inquiries, please send me a message at email@example.com.