Blosc2 NDim#

It contains both the data and metalayer that stores the dimensional info for the array. Blosc2 NDim has a managed internal context that stores the different properties of each array.

Context#

typedef struct b2nd_context_s b2nd_context_t#: General parameters needed for the creation of a b2nd array.

Creation#

b2nd_context_t *b2nd_create_ctx(const blosc2_storage *b2_storage, int8_t ndim, const int64_t *shape, const int32_t *chunkshape, const int32_t *blockshape, const char *dtype, int8_t dtype_format, const blosc2_metalayer *metalayers, int32_t nmetalayers)#

Create b2nd params.

Parameters:

b2_storage – The Blosc2 storage params.
ndim – The dimensions.
shape – The shape.
chunkshape – The chunk shape.
blockshape – The block shape.
dtype – The data type expressed as a string version.
dtype_format – The data type format; DTYPE_NUMPY_FORMAT should be chosen for NumPy compatibility.
metalayers – The memory pointer to the list of the metalayers desired.
nmetalayers – The number of metalayers.

Returns:

A pointer to the new b2nd params. NULL is returned if this fails.

Note

The pointer returned must be freed when not used anymore with b2nd_free_ctx.

Destruction#

int b2nd_free_ctx(b2nd_context_t *ctx)#

Free the resources associated with b2nd_context_t.

Parameters:

ctx – The b2nd context to free.

Returns:

An error code.

Note

This is safe in the sense that it will not free the schunk pointer in internal cparams.

Array#

A Blosc2 NDim array is a n-dimensional object that can be managed by the associated functions. The functions let users to perform different operations with these arrays like copying, getting, setting or converting data into buffers or files and vice-versa. Furthermore, Blosc2 NDim only stores the type size (not the data type), and every item of an array has the same size.

The b2nd_array_t type struct is where all data and metadata for an array is stored:

struct b2nd_array_t#: A multidimensional array of data that can be compressed.

Creation#

Constructors#

int b2nd_uninit(b2nd_context_t *ctx, b2nd_array_t **array)#

Create an uninitialized array.

Parameters:

ctx – The b2nd context for the new array.
array – The memory pointer where the array will be created.

Returns:

An error code.

int b2nd_empty(b2nd_context_t *ctx, b2nd_array_t **array)#

Create an empty array.

Parameters:

ctx – The b2nd context for the new array.
array – The memory pointer where the array will be created.

Returns:

An error code.

int b2nd_zeros(b2nd_context_t *ctx, b2nd_array_t **array)#

Create an array, with zero being used as the default value for uninitialized portions of the array.

Parameters:

ctx – The b2nd context for the new array.
array – The memory pointer where the array will be created.

Returns:

An error code.

int b2nd_nans(b2nd_context_t *ctx, b2nd_array_t **array)#

Create an array, with NaN being used as the default value for uninitialized portions of the array.

Should only be used with type sizes of either 4 or 8. Other sizes generate an error.

Parameters:

ctx – The b2nd context for the new array.
array – The memory pointer where the array will be created.

Returns:

An error code.

int b2nd_full(b2nd_context_t *ctx, b2nd_array_t **array, const void *fill_value)#

Create an array, with fill_value being used as the default value for uninitialized portions of the array.

Parameters:

ctx – The b2nd context for the new array.
array – The memory pointer where the array will be created.
fill_value – Default value for uninitialized portions of the array.

Returns:

An error code.

From/To buffer#

int b2nd_from_cbuffer(b2nd_context_t *ctx, b2nd_array_t **array, const void *buffer, int64_t buffersize)#

Create a b2nd array from a C buffer.

Parameters:

ctx – The b2nd context for the new array.
array – The memory pointer where the array will be created.
buffer – The buffer where source data is stored.
buffersize – The size (in bytes) of the buffer.

Returns:

An error code.

int b2nd_to_cbuffer(const b2nd_array_t *array, void *buffer, int64_t buffersize)#

Extract the data from a b2nd array into a C buffer.

Parameters:

array – The b2nd array.
buffer – The buffer where the data will be stored.
buffersize – Size (in bytes) of the buffer.

Returns:

An error code.

From/To file#

int b2nd_open(const char *urlpath, b2nd_array_t **array)#

Open a b2nd array from a file.

Parameters:

urlpath – The path of the b2nd array on disk.
array – The memory pointer where the array info will be stored.

Returns:

An error code.

int b2nd_open_offset(const char *urlpath, b2nd_array_t **array, int64_t offset)#

Open a b2nd array from a file using an offset.

Parameters:

urlpath – The path of the b2nd array on disk.
array – The memory pointer where the array info will be stored.
offset – The offset in the file where the b2nd array frame starts.

Returns:

An error code.

int b2nd_save(const b2nd_array_t *array, char *urlpath)#

Save b2nd array into a specific urlpath.

Parameters:

array – The array to be saved.
urlpath – The urlpath where the array will be stored.

Returns:

An error code.

From Blosc object#

int b2nd_from_schunk(blosc2_schunk *schunk, b2nd_array_t **array)#

Create a b2nd array from a super-chunk.

It can only be used if the array is backed by a blosc super-chunk.

Parameters:

schunk – The blosc super-chunk where the b2nd array is stored.
array – The memory pointer where the array will be created.

Returns:

An error code.

int b2nd_from_cframe(uint8_t *cframe, int64_t cframe_len, bool copy, b2nd_array_t **array)#

Create a b2nd array from a serialized super-chunk.

Parameters:

cframe – The buffer of the in-memory array.
cframe_len – The size (in bytes) of the in-memory array.
copy – Whether b2nd should make a copy of the cframe data or not. The copy will be made to an internal sparse frame.
array – The memory pointer where the array will be created.

Returns:

An error code.

int b2nd_to_cframe(const b2nd_array_t *array, uint8_t **cframe, int64_t *cframe_len, bool *needs_free)#

Create a serialized super-chunk from a b2nd array.

Parameters:

array – The b2nd array to be serialized.
cframe – The pointer of the buffer where the in-memory array will be copied.
cframe_len – The length of the in-memory array buffer.
needs_free – Whether the buffer should be freed or not.

Returns:

An error code

Modify data#

int b2nd_insert(b2nd_array_t *array, const void *buffer, int64_t buffersize, int8_t axis, int64_t insert_start)#

Insert given buffer in an array extending the given axis.

Parameters:

array – The array to insert the data in.
buffer – The buffer data to be inserted.
buffersize – The size (in bytes) of the buffer.
axis – The axis that will be extended.
insert_start – The position inside the axis to start inserting the data.

Returns:

An error code.

int b2nd_append(b2nd_array_t *array, const void *buffer, int64_t buffersize, int8_t axis)#

Append a buffer at the end of a b2nd array.

Parameters:

array – The array to append the data in.
buffer – The buffer data to be appended.
buffersize – Size (in bytes) of the buffer.
axis – The axis that will be extended to append the data.

Returns:

An error code.

int b2nd_delete(b2nd_array_t *array, int8_t axis, int64_t delete_start, int64_t delete_len)#

Delete shrinking the given axis delete_len items.

Parameters:

array – The array to shrink.
axis – The axis to shrink.
delete_start – The start position from the axis to start deleting chunks.
delete_len – The number of items to delete to the array->shape[axis]. The newshape[axis] will be the old array->shape[axis] - delete_len

Returns:

An error code.

Note

Copying#

int b2nd_copy(b2nd_context_t *ctx, const b2nd_array_t *src, b2nd_array_t **array)#

Make a copy of the array data.

The copy is done into a new b2nd array.

Parameters:

ctx – The b2nd context for the new array.
src – The array from which data is copied.
array – The memory pointer where the array will be created.

Returns:

An error code

Note

The ndim and shape in ctx will be overwritten by the src ctx.

Slicing#

int b2nd_get_slice(b2nd_context_t *ctx, b2nd_array_t **array, const b2nd_array_t *src, const int64_t *start, const int64_t *stop)#

Get a slice from an array and store it into a new array.

Parameters:

ctx – The b2nd context for the new array.
array – The memory pointer where the array will be created.
src – The array from which the slice will be extracted
start – The coordinates where the slice will begin.
stop – The coordinates where the slice will end.

Returns:

An error code.

Note

The ndim and shape from ctx will be overwritten by the src and stop-start respectively.

int b2nd_get_slice_cbuffer(const b2nd_array_t *array, const int64_t *start, const int64_t *stop, void *buffer, const int64_t *buffershape, int64_t buffersize)#

Get a slice from an array and store it into a C buffer.

Parameters:

array – The array from which the slice will be extracted.
start – The coordinates where the slice will begin.
stop – The coordinates where the slice will end.
buffershape – The shape of the buffer.
buffer – The buffer where the data will be stored.
buffersize – The size (in bytes) of the buffer.

Returns:

An error code.

int b2nd_set_slice_cbuffer(const void *buffer, const int64_t *buffershape, int64_t buffersize, const int64_t *start, const int64_t *stop, b2nd_array_t *array)#

Set a slice in a b2nd array using a C buffer.

Parameters:

buffer – The buffer where the slice data is.
buffershape – The shape of the buffer.
buffersize – The size (in bytes) of the buffer.
start – The coordinates where the slice will begin.
stop – The coordinates where the slice will end.
array – The b2nd array where the slice will be set

Returns:

An error code.

int b2nd_get_orthogonal_selection(const b2nd_array_t *array, int64_t **selection, int64_t *selection_size, void *buffer, int64_t *buffershape, int64_t buffersize)#

Get an element selection along each dimension of an array independently.

Parameters:

array – The array to get the data from.
selection – The elements along each dimension.
selection_size – The size of the selection along each dimension.
buffer – The buffer for getting the data.
buffershape – The shape of the buffer.
buffersize – The buffer size (in bytes).

Returns:

An error code.

Note

Utils#

int b2nd_print_meta(const b2nd_array_t *array)#

Print metalayer parameters.

Parameters:

array – The array where the metalayer is stored.

Returns:

An error code

int b2nd_serialize_meta(int8_t ndim, const int64_t *shape, const int32_t *chunkshape, const int32_t *blockshape, const char *dtype, int8_t dtype_format, uint8_t **smeta)#

Create the metainfo for the b2nd metalayer.

Parameters:

ndim – The number of dimensions in the array.
shape – The shape of the array.
chunkshape – The shape of the chunks in the array.
blockshape – The shape of the blocks in the array.
dtype – A string representation of the data type of the array.
dtype_format – The format of the dtype representation. 0 means NumPy.
smeta – The msgpack buffer (output).

Returns:

An error code.

int b2nd_deserialize_meta(const uint8_t *smeta, int32_t smeta_len, int8_t *ndim, int64_t *shape, int32_t *chunkshape, int32_t *blockshape, char **dtype, int8_t *dtype_format)#

Read the metainfo in the b2nd metalayer.

Parameters:

smeta – The msgpack buffer (input).
smeta_len – The length of the smeta buffer (input).
ndim – The number of dimensions in the array (output).
shape – The shape of the array (output).
chunkshape – The shape of the chunks in the array (output).
blockshape – The shape of the blocks in the array (output).
dtype – A string representation of the data type of the array (output).
dtype_format – The format of the dtype representation (output). 0 means NumPy (the default).

Returns:

An error code.

Note

This function is inlined and available even when not linking with libblosc2.

int b2nd_resize(b2nd_array_t *array, const int64_t *new_shape, const int64_t *start)#

Resize the shape of an array.

Parameters:

array – The array to be resized.
new_shape – The new shape from the array.
start – The position in which the array will be extended or shrunk.

Returns:

An error code

Destruction#

int b2nd_free(b2nd_array_t *array)#

Free an array.

Parameters:

array – The array.

Returns:

An error code.

Utilities#

These functions may be used for working with plain C buffers representing multidimensional arrays.

int b2nd_copy_buffer(int8_t ndim, uint8_t itemsize, const void *src, const int64_t *src_pad_shape, const int64_t *src_start, const int64_t *src_stop, void *dst, const int64_t *dst_pad_shape, const int64_t *dst_start)#

Copy a slice of a source array into another array.

The arrays have the same number of dimensions (though their shapes may differ), the same item size, and they are stored as C buffers with contiguous data (any padding is considered part of the array).

Parameters:

ndim – The number of dimensions in both arrays.
itemsize – The size of the individual data item in both arrays.
src – The buffer for getting the data from the source array.
src_pad_shape – The shape of the source array, including padding.
src_start – The source coordinates where the slice will begin.
src_stop – The source coordinates where the slice will end.
dst – The buffer for setting the data into the destination array.
dst_pad_shape – The shape of the destination array, including padding.
dst_start – The destination coordinates where the slice will be placed.

Returns:

An error code.

Note

Please make sure that slice boundaries fit within the source and destination arrays before using this function, as it does not perform these checks itself.