blosc2.open#

blosc2.open(urlpath: str | Path | URLPath, mode: str = 'a', offset: int = 0, **kwargs: dict) → SChunk | NDArray | C2Array | LazyArray | Proxy#

Open a persistent SChunk, NDArray, a remote C2Array or a Proxy

See the Notes section for more info on opening Proxy objects.

Parameters:

  • urlpath (str | pathlib.Path | URLPath class) – The path where the SChunk (or NDArray) is stored. If it is a remote array, a URLPath class must be passed.

  • mode (str, optional) – The open mode.

  • offset (int, optional) – An offset in the file where super-chunk or array data is located (e.g. in a file containing several such objects).

  • kwargs (dict, optional) –

    mmap_mode: The memory mapping mode. initial_mapping_size: The initial size of the memory mapping. cparams: dict

    A dictionary with the compression parameters, which are the same that can be used in the compress2() function. Typesize and blocksize cannot be changed.

    dparams: dict

    A dictionary with the decompression parameters, which are the same that can be used in the decompress2() function.

Returns:

out – The SChunk or NDArray (if there is a “b2nd” metalayer”) or the C2Array if urlpath is a blosc2.URLPath instance.

Return type:

SChunk, NDArray or C2Array

Notes

  • This is just a ‘logical’ open, so there is no close() counterpart because currently, there is no need for it.

  • If urlpath is a URLPath class instance, mode must be ‘r’, offset must be 0, and kwargs cannot be passed.

  • If the original object saved in urlpath is a Proxy, this function will only return a Proxy if its source is a local SChunk, NDArray or a remote C2Array. Otherwise, it will return the Python-Blosc2 container used to cache the data which can be a SChunk or a NDArray and may not have all the data initialized (e.g. if the user has not accessed to it yet).

  • When opening a LazyExpr keep in mind the note above regarding operands.

Examples

>>> import blosc2
>>> import numpy as np
>>> import os
>>> import tempfile
>>> tmpdirname = tempfile.mkdtemp()
>>> urlpath = os.path.join(tmpdirname, 'b2frame')
>>> storage = blosc2.Storage(contiguous=True, urlpath=urlpath, mode="w")
>>> nelem = 20 * 1000
>>> nchunks = 5
>>> chunksize = nelem * 4 // nchunks
>>> data = np.arange(nelem, dtype="int32")
>>> # Create SChunk and append data
>>> schunk = blosc2.SChunk(chunksize=chunksize, data=data.tobytes(), storage=storage)
>>> # Open SChunk
>>> sc_open = blosc2.open(urlpath=urlpath)
>>> for i in range(nchunks):
...     dest = np.empty(nelem // nchunks, dtype=data.dtype)
...     schunk.decompress_chunk(i, dest)
...     dest1 = np.empty(nelem // nchunks, dtype=data.dtype)
...     sc_open.decompress_chunk(i, dest1)
...     np.array_equal(dest, dest1)
True
True
True
True
True

To open the same schunk memory-mapped, we simply need to pass the mmap_mode parameter:

>>> sc_open_mmap = blosc2.open(urlpath=urlpath, mmap_mode="r")
>>> sc_open.nchunks == sc_open_mmap.nchunks
True
>>> all(sc_open.decompress_chunk(i, dest1) == sc_open_mmap.decompress_chunk(i, dest1) for i in range(nchunks))
True
On this page