Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
Python-Blosc2 documentation
Logo
Python-Blosc2 documentation
  • Getting Started
    • Overview
    • Installation
  • Tutorials
    • NDArray: A NDim, Compressed Data Container
    • Expressions containing NDArray objects
    • User Defined Functions
    • Lazy array kernels with Menudet
    • Optimizing data reductions with NDArrays
    • Advanced Lazy Expressions and Persistent Reductions
    • Using Proxies for Efficient Handling of Remote Multidimensional Data
    • Compressing data with the SChunk class
    • Slicing, extending and serializing with SChunks
    • User-defined codecs and filters
    • Prefilters, postfilters and fillers
    • Working with Containers
    • Working with ObjectArray
    • Working with BatchArray
    • CTable Tutorial
    • Indexing Arrays
    • Indexing CTables
  • Guides
    • Performance & Benchmarks
    • Optimization tips
    • Menudet: portable computation
    • Migrating to Menudet
    • Working with Remote Data
    • Working with Remote Arrays
    • Working with Remote Tables
    • Sharing Containers Across Processes
    • Using Blosc2 with pandas
    • b2view: Browse TreeStore Bundles in the Terminal
    • Parquet to Blosc2 Walkthrough
  • API Reference
    • Blosc2 Classes
      • NDArray
      • C2Array
      • LazyArray
      • BatchArray
      • ListArray
      • ObjectArray
      • Proxy
      • RemoteObject
      • RemoteArray
      • RemoteStore
      • RemoteRepository
      • RemoteFile
      • RemoteCTable
      • ProxySource
      • ProxyNDSource
      • ByteRangeNDSource
      • FsspecNDSource
      • ZarrNDSource
      • HDF5NDSource
      • B2ZNDSource
      • SimpleProxy
      • Traffic
      • EmbedStore
      • DictStore
      • TreeStore
      • CTable
      • Column
      • NestedColumn
      • Schema Specs
      • Index
      • SChunk
      • Array
      • NDField
      • Ref
    • Operations with arrays
      • Universal Functions (ufuncs)
      • Reduction Functions
      • Linear Algebra
      • Random Functions
      • Additional Functions and Type Utilities
      • Indexing and Manipulation Functions and Utilities
    • Miscellaneous
    • Compression Utilities
      • blosc2.compress
      • blosc2.compress2
      • blosc2.decompress
      • blosc2.decompress2
      • blosc2.pack
      • blosc2.pack_array
      • blosc2.pack_array2
      • blosc2.pack_tensor
      • blosc2.unpack
      • blosc2.unpack_array
      • blosc2.unpack_array2
      • blosc2.unpack_tensor
      • blosc2.clib_info
      • blosc2.compressor_list
      • blosc2.detect_number_of_cores
      • blosc2.free_resources
      • blosc2.get_clib
      • blosc2.nthreads
      • blosc2.print_versions
      • blosc2.register_codec
      • blosc2.register_filter
      • blosc2.set_blocksize
      • blosc2.set_nthreads
      • blosc2.set_prefetch
      • blosc2.set_releasegil
      • blosc2.set_compressor
      • blosc2.get_compressor
      • blosc2.get_blocksize
      • blosc2.get_cbuffer_sizes
      • blosc2.cparams_dflts
      • blosc2.dparams_dflts
      • blosc2.storage_dflts
      • blosc2.Codec
      • blosc2.Filter
      • blosc2.SpecialValue
      • blosc2.SplitMode
      • blosc2.Tuner
      • blosc2.compute_chunks_blocks
      • blosc2.get_slice_nchunks
      • blosc2.remove_urlpath
      • blosc2.blosclib_version
      • blosc2.DEFINED_CODECS_STOP
      • blosc2.GLOBAL_REGISTERED_CODECS_STOP
      • blosc2.USER_REGISTERED_CODECS_STOP
      • blosc2.EXTENDED_HEADER_LENGTH
      • blosc2.MAX_BUFFERSIZE
      • blosc2.MAX_BLOCKSIZE
      • blosc2.MAX_OVERHEAD
      • blosc2.MAX_TYPESIZE
      • blosc2.MIN_HEADER_LENGTH
      • blosc2.prefilter_funcs
      • blosc2.postfilter_funcs
      • blosc2.ucodecs_registry
      • blosc2.ufilters_registry
      • blosc2.VERSION_DATE
      • blosc2.VERSION_STRING
      • blosc2.__version__
    • Compression, decompression and storage parameters
    • Save and load
    • Expression Utilities
    • Menudet syntax
    • Menudet API
    • Menudet numerical behavior
    • JIT execution options
  • Glossary
  • Development
    • Contributing guidelines
    • Code of Conduct
    • List of desired features for a 4.0 release
    • Remote cache design
    • Menudet numerical qualification
    • Typed expression evaluator rollout
  • Release Notes
Back to top

Menudet API¶

Menudet is the supported language for portable computation on arrays and tables. For authoring and persistence examples, see Menudet: portable computation. To replace the retired alpha decorator, see Migrating to Menudet.

Menudet 0.1 is beta. API and format changes remain possible and will come with migration guidance.

Key contracts¶

  • Author with @blosc2.menudet. It is distinct from the eager/tracing @jit convenience decorator; native JIT is an execution policy, not an authoring mode.

  • Decorated calls return lazy arrays; result[:] returns values and result.compute() materializes a Blosc2 array.

  • MenudetFunction knows how to specialize; MenudetKernel is already specialized. Decorated calls specialize automatically; explicit specialize() is for working with the typed kernel itself.

  • The Python body is never executed. Native failures propagate without replay through Python or the legacy DSL.

  • Annotated input dtypes must match exactly. Scalar parameters require dtype annotations. Only supported scalar globals and closures can be captured.

  • Lazy persistence requires durable operand references and stores validated Menudet artifacts, not executable Python function recipes.

  • Historical Python-source recipes are unsupported, even with deserialize="full". Materialize them in a previous release before upgrading.

See Menudet syntax, Menudet numerical behavior, and Save and load for syntax, numerical rules, and deserialization policies. Menudet is not a general sandbox for arbitrary untrusted source or data.

Specifications¶

For the full language, format, and grammar contracts, see miniexpr’s canonical documentation:

  • Menudet introduction & portability model

  • Menudet canonical syntax reference

  • Menudet 0.1 language specification

  • Menudet 0.1 artifact specification

API¶

blosc2.menudet(func=None, *, jit=None, require_jit=False, partitions=None, cardinality='elementwise', ndim=0, casting='unsafe')[source]¶

Decorate a function as a lazy, typed Menudet function (automatic execution policy).

Use @menudet or @menudet(jit=False). Supported dtype annotations declare element types; unannotated NumPy/Blosc2 arrays supply their dtypes. Annotated inputs must match exactly, without implicit casts. Numeric return inference uses native metadata. Scalar arguments require dtype annotations. Native interpreter fallback is allowed unless require_jit=True; the original Python function is never executed, including after native errors. partitions and cardinality retain MenudetKernel.lazy block semantics.

class blosc2.MenudetFunction(func, *, jit=None, require_jit=False, partitions=None, cardinality='elementwise', ndim=0, casting='unsafe')[source]¶

Callable Menudet author; dtype specializations own Menudet native kernels.

Calls bind arrays and return a lazy result. Annotations specify element dtypes, not array shapes. Missing input annotations use array dtypes; missing output annotations use native numeric result inference, never Python execution. Scalar captures are snapshotted at decoration; array values are never cached.

Methods

__call__(*args, **kwargs)

Call self as a function.

specialize(*args, **kwargs)

Return the cached MenudetKernel for these arguments without evaluating them.

specialize(*args, **kwargs)[source]¶

Return the cached MenudetKernel for these arguments without evaluating them.

class blosc2.MenudetKernel[source]¶

An already specialized Menudet kernel, with no Python fallback.

Obtain a kernel with blosc2.MenudetFunction.specialize() and use evaluate to return NumPy values. Persist bound lazy computations with save and reopen them with blosc2.open(). Eager inputs are same-shaped logical arrays, bound by name.

Attributes:
context_ndim
entry_point
execution_kind

Native structural classification: expression, program, or unknown on older runtimes.

has_jit
inferred_dtype

Metadata-only result dtype before final conversion (Menudet 0.1 beta runtime required).

input_dtypes
native_partitionable

Native lane-independence proof: True if proved, False if unproved, None on older runtimes.

output_dtype
result_cardinality
schema_version
source

Methods

evaluate(inputs, *[, shape, return_status, ...])

Evaluate named arrays into a new NumPy array, preserving their common shape.

evaluate_array(inputs, *[, shape, ...])

Menudet 0.1 beta logical broadcasting/axis reductions with bounded iterator scratch.

evaluate_block(inputs, *[, block_shape, ...])

Evaluate one explicit Menudet 0.1 beta block, returning its declared cardinality.

evaluate_sources(inputs, *[, shape, ...])

Evaluate eligible lane-local kernels using bounded storage readers.

lazy(inputs, *[, shape, partitions])

Bind native inputs to an immutable logical block grid, not a storage grid.

evaluate(inputs, *, shape=None, return_status=False, fp_errors='ignore')[source]¶

Evaluate named arrays into a new NumPy array, preserving their common shape.

No implicit dtype conversion or array broadcasting is allowed. Host endian, alignment, and strides are normalized into temporary contiguous buffers. Constant-only kernels require an explicit output shape (possibly empty). Draft block-scalar kernels return a scalar NumPy array. ND kernels require evaluate_block with explicit logical context. Menudet 0.1 beta can return (values, status) with return_status=True. Status has stable flags bits (invalid=1, divide=2, overflow=4, underflow=8) and a supported capability. fp_errors='raise' raises for any of these flags; the default 'ignore' emits no warnings.

evaluate_array(inputs, *, shape=None, reduction=None, axis=None, keepdims=False, dtype=None, initial=None, where=None, tile_items=1024, return_report=False)[source]¶

Menudet 0.1 beta logical broadcasting/axis reductions with bounded iterator scratch.

Supports C/F, negative-stride and nonnative-endian numeric arrays. Results own C-order storage; no mutable-view or in-place guarantees. Grouping is serial logical C order, independent of tile size/storage chunks. This API is separate from explicit block-scalar evaluation.

evaluate_block(inputs, *, block_shape=None, logical_shape=None, block_origin=None, valid_mask=None, return_status=False, fp_errors='ignore', _validated=False)[source]¶

Evaluate one explicit Menudet 0.1 beta block, returning its declared cardinality.

This is standalone block execution, not lazy block-grid scheduling. Native validation owns coordinates, participating masks and result allocation. Floating status is per call, includes only evaluated active operations, and must be aggregated explicitly by callers across blocks. This is not full numpy.seterr emulation.

evaluate_sources(inputs, *, shape=None, selection=None, tile_items=65536, return_report=False)[source]¶

Evaluate eligible lane-local kernels using bounded storage readers.

NumPy buffers remain resident; compressed inputs are read one native tile at a time. Basic selections do not read discarded lanes. Block reductions, locals/loops, context-dependent kernels and final conversions reject before reads. Unsupported runtimes reject; numerical/storage failures never replay. Report scratch excludes output, Python tile metadata and codec internals.

property execution_kind¶

expression, program, or unknown on older runtimes.

Type:

Native structural classification

property inferred_dtype¶

Metadata-only result dtype before final conversion (Menudet 0.1 beta runtime required).

lazy(inputs, *, shape=None, partitions=None)[source]¶

Bind native inputs to an immutable logical block grid, not a storage grid.

property native_partitionable¶

True if proved, False if unproved, None on older runtimes.

Type:

Native lane-independence proof

class blosc2.MenudetError(message, *, status, native_status=0, line=0, column=0)[source]¶

Artifact failure with a stable category and optional native/source diagnostics.

Attributes:
args

Methods

add_note(object, /)

Exception.add_note(note) -- add a note to the exception

with_traceback(object, /)

Exception.with_traceback(tb) -- set self.__traceback__ to tb and return self.

blosc2.validate_menudet(source, input_dtypes, output_dtype, *, language_version='0.1', ndim=0, cardinality=None)[source]¶

Check raw native source and an exact typed signature against the Menudet 0.1 beta profile.

Return a dictionary with valid, status, line, column, and error. Validation neither executes a kernel nor invokes a JIT compiler. It does not normalize frontend syntax, inspect globals, or certify runtime input ranges, target compatibility, or sandbox safety. Native support is required; older builds report runtime_unsupported without Python fallback.

input_dtypes maps parameter names to explicit dtype descriptions. Parameter binding is by name; mapping order may differ from source order. Dtypes describe logical values, not the storage layout of any actual input arrays. The default language_version="0.1" is an beta specification admitting mixed numeric and fixed-width S/U signatures. ndim declares the logical rank for reserved ND symbols; missing or insufficient context rejects explicitly. cardinality may be elementwise or block_scalar to assert the native return contract; None validates the inferred contract. Neither arrays nor an artifact JSON loader are needed. Exact string output width is checked natively.

Next
Menudet numerical behavior
Previous
Menudet syntax
Copyright © 2019-present, The Blosc Developers
Made with Sphinx and @pradyunsg's Furo
On this page
  • Menudet API
    • Key contracts
    • Specifications
    • API
      • menudet()
      • MenudetFunction
        • MenudetFunction.specialize()
      • MenudetKernel
        • MenudetKernel.evaluate()
        • MenudetKernel.evaluate_array()
        • MenudetKernel.evaluate_block()
        • MenudetKernel.evaluate_sources()
        • MenudetKernel.execution_kind
        • MenudetKernel.inferred_dtype
        • MenudetKernel.lazy()
        • MenudetKernel.native_partitionable
      • MenudetError
      • validate_menudet()