Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
Python-Blosc2 documentation
Logo
Python-Blosc2 documentation
  • Getting Started
    • Overview
    • Installation
  • Tutorials
    • NDArray: A NDim, Compressed Data Container
    • Expressions containing NDArray objects
    • User Defined Functions
    • Lazy array kernels with Menudet
    • Optimizing data reductions with NDArrays
    • Advanced Lazy Expressions and Persistent Reductions
    • Using Proxies for Efficient Handling of Remote Multidimensional Data
    • Compressing data with the SChunk class
    • Slicing, extending and serializing with SChunks
    • User-defined codecs and filters
    • Prefilters, postfilters and fillers
    • Working with Containers
    • Working with ObjectArray
    • Working with BatchArray
    • CTable Tutorial
    • Indexing Arrays
    • Indexing CTables
  • Guides
    • Performance & Benchmarks
    • Optimization tips
    • Menudet: portable computation
    • Migrating to Menudet
    • Working with Remote Data
    • Working with Remote Arrays
    • Working with Remote Tables
    • Sharing Containers Across Processes
    • Using Blosc2 with pandas
    • b2view: Browse TreeStore Bundles in the Terminal
    • Parquet to Blosc2 Walkthrough
  • API Reference
    • Blosc2 Classes
      • NDArray
      • C2Array
      • LazyArray
      • BatchArray
      • ListArray
      • ObjectArray
      • Proxy
      • RemoteObject
      • RemoteArray
      • RemoteStore
      • RemoteRepository
      • RemoteFile
      • RemoteCTable
      • ProxySource
      • ProxyNDSource
      • ByteRangeNDSource
      • FsspecNDSource
      • ZarrNDSource
      • HDF5NDSource
      • B2ZNDSource
      • SimpleProxy
      • Traffic
      • EmbedStore
      • DictStore
      • TreeStore
      • CTable
      • Column
      • NestedColumn
      • Schema Specs
      • Index
      • SChunk
      • Array
      • NDField
      • Ref
    • Operations with arrays
      • Universal Functions (ufuncs)
      • Reduction Functions
      • Linear Algebra
      • Random Functions
      • Additional Functions and Type Utilities
      • Indexing and Manipulation Functions and Utilities
    • Miscellaneous
    • Compression Utilities
      • blosc2.compress
      • blosc2.compress2
      • blosc2.decompress
      • blosc2.decompress2
      • blosc2.pack
      • blosc2.pack_array
      • blosc2.pack_array2
      • blosc2.pack_tensor
      • blosc2.unpack
      • blosc2.unpack_array
      • blosc2.unpack_array2
      • blosc2.unpack_tensor
      • blosc2.clib_info
      • blosc2.compressor_list
      • blosc2.detect_number_of_cores
      • blosc2.free_resources
      • blosc2.get_clib
      • blosc2.nthreads
      • blosc2.print_versions
      • blosc2.register_codec
      • blosc2.register_filter
      • blosc2.set_blocksize
      • blosc2.set_nthreads
      • blosc2.set_prefetch
      • blosc2.set_releasegil
      • blosc2.set_compressor
      • blosc2.get_compressor
      • blosc2.get_blocksize
      • blosc2.get_cbuffer_sizes
      • blosc2.cparams_dflts
      • blosc2.dparams_dflts
      • blosc2.storage_dflts
      • blosc2.Codec
      • blosc2.Filter
      • blosc2.SpecialValue
      • blosc2.SplitMode
      • blosc2.Tuner
      • blosc2.compute_chunks_blocks
      • blosc2.get_slice_nchunks
      • blosc2.remove_urlpath
      • blosc2.blosclib_version
      • blosc2.DEFINED_CODECS_STOP
      • blosc2.GLOBAL_REGISTERED_CODECS_STOP
      • blosc2.USER_REGISTERED_CODECS_STOP
      • blosc2.EXTENDED_HEADER_LENGTH
      • blosc2.MAX_BUFFERSIZE
      • blosc2.MAX_BLOCKSIZE
      • blosc2.MAX_OVERHEAD
      • blosc2.MAX_TYPESIZE
      • blosc2.MIN_HEADER_LENGTH
      • blosc2.prefilter_funcs
      • blosc2.postfilter_funcs
      • blosc2.ucodecs_registry
      • blosc2.ufilters_registry
      • blosc2.VERSION_DATE
      • blosc2.VERSION_STRING
      • blosc2.__version__
    • Compression, decompression and storage parameters
    • Save and load
    • Expression Utilities
    • Menudet syntax
    • Menudet API
    • Menudet numerical behavior
    • JIT execution options
  • Glossary
  • Development
    • Contributing guidelines
    • Code of Conduct
    • List of desired features for a 4.0 release
    • Remote cache design
    • Menudet numerical qualification
    • Typed expression evaluator rollout
  • Release Notes
Back to top

Menudet syntax¶

Use @blosc2.menudet for portable array kernels. See the guide for usage and the API reference for options.

Canonical specification: The single source of truth for the Menudet language grammar, statements, pragmas, and operator semantics is maintained in miniexpr’s Menudet syntax reference. This document highlights the Python-Blosc2 authoring bindings and conventions.

The old DSL interface is retired; see the migration guide.

Quick start¶

Decorate a function, call it with arrays, and read the lazy result with [:]:

import blosc2
import numpy as np


@blosc2.menudet
def kernel(x, y):
    temp = np.sin(x) ** 2
    return temp + np.cos(y) ** 2


x = np.array([0.0, 0.5, 1.0])
values = kernel(x, x)[:]
np.testing.assert_allclose(values, [1.0, 1.0, 1.0])

Calls specialize and validate automatically before execution. [:] returns NumPy values; .compute() materializes a Blosc2 array. The Python body is never executed as a fallback.

Captures and persistence¶

Supported scalar globals and closure values are snapshotted at decoration:

SCALE = 2.0


@blosc2.menudet
def scaled(x):
    return x * SCALE


SCALE = 99.0  # Does not change the captured value.
values = scaled(np.arange(4, dtype="float64"))[:]
# array([0., 2., 4., 6.])

NumPy scalar captures retain their dtype; use np.float32(2) when that dtype matters. Arrays, arbitrary objects and callbacks cannot be captured. Pass arrays as arguments instead.

Save a bound lazy computation using persistent operands:

stored = blosc2.asarray(np.arange(4, dtype="float64"), urlpath="input.b2nd", mode="w")
recipe = scaled(stored)
recipe.save("scaled.b2nd")
reopened = blosc2.open("scaled.b2nd")
values = reopened[:]

Saved computations preserve their captures, operand references and logical partitions. Loading does not reconstruct a Python function.

Program shape¶

  • Write one function using Python indentation. A leading docstring is allowed.

  • Use named positional parameters; calls can supply keywords and defaults. *args, **kwargs and keyword-only parameters are not supported.

  • Array inputs supply their element dtypes and can broadcast. Annotations declare element dtypes, not shapes, and annotated inputs must match exactly.

  • Scalar parameters and nonnumeric returns require dtype annotations.

  • Return a value on every executed path. Local variables get their types from their expressions; no separate declarations are needed.

Statements and control flow¶

Supported statements include assignments, compound assignments, return, if/elif/else, for ... in range(...), while, break, continue and pass.

@blosc2.menudet
def adjusted(x):
    if x < 0:
        y = -x
    else:
        y = x
    for i in range(3):
        y = y * 0.5
    return y

Conditions and loop control can differ between array elements. range accepts one, two or three arguments; a zero step raises an error. while loops have an iteration limit. Unsupported constructs and runtime errors raise rather than falling back to Python.

Expressions and function calls¶

  • Arithmetic, comparisons, Boolean operations (and, or, not) and supported integer bitwise operations are available.

  • Chained comparisons such as 0 <= x < 10 short-circuit left to right.

  • Recognized mathematical calls include np.sin(x), np.sqrt(x) and their native spellings. This does not make arbitrary NumPy functions available.

  • np.where(condition, yes, no) (or bare where(...)) evaluates only the selected branch for each element. Prefer np.where(...) to avoid linter warnings and avoid conflicts with from blosc2 import where in module globals.

  • int(x), float(x) and bool(x) perform explicit conversions; fixed-width conversions such as int32(x) are also available.

  • Numeric literals support digit separators and binary, octal and hexadecimal integer prefixes. Integer arithmetic is fixed-width, not arbitrary precision.

Menudet supports Boolean, standard integer, float32/float64, complex and fixed-width bytes/Unicode types. Intermediate types depend on the operands; requesting a wider output does not widen intermediate arithmetic. See numerical behavior for overflow, conversion and accuracy rules.

Block reductions¶

Use block_sum, block_prod, block_min, block_max, block_mean, block_any and block_all to reduce elements within a logical partition. Each takes one argument, without axis or keepdims.

These operations do not combine different partitions. Use ordinary lazy expressions for whole-array or axis reductions. See the guide for partitions and cardinality examples.

Array coordinates¶

Reserved symbols expose coordinates within the logical array:

Symbol

Meaning

_i0, _i1, …

Coordinate along each dimension

_n0, _n1, …

Size of each dimension

_ndim

Number of dimensions

_flat_idx

C-order linear index

Set ndim on the decorator when using these symbols. Do not use them for your own variables. Slicing a lazy result retains its original coordinates and reduction groups.

Execution backends¶

The default policy chooses native interpretation or JIT. Use jit=False for interpretation, jit=True to prefer JIT, or require_jit=True to require it. Unsupported JIT features can use the interpreter before execution; numerical errors are never retried through another backend. Complex kernels currently use the interpreter only. See JIT options for advanced controls.

Unsupported Python features¶

Menudet is a restricted language, not arbitrary Python. Unsupported features include callbacks, side effects such as print, nested function definitions, arbitrary dynamic indexing, ternary expressions, and for/while ... else.

Optional native source validation¶

Decorated calls validate automatically. Use validate_menudet() only when you want to check raw native source without executing it:

source = "def affine(x):\n    return x * 2.0 + 1.0\n"
info = blosc2.validate_menudet(source, {"x": "float64"}, "float64")
if not info["valid"]:
    print(info["status"], info["line"], info["column"], info["error"])

Raw source validation does not resolve Python globals or rewrite NumPy calls; use native spellings such as sin(x). A successful check cannot guarantee that all runtime data avoids numerical errors. Menudet is not a sandbox for arbitrary untrusted source or data.

Further reading¶

  • Canonical Menudet syntax reference: complete grammar, statements, pragmas, and expressions.

  • Menudet introduction & portability model: language goals, 4 dimensions of portability, and crash-proof safety.

  • Menudet user guide: authoring, persistence and table integration.

  • API reference: decorators, kernels and validation.

  • Numerical behavior: typing, conversions and accuracy.

  • Migration guide: replacing legacy DSL code and files.

Next
Menudet API
Previous
Expression Utilities
Copyright © 2019-present, The Blosc Developers
Made with Sphinx and @pradyunsg's Furo
On this page
  • Menudet syntax
    • Quick start
    • Captures and persistence
    • Program shape
    • Statements and control flow
    • Expressions and function calls
    • Block reductions
    • Array coordinates
    • Execution backends
    • Unsupported Python features
    • Optional native source validation
    • Further reading