miniexpr DSL Syntax (Canonical Reference)

This is the practical reference for the DSL used by @blosc2.dsl_kernel functions (and checked by blosc2.validate_dsl()). It focuses on what works today and the most common gotchas. For usage walkthroughs and end-to-end examples, see the LazyArray UDF DSL kernels tutorial.

Quick start

A valid DSL program is one function:

def kernel(x, y):
    temp = sin(x) ** 2
    return temp + cos(y) ** 2

Use Python-style indentation and always return a value on the paths you execute.

@blosc2.jit auto-detects this DSL: a decorated function whose body contains an if/for/while and that compiles under this grammar is dispatched here automatically, so its branches and loops actually run, once per chunk, instead of jit’s normal approach of calling the function only once to record a single expression — which would otherwise capture just whichever branch that one call happened to take, silently dropping the rest. @blosc2.dsl_kernel remains the explicit form — it always requires the DSL to compile, equivalent to jit(strict=True).

Program shape

  • Exactly one top-level def ...: function is expected.

  • Leading blank lines and header comments are allowed.

  • Any extra trailing content after the function is a parse error.

  • Nested def inside the function body is not allowed.

  • The first statement may be a docstring: single/double-quoted or triple-quoted, including multiline strings and r/u prefixes. It is ignored during execution; @blosc2.dsl_kernel preserves the Python function’s __doc__.

Header pragmas

Supported file-header pragmas:

  • # me:fp=strict|contract|fast

  • # me:compiler=tcc|cc

Notes:

  • Pragma keys must be unique.

  • Unknown me:* pragmas are errors.

  • Malformed pragma values are errors.

Function signature and inputs

  • Parameters are positional names: def kernel(a, b, c): ...

  • Parameter names must be unique.

  • At compile time, DSL parameter names must match input variable names by set membership (order may differ, count must match).

Statements

Supported statement forms:

  • Assignment: a = expr

  • Compound assignment: +=, -=, *=, /=, //=

  • Expression statement: expr

  • Return: return expr

  • Print: print(...)

  • Conditionals: if / elif / else

  • While loop: while cond:

  • For loop: for i in range(...):

  • Loop control: break, continue

  • No-op: pass (also valid as the sole statement in a branch or loop body)

General rules:

  • Python-style indentation is required.

  • Empty blocks are invalid.

  • elif/else must belong to a matching if.

  • Deprecated forms like break if cond / continue if cond are not part of DSL syntax.

  • Simple statements may share a line, separated by ;, including inside indented blocks. A trailing ; is allowed; empty statements (;;) are not.

  • Compound statements (if, for, while) require their own lines and indented bodies; they cannot follow a semicolon. Inline suites such as if x: return x are not supported.

  • Expressions and calls can continue across lines inside parentheses, including comments. This does not add list literals or indexing to the expression grammar.

Docstrings and semicolon-separated statements

@blosc2.dsl_kernel
def kernel(x):
    """Transform each element.

    Documentation is not evaluated by the interpreter or JIT.
    """
    y = x + 1; z = y * y  # fmt: skip
    if z > 4:
        z -= 2; z *= 3  # fmt: skip
    return z

if / elif / else example

def kernel(x):
    if x > 0:
        y = x
    elif x == 0:
        y = 1
    else:
        y = -x
    return y

for example

def kernel(n):
    acc = 0
    for i in range(0, n, 1):
        acc += i
    return acc

while example

def kernel(x):
    i = 0
    y = x
    while i < 3:
        y = y * 2
        i += 1
    return y

Expressions and function calls

Expressions are compiled by miniexpr with DSL checks.

Numeric literals

Python-style digit separators are accepted in integers and floating-point literals: 1_000, 1_000.2_5, .1_25, and 1e1_0. Integer literals also accept binary (0b1010), octal (0o755), and hexadecimal (0xff) prefixes, including uppercase prefixes and separators such as 0x_FF or 0b10_10.

Malformed separators, invalid base digits, and nonzero decimal integers with leading zeros are rejected. Prefixed integer magnitudes must fit in an unsigned 64-bit value; numeric evaluation retains the existing dtype and precision rules, not Python’s arbitrary-precision integer arithmetic. Strings and identifiers (such as x_1) are not modified.

Commonly supported:

  • Names and numeric constants

  • Unary operators: +, -, logical not (not / !)

  • Arithmetic and bitwise binary operators

  • Comparisons: ==, !=, <, <=, >, >=

  • Function calls to supported miniexpr functions

  • User-registered C functions/closures passed in me_variable

DSL kernels accept chained comparisons such as 0 <= x < 10 or a < b <= c != d. Operands are evaluated left-to-right, each at most once; later operands are skipped as soon as a comparison fails. Chains work in expressions, assignments, branches, and loop conditions (including continue). The native miniexpr DSL front end lowers them to temporary variables and guarded statements for both the interpreter and TCC/CC JIT. C and other callers can pass raw chain syntax directly, without Python preprocessing. JavaScript emission performs equivalent lowering separately. Existing DSL and/or Boolean-result rules apply. This syntax extension applies to DSL kernels, not the separate classic expression API.

Chaining does not expand the supported operand types or functions: existing string-comparison and per-element reduction restrictions still apply.

Cast intrinsics:

  • int(expr)

  • float(expr)

  • bool(expr)

Cast rules:

  • Use function-call form only.

  • Exactly one argument.

Temporary variable type inference

Local temporaries get their dtype from the expression assigned to them.

Example:

def kernel(x):
    temp = sin(x) ** 2
    return temp + cos(x) ** 2

In this example, temp is inferred from sin(x) ** 2 (typically a floating type).

Notes:

  • You do not need to declare local variable types.

  • Boolean output does not force numeric temporaries to Boolean: operand types are inferred independently, and the return value is converted to Boolean.

  • If you assign a value with an incompatible dtype to the same local later, compilation fails.

Loops

for ... in range(...)

Supported forms:

for i in range(stop):
    ...
for i in range(start, stop):
    ...
for i in range(start, stop, step):
    ...

Rules:

  • range takes 1, 2, or 3 arguments.

  • step == 0 raises a runtime evaluation error.

while

  • while condition is a regular DSL expression.

  • Runtime iteration cap is enforced by ME_DSL_WHILE_MAX_ITERS.

Reductions inside control flow

sum, max, min and other block reductions collapse the whole chunk being evaluated to one value, not one value per element. Using one as the condition of if/while, or assigning one to a local that per-element code later reads (e.g. y = max(x) followed by if x > 0: y = y + 1), does not raise a compile-time or runtime error – it compiles and runs, and produces results that are only correct for element 0 of the block; every other element sees a stale/zero value where the reduction result should be. This is a rough edge in the underlying miniexpr compiler, not something this Python layer validates today. Write the per-element form instead (drop the reduction, e.g. if abs(diff) < tol rather than if max(abs(diff)) < tol) whenever the intent is a per-element, not whole-block, decision.

print(...)

print is supported as a DSL statement.

Rules:

  • At least one argument is required.

  • First argument may be a format string.

  • Placeholder count must match provided values.

  • Printed expressions must be uniform/scalar for the block.

Reserved names

Do not use these as user variable/function names in DSL:

  • print, int, float, bool, def, return

  • _ndim

  • _i<d> and _n<d> (reserved ND symbols)

  • _flat_idx

ND reserved symbols

When referenced, these are synthesized by DSL compiler/runtime:

  • _i0, _i1, … (index per dimension)

  • _n0, _n1, … (shape per dimension)

  • _ndim

  • _flat_idx (global C-order linear index)

Typing and return behavior

  • Reassigning incompatible dtypes to the same local is a compile-time error.

  • Return dtype must be consistent across all return statements.

  • Non-guaranteed return paths may compile; if execution reaches a missing return path, evaluation fails at runtime.

Compute dtype and integer exactness

The kernel’s output dtype generally determines the arithmetic compute dtype:

  • With an integer output dtype, arithmetic is exact int64. Intermediates must fit in int64: products at or above 2^63 overflow and give wrong results.

  • With a float output dtype, integer arithmetic is generally evaluated in float64, where integer operations are exact only below 2^53. Keep products under that bound (e.g. a 32-bit value times a multiplier below 2^21); larger products silently lose low bits.

  • Native chained-comparison captures retain each operand’s inferred dtype, so comparisons of int64 operands can remain exact even with a float output. Automatic JavaScript dispatch therefore leaves 64-bit integer input arrays on miniexpr; the JavaScript bridge converts inputs to float64.

  • Values outside the output dtype’s range wrap two’s-complement on the final store (e.g. returning a value in [0, 2^32) into an int32 output yields the full [-2^31, 2^31) range).

Compound assignment desugaring

  • a += b -> a = a + b

  • a -= b -> a = a - b

  • a *= b -> a = a * b

  • a /= b -> a = a / b

  • a //= b -> a = floor(a / b)

Compile-time vs runtime errors

Compile-time error examples:

  • Invalid program shape or signature

  • Unsupported statement forms

  • Invalid range(...) arity

  • Invalid cast intrinsic arity

  • Reserved-name misuse

  • Return dtype mismatch

Runtime error examples:

  • range(..., step=0)

  • Missing return on executed control path

  • While-loop iteration cap exceeded

Execution backends

A DSL kernel can use the following execution backends. Set jit / jit_backend on lazyudf() or compute(); indexing uses the configured/default settings. See the JIT options reference for defaults, environment precedence, caching, and diagnostics:

  • miniexpr (default on native builds): a runtime JIT (TinyCC, jit_backend="tcc") with an interpreter fallback. jit=False skips JIT; jit=True is best effort and also falls back if executable allocation or compilation is denied. Supports the full DSL described here, including integer/complex dtypes and reductions.

  • System C compiler (jit_backend="cc"): miniexpr generates optimized shared libraries using an installed compiler and caches them for subsequent processes. Compilation or loading failures also fall back to the interpreter. TCC instead compiles in memory without creating JIT cache artifacts.

  • JavaScript (jit_backend="js"): transpiles the kernel to JavaScript and runs it through the browser’s JIT. WebAssembly/Pyodide only — requesting it elsewhere raises. Under WebAssembly it is also the default for eligible kernels (set jit=False or strict_miniexpr=True to opt out), and silently falls back to miniexpr for anything it cannot handle.

The JavaScript backend computes in float64 and covers floating-point element-wise kernels: arithmetic, comparisons, where, if/elif/else, for ... in range(...)/while loops, the index/shape symbols (_i0/_n0/_ndim/_flat_idx), and the standard math functions. It also accepts integer inputs when the output dtype is floating. It does not support integer/complex output, reductions, or constructs outside the transpiled subset; those stay on miniexpr (or, with an explicit jit_backend="js", raise).

Python syntax that is out of DSL scope

These Python features are not part of this DSL:

  • Ternary expression: a if cond else b

  • for ... else and while ... else

  • Keyword-argument calls and other call forms outside the supported subset

  • Docstrings (or any other bare string-literal statement) inside the kernel body – this is a compile-time parse error at the miniexpr level, not a silently-ignored statement.