Free-Threaded Python

Tensogram supports free-threaded Python (CPython 3.13t / 3.14t), which removes the Global Interpreter Lock (GIL) and allows true multi-threaded parallelism from Python.

What This Means

On standard CPython, the GIL serializes access to the interpreter — only one thread runs Python code at a time. Tensogram already releases the GIL during Rust computation (py.detach()), which helps, but the GIL is still re-acquired for numpy array construction and Python object creation.

On free-threaded CPython (3.13t / 3.14t), there is no GIL at all. Multiple threads can call tensogram.encode() and tensogram.decode() in true parallel. Use the included benchmark (rust/benchmarks/python/bench_threading.py) to measure scaling on your hardware.

Building for Free-Threaded Python

Install a free-threaded Python build:

# uv (recommended)
uv python install cpython-3.14+freethreaded

# Or via pyenv
pyenv install 3.14t

Build tensogram:

uv venv .venv --python python3.14t
source .venv/bin/activate
uv pip install maturin "numpy>=2.1"
cd python/bindings && maturin develop --release

Verify the GIL is disabled:

import sys
print(sys._is_gil_enabled())  # False

Thread-Safe API

All tensogram read operations are safe to call from multiple threads simultaneously:

import threading
import numpy as np
import tensogram

data = np.random.randn(1_000_000).astype(np.float32)
meta = {"base": [{}]}
desc = {"type": "ntensor", "shape": [1_000_000], "dtype": "float32"}
msg = tensogram.encode(meta, [(desc, data)])

def decode_worker():
    for _ in range(100):
        result = tensogram.decode(msg)

threads = [threading.Thread(target=decode_worker) for _ in range(8)]
for t in threads:
    t.start()
for t in threads:
    t.join()

Each thread can independently:

• Encode and decode messages
• Scan buffers
• Validate messages and files
• Read from TensogramFile instances (same handle or separate handles)
• Use StreamingEncoder (separate instances per thread)

TensogramFile Thread Safety

All read methods on TensogramFile (decode_message, read_message, decode_metadata, decode_descriptors, decode_object, decode_range, __getitem__, __len__, __iter__) use &self and support concurrent access from multiple threads on the same handle:

f = tensogram.TensogramFile.open("data.tgm")

def worker(thread_id):
    # Multiple threads can read from the same handle concurrently
    msg = f.decode_message(thread_id % len(f))

threads = [threading.Thread(target=worker, args=(i,)) for i in range(8)]
for t in threads:
    t.start()
for t in threads:
    t.join()

Only append() requires exclusive access — calling it while other threads are reading will raise RuntimeError (PyO3 runtime borrow check).

Benchmark Results

Measured on Linux x86_64 (20 cores), NumPy 2.4.4, release build. Same-version paired comparisons to isolate the GIL effect.

All scaling below comes from Python-level threading (threading.Thread). Each call into Rust is single-threaded — there is no rayon or internal parallelism within a single encode/decode. The speedups reflect multiple Python threads entering Rust concurrently via py.detach(). A future Rust-level parallel pipeline would multiply on top of these numbers.

Headline: Decode Throughput (1M float32, no codec)

Headline: Encode Throughput (1M float32, no codec)

Small Messages (16K float32, no codec)

Other Operations (1M float32)

Scan (message boundary detection — ~0.2µs/call, GIL overhead dominates):

Validate (full message validation — CPU-bound, scales well on both):

Decode-range (sub-array extraction, 2x1K slices from 1M):

Iter-messages (3 messages, 100K f32 each):

Key Takeaways

Methodology: 5 runs per configuration, median reported. 200–500 warmup iterations for fast operations.

• Validate scales near-linearly on both GIL and free-threaded — 8.9x (GIL) and 10.5x (free-threaded) at 16 threads. This is the most CPU-bound operation and benefits fully from py.detach() regardless of GIL.
• Free-threaded decode scales to 4.7x at 8 threads for the headline workload (1M f32, no codec). GIL-enabled stays near 1.0x because numpy array construction dominates and serializes under the GIL.
• GIL-enabled decode-range plateaus at ~1.7x — py.detach() allows 2 threads of overlap but the lightweight result construction can’t overlap further. Free-threaded reaches 11.8x at 16 threads.
• Scan shows dramatic free-threaded scaling — free-threaded reaches 15.5x at 16 threads. GIL-enabled scales to 2.0x at 4 threads but drops back at higher thread counts due to contention.
• Small messages (16K) reach 13.0x at 16 threads on free-threaded (3.14t) vs 1.2x on GIL-enabled.
• iter_messages scales to 4.7x at 8 threads on free-threaded, then drops due to contention. GIL-enabled stays flat (~1.0x).
• Single-thread trade-off — free-threaded single-thread performance varies by workload: decode is within ~5% of GIL-enabled (396 vs 408 op/s on 3.14), encode varies by version (3.14t is 18% faster than 3.14, while 3.13t is 6% slower than 3.13). Validate is ~20% slower (4,347 vs 5,457 op/s) and scan ~4x slower due to reference counting overhead on returned Python objects — both recover by 2 threads.

These numbers are machine-specific. Run the benchmark on your hardware:

python rust/benchmarks/python/bench_threading.py              # full suite
python rust/benchmarks/python/bench_threading.py --headline   # quick comparison
python rust/benchmarks/python/bench_threading.py --quick      # CI smoke test

Reference Comparison: Tensogram (Python) vs ecCodes (C)

This section measures Tensogram’s Python throughput against ecCodes’ native C performance on the same pipeline — 10 million float64 values (80 MiB), 24-bit simple packing + szip compression — as a concrete reference point. The pipeline is common in operational weather forecasting and is representative of scientific-quantisation workloads more broadly.

What we measured

Both sides are measured end-to-end: from a float64 array to serialized compressed bytes (encode), and back to a float64 array (decode). Both include metadata serialization, framing, and integrity overhead — not just the raw packing step.

ecCodes (C, single-threaded): The Rust benchmark (rust/benchmarks/src/bin/grib_comparison.rs) calls ecCodes’ C library directly via FFI. Encode: allocate a GRIB handle, configure the grid (10M regular lat/lon), set packing type to CCSDS at 24 bits, write the values array, serialize to GRIB bytes. Decode: load the GRIB message from bytes, extract the values array. No Python involved. Median of 10 iterations, 3 warmup.

Tensogram (Python, multi-threaded): The same 10M float64 values, same 24-bit quantization, same szip compression. Encode: pass a numpy array + CBOR metadata dict to tensogram.encode(), which crosses the PyO3 boundary, quantizes, compresses, frames, computes the integrity hash, and returns Python bytes. Decode: pass bytes to tensogram.decode(), which deframes, decompresses, dequantizes, and returns a numpy array. Each Python thread makes independent encode/decode calls. The GIL is released during the Rust computation.

Why scaling depends on the codec

Threading helps most when the Rust computation (compression, quantization) is the dominant cost. With simple packing + szip, each encode/decode spends ~170 ms in Rust and ~20 ms in Python/numpy — so ~89% of the time runs with the GIL released and threads scale well. Without compression, the Rust work is trivial (~1 ms) and the Python overhead limits parallelism.

The tables above measure uncompressed data to isolate the threading mechanism. The results below use the production pipeline (24-bit packing + szip) and show what real workloads achieve.

Results

ecCodes CCSDS (Rust FFI, single-threaded): 870 MB/s encode, 531 MB/s decode.

Tensogram from Python (free-threaded 3.14t, 5-run median, 10M float64 24-bit packing+szip):

Decode:

Encode:

Single-threaded Tensogram from Python is slower than ecCodes from C (the PyO3 boundary costs ~10-15% on decode, ~50% on encode due to numpy data extraction for 80 MiB). But at 2 threads, decode already surpasses ecCodes. At 4 threads, both encode and decode exceed ecCodes. At 8 threads, decode reaches 4.9x ecCodes throughput — from Python.

Requirements

• Python >= 3.13t for free-threaded mode (3.12/3.13 GIL-enabled also works)
• NumPy >= 2.1 (free-threaded support)
• maturin >= 1.8 (free-threaded wheel building)

Known Limitations

Inherent:

• Shared mutable numpy arrays across threads can cause data races (same as any Python threading)
• xarray and zarr backends have their own threading models (dask, zarr locking)

By design:

• TensogramFile read methods (decode_message, read_message, __getitem__, etc.) support concurrent access from multiple threads on the same handle. Only append() requires exclusive access.
• bytes inputs to decode/scan/validate are zero-copy across the GIL release. bytearray inputs are copied once internally by PyO3.
• iter_messages / PyBufferIter own a full buffer copy (the buffer must outlive iteration).