Benchmarks

Tensogram ships with a benchmark suite that measures all encoding and compression combinations on synthetic data. It produces tabular comparisons of speed, compressed size, and decode fidelity. The benchmarks can be re-run at any time to measure the effect of changes.

Codec Matrix Benchmark

Tests all valid encoder × compressor × bit-width combinations on 16 million synthetic float64 values.

Quick start

cargo run --release -p tensogram-benchmarks --bin codec-matrix

Override parameters with CLI flags:

cargo run --release -p tensogram-benchmarks --bin codec-matrix -- \
    --num-points 16000000 \
    --iterations 10 \
    --warmup 3 \
    --seed 42

Combinations measured

For actual results, see Benchmark Results.

How to read the results

The results page splits each benchmark into a performance table (timing, throughput, compressed size) and a fidelity table (error norms for lossy codecs).

For lossless codecs all three error norms are zero. Errors are absolute, in the same units as the input data.

Quick rules of thumb:

• If you need exact data back, use one of the lossless codecs.
• If you can tolerate some loss, compare Ratio vs error norms for your use case.
• Throughput (MB/s) is the most useful speed metric — it accounts for data size and lets you compare across different payload sizes.

Reference Comparison: ecCodes GRIB Encoding

Scientific codecs are easiest to understand alongside an established reference. ecCodes is a widely-deployed GRIB encoder used throughout operational weather forecasting. This benchmark compares Tensogram’s 24-bit SimplePacking + szip pipeline against ecCodes’ built-in packing methods on 10 million float64 values. Both sides are timed symmetrically: encoding measures the full path from a float64 array to compressed bytes, and decoding measures the reverse.

Requirements

• ecCodes C library installed (brew install eccodes on macOS, apt install libeccodes-dev on Debian/Ubuntu)
• Build with --features eccodes

Quick start

cargo run --release -p tensogram-benchmarks --bin grib-comparison --features eccodes

cargo run --release -p tensogram-benchmarks --bin grib-comparison --features eccodes -- \
    --num-points 10000000 \
    --iterations 10 \
    --warmup 3 \
    --seed 42

Methods compared

For actual results, see Benchmark Results.

Benchmark pipeline flow

flowchart TD
    G[Generate synthetic field] --> W[Warm-up iterations]
    W --> T[Timed iterations]
    T --> E[Encode]
    E --> D[Decode]
    D --> T
    T --> F[Fidelity check]
    F --> R[Print report]

    style G fill:#388e3c,stroke:#2e7d32,color:#fff
    style T fill:#1565c0,stroke:#0d47a1,color:#fff
    style F fill:#c62828,stroke:#b71c1c,color:#fff

Each timed iteration runs a full encode → decode cycle. After all iterations complete, the last decoded output is compared against the original to produce the fidelity metrics.

Things to know

Compression expansion

Some compressors (especially LZ4 on raw 64-bit floats) may produce output larger than the input (Ratio > 100%). This is normal — high-entropy data can’t always be compressed. The baseline row is a raw copy and always shows 100%.

Szip alignment

The codec matrix may round num_points up by 1–3 values for szip block alignment. This only matters for very small inputs.

Small data sizes

With --num-points 1, timing is dominated by per-call overhead rather than compression throughput. Use ≥ 10 000 points for meaningful comparisons.

GRIB grid shape

For prime num_points, the GRIB benchmark creates a 1 × N grid (not a realistic near-square grid). Use composite sizes for representative results (e.g. --num-points 10000000).

Reproducibility

The data generator is deterministic for a given --seed, so repeated runs on the same machine produce comparable timing. Compression ratios, sizes, and fidelity are reproducible across machines. Timing and throughput are not.

Error handling

If a single codec fails, the benchmark logs the error and continues with the remaining combinations. The summary line reports how many succeeded and failed. The CLI exits with code 1 if any combination failed.

Running in CI

For fast CI validation, pass --num-points 10000 --iterations 1 --warmup 1:

cargo run -p tensogram-benchmarks --bin codec-matrix -- \
    --num-points 10000 --iterations 1 --warmup 1

The smoke test suite (cargo test -p tensogram-benchmarks) uses 500–1000 points and completes in under 5 seconds.