C API

Tensogram exposes a flat C ABI through the tensogram-ffi crate. The generated header is tensogram.h; all public functions are prefixed tgm_, most public types follow the tgm_*_t pattern (the PODs TgmEncodeMaskOptions, TgmDecodeMaskOptions, TgmEncodeOptions, TgmScanOptions, TgmRemoteScanOptions, TgmFrame, and TgmMessageHeader are PascalCase exceptions inherited from the underlying Rust types), and all error codes are members of the tgm_error enum.

The C++ wrapper at cpp/include/tensogram.hpp is built directly on top of this C API; see C++ API for the higher-level RAII-and-exceptions front-end. This page covers the C side: how to get the library, how to link it, and what to be aware of regarding versioning.

Three install paths

Prebuilt binary tarball (recommended for users)

Each tagged release publishes two tarballs at https://github.com/ecmwf/tensogram/releases:

• tensogram-ffi-<VERSION>-linux-x86_64.tar.gz
• tensogram-ffi-<VERSION>-macos-aarch64.tar.gz

The Linux tarball is built on AlmaLinux 8 (glibc 2.28; ABI-compatible with the manylinux_2_28 wheel platform tag, works on RHEL 8 / Debian 11 / Ubuntu 20.04 and newer). The macOS tarball is built on Apple Silicon. For other platforms (linux-aarch64, macos-x86_64, etc.) build from source with cargo cinstall below.

Each tarball is rooted for /usr/local (the bundled tensogram.pc hard-codes prefix=/usr/local), and is packed with uid=0 / gid=0 so extraction under sudo produces root-owned files. The default install is:

VERSION=<release-version>          # e.g. 0.20.0
PLATFORM=linux-x86_64              # or macos-aarch64
ASSET=tensogram-ffi-${VERSION}-${PLATFORM}.tar.gz

curl -LO "https://github.com/ecmwf/tensogram/releases/download/${VERSION}/${ASSET}"
sudo tar --no-same-owner -C /usr/local -xzf "${ASSET}"
sudo ldconfig                      # Linux: refresh dynamic linker cache
pkg-config --modversion tensogram  # → ${VERSION}

After extraction the layout under /usr/local is:

/usr/local/
├── lib/
│   ├── libtensogram.so.0.20.0          (Linux: real shared library)
│   ├── libtensogram.so.0.20            → libtensogram.so.0.20.0
│   ├── libtensogram.so                 → libtensogram.so.0.20.0
│   ├── libtensogram.0.20.0.dylib       (macOS: real dylib)
│   ├── libtensogram.0.20.dylib         → libtensogram.0.20.0.dylib
│   ├── libtensogram.dylib              → libtensogram.0.20.0.dylib
│   ├── libtensogram.a
│   └── pkgconfig/tensogram.pc
└── include/
    └── tensogram/
        └── tensogram.h

Plus LICENSE, README.md, and INSTALL.md at the install root.

For a non-default prefix, use cargo cinstall (next section) — the .pc file embeds the prefix at build time, so simply extracting the tarball under a different directory will leave pkg-config returning broken paths.

cargo cinstall (recommended for custom prefixes)

cargo-c is a cargo subcommand that builds and installs the C-callable artefacts in one step. Unlike plain cargo build, it produces a properly versioned shared library with SONAME symlinks, a pkg-config .pc file, and an installed header at the right path.

# One-time: install cargo-c (uses your default stable toolchain).
cargo install cargo-c

# Build + install Tensogram FFI under any prefix you control.
# --libdir=lib pins the layout (otherwise Debian/Ubuntu multiarch
# would pick lib/<triplet>, breaking PKG_CONFIG_PATH guesses below).
cargo cinstall --release -p tensogram-ffi \
    --prefix="$HOME/.local" --libdir=lib

# Verify pkg-config can see it.
export PKG_CONFIG_PATH="$HOME/.local/lib/pkgconfig:$PKG_CONFIG_PATH"
pkg-config --modversion tensogram

The cargo-c metadata is committed in rust/tensogram-ffi/Cargo.toml, so no extra configuration is needed.

Build from source

Plain cargo build continues to work for in-tree development: it produces target/release/libtensogram_ffi.{a,so,dylib} (note the _ffi suffix). The C++ wrapper’s CMake integration uses this path.

cargo build --release -p tensogram-ffi
# Outputs:
#   target/release/libtensogram_ffi.a
#   target/release/libtensogram_ffi.so   (or .dylib on macOS)
#   rust/tensogram-ffi/tensogram.h       (regenerated by build.rs cbindgen)

There is no SONAME, no pkg-config file, and no header at a system include path — those are cargo-c’s job. cargo build is the contributor flow; cargo cinstall is the user flow.

Linking against the installed library

pkg-config (recommended)

cc $(pkg-config --cflags tensogram) my_program.c \
   $(pkg-config --libs tensogram) \
   -o my_program

--cflags returns -I<includedir> so your code uses #include <tensogram/tensogram.h>. --libs returns -L<libdir> -ltensogram.

CMake

cmake_minimum_required(VERSION 3.16)
project(my_project C)

find_package(PkgConfig REQUIRED)
pkg_check_modules(TENSOGRAM REQUIRED IMPORTED_TARGET tensogram)

add_executable(my_program my_program.c)
target_link_libraries(my_program PRIVATE PkgConfig::TENSOGRAM)

Manual flags (when pkg-config is not available)

# Linux
cc -I/usr/local/include my_program.c \
   -L/usr/local/lib -ltensogram \
   -ldl -lpthread -lm \
   -o my_program

# macOS
cc -I/usr/local/include my_program.c \
   -L/usr/local/lib -ltensogram \
   -framework CoreFoundation -framework Security -framework SystemConfiguration \
   -lc++ -lm \
   -o my_program

The Libs.private field of tensogram.pc lists the platform-specific support libraries; pkg-config picks them up automatically when linking the static archive (pkg-config --static --libs tensogram).

Quick start

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#include <tensogram/tensogram.h>

int main(void) {
    const char *meta_json =
        "{\"descriptors\":[{"
        "\"type\":\"ntensor\",\"ndim\":1,\"shape\":[4],\"strides\":[4],"
        "\"dtype\":\"float32\",\"byte_order\":\"little\","
        "\"encoding\":\"none\",\"filter\":\"none\",\"compression\":\"none\""
        "}]}";

    float in[4] = { 1.0f, 2.0f, 3.0f, 4.0f };
    const uint8_t *ptrs[1] = { (const uint8_t*)in };
    const size_t lens[1] = { sizeof(in) };

    tgm_bytes_t enc = {0};
    if (tgm_encode(meta_json, ptrs, lens, 1, "xxh3", 0, &enc) != TGM_ERROR_OK) {
        fprintf(stderr, "encode failed: %s\n", tgm_last_error());
        return 1;
    }

    tgm_message_t *msg = NULL;
    if (tgm_decode(enc.data, enc.len,
                   /*native_byte_order=*/1, /*threads=*/0,
                   /*verify_hash=*/0, &msg) != TGM_ERROR_OK) {
        fprintf(stderr, "decode failed: %s\n", tgm_last_error());
        tgm_bytes_free(enc);
        return 1;
    }

    size_t out_len = 0;
    const uint8_t *out_data = tgm_object_data(msg, 0, &out_len);
    printf("decoded %zu bytes, equal=%s\n", out_len,
           (out_len == sizeof(in) && memcmp(out_data, in, out_len) == 0) ? "yes" : "no");

    tgm_message_free(msg);
    tgm_bytes_free(enc);
    return 0;
}

Synchronous remote access (opt-in remote feature)

A remote .tgm (S3, GCS, Azure, HTTP) can be read through the ordinary blocking file API — no async ABI required:

const char *keys[]   = { "aws_region", "aws_access_key_id" };
const char *values[] = { "eu-west-1",  "AKIA..." };
TgmRemoteScanOptions opts = { .bidirectional = true };  /* NULL = defaults */

tgm_file_t *file = NULL;
tgm_error rc = tgm_file_open_remote("s3://bucket/forecast.tgm",
                                    keys, values, 2, &opts, &file);
if (rc != TGM_ERROR_OK) {
    fprintf(stderr, "open_remote failed: %s\n", tgm_last_error());
    return 1;
}

/* `file` is a PLAIN tgm_file_t — every existing file function works. */
size_t n = 0;
tgm_file_message_count(file, &n);
tgm_file_close(file);

keys / values are parallel arrays of n_options backend storage options (credentials, region, endpoint, …) forwarded verbatim to the object-store backend; pass 0 / NULL for none. tgm_is_remote_url tells the two backends apart before you open anything — the recognised schemes are s3, s3a, gs, az, azure, http, https (compared case-insensitively). Plain paths and file:// URLs are not remote; they belong to tgm_file_open.

This is an opt-in Cargo feature and is NOT in the published tarballs. The prebuilt tensogram-ffi-<VERSION>-<PLATFORM>.tar.gz assets are built with default features, exactly like the existing async surface. To get a library that can actually open a remote source, build it yourself:

cargo cinstall --release -p tensogram-ffi --features=remote \
    --prefix="$HOME/.local" --libdir=lib
# in-tree / C++ wrapper builds:
cargo build --release -p tensogram-ffi --features=remote
cmake -S cpp -B build -DTENSOGRAM_REMOTE=ON     # default OFF

Both symbols are exported either way, so you never hit an undefined symbol at link time. In a build without the feature:

• tgm_is_remote_url returns false for every input — such a build genuinely cannot open any URL, so “not remote for me” is the honest answer;
• tgm_file_open_remote returns TGM_ERROR_REMOTE with a tgm_last_error() message naming --features=remote.

Argument validation (NULL source / out, mismatched option arrays) runs before the feature check, so a genuine mistake is reported as TGM_ERROR_INVALID_ARG in both builds.

The remote feature is independent of async — neither implies the other. For the async equivalent see C++ Async API; for the transport behaviour (request budget, bidirectional scan, limitations) see Remote Access.

Frame introspection

tgm_message_header reads a message’s 24-byte envelope into a TgmMessageHeader POD (wire version, total_length, and eight has_* booleans), and tgm_frame_iter_create / _next / _free walk that message’s frames, filling a caller-provided TgmFrame per step:

TgmMessageHeader h;
tgm_message_header(msg, msg_len, &h);   /* 24 bytes; walks nothing */

tgm_frame_iter_t *it = tgm_frame_iter_create(msg, msg_len);
TgmFrame f;
while (tgm_frame_iter_next(it, &f)) {
    /* f.frame_type is a tgm_frame_type; f.payload borrows msg */
    if (tgm_frame_has_hash(&f)) { /* the inline slot holds a digest */ }
}
const char *err = tgm_last_error();   /* NULL => clean end, else framing */
tgm_frame_iter_free(it);

f.payload / f.payload_len are the frame content (the 16-byte frame header and the type-specific footer are stripped) and point into msg: nothing to free, valid for as long as msg lives, and unaffected by later _next calls or by tgm_frame_iter_free. The iterator borrows msg for its whole lifetime, so msg must outlive it.

Both entry points take one message — use tgm_scan to locate the boundaries in a multi-message buffer and pass each slice. The full contract (frame types, offsets, payload boundaries, the streaming-flag nuance) is in Frame Introspection.

Typed enums and the full encode options

Four new enums make descriptor and option values switch-able instead of strcmp-able. Each one’s zero value is the library default, so a zero-initialised option struct asks for exactly the previous behaviour:

tgm_frame_type (above) follows the same pattern, except that its numbers really are wire values.

Two typed accessors join the existing string getters, which are unchanged and still supported:

switch (tgm_object_dtype_enum(msg, 0)) {          /* vs tgm_object_dtype  */
    case TGM_DTYPE_FLOAT32: /* 4 bytes per element */ break;
    default: break;
}
tgm_byte_order bo = tgm_object_byte_order_enum(msg, 0);

An enum return has no spare code for failure: a NULL handle or an out-of-range index records the reason in tgm_last_error() and returns the zero variant (TGM_DTYPE_FLOAT16, TGM_BYTE_ORDER_LITTLE). To bounds-check unambiguously, compare the index against tgm_message_num_objects, or call the paired string getter, which returns NULL for exactly the same inputs.

TgmEncodeOptions supersedes TgmEncodeMaskOptions (which stays for source compatibility): it carries the same six mask fields plus the three knobs that previously had no C surface at all — the hash algorithm, the aggregate-hash placement, and the codec backend:

TgmEncodeOptions opts = {0};          /* every zero IS the default */
opts.hash = "xxh3";                   /* NULL / omitted = no hashing */
opts.aggregate_hash = TGM_AGGREGATE_HASH_POLICY_BOTH;
opts.compression_backend = TGM_COMPRESSION_BACKEND_PURE;

tgm_bytes_t enc = {0};
tgm_encode_with_encode_options(meta_json, ptrs, lens, 1, 0, &opts, &enc);

There is one *_with_encode_options entry point per encode target — tgm_encode_with_encode_options, tgm_file_append_with_encode_options, and tgm_streaming_encoder_create_with_encode_options — and passing NULL options means the library defaults. Note that the algorithm name lives in options->hash, so these functions have no separate hash_algo argument. TGM_AGGREGATE_HASH_POLICY_HEADER and ..._BOTH are buffered-mode only: the streaming constructor rejects them with TGM_ERROR_ENCODING, because a streaming writer emits its header before any data object exists.

Memory ownership

• Handles returned by tgm_* constructors (e.g. tgm_decode returns tgm_message_t **) are owned by the caller. Free them with the matching tgm_*_free function.
• Pointers returned by accessor functions (e.g. tgm_object_data, tgm_object_shape) are borrowed from the parent handle and are valid only until the parent is freed.
• TgmFrame::payload borrows the caller’s message buffer, not the iterator: it stays valid for as long as that buffer lives, and tgm_frame_iter_free does not invalidate it.
• tgm_bytes_t returned by encode functions must be freed with tgm_bytes_free.
• tgm_last_error() returns a thread-local pointer to the most recent error message. The string is owned by the FFI layer; do not free it. Treat it as valid only until the next tgm_* call on the same thread.

Error handling

Every fallible tgm_* function returns a tgm_error enum value. TGM_ERROR_OK (= 0) is success; any other value indicates failure and populates the thread-local error string accessible via tgm_last_error().

tgm_error rc = tgm_encode(...);
if (rc != TGM_ERROR_OK) {
    const char *msg = tgm_last_error();
    fprintf(stderr, "encode failed [code=%d]: %s\n",
            (int)rc, msg ? msg : "(no detail)");
    return 1;
}

The error enum is non-exhaustive; future Tensogram releases may add new variants without bumping the major version. Always treat any non-TGM_ERROR_OK value as failure and rely on tgm_last_error() for human-readable detail.

Versioning policy

Tensogram is currently labelled “Emerging” software (pre-1.0). The ABI policy reflects that:

• The C library SONAME is MAJOR.MINOR (e.g. libtensogram.so.0.20). Every minor release bumps the SONAME, so consumers must rebuild. Patch releases (0.20.x → 0.20.y) keep the SONAME stable.
• The C ABI may change between minor releases without an explicit deprecation cycle.
• The wire format version is independent of the SONAME and lives in the TGM_WIRE_VERSION constant exposed by tensogram.h. See plans/WIRE_FORMAT.md for the wire-level versioning policy.
• When the project crosses 1.0, the SONAME suffix policy will be reviewed; the cargo-c version_suffix_components = 2 lock in Cargo.toml is the explicit mechanism for that decision.

If you depend on Tensogram from C, expect to rebuild on every minor release for now. The Rust API has the same caveat.

See also

• C++ API — RAII / exceptions wrapper over the C API.
• Frame Introspection — the language-neutral contract behind tgm_frame_iter_* / tgm_message_header.
• Remote Access — transport behaviour of the object-store backend behind tgm_file_open_remote.
• Error Handling — taxonomy of tgm_error variants and what each one means.
• Internals — wire format, encoding pipeline.
• rust/tensogram-ffi/README.md — crate-level overview.
• The full C header: tensogram.h, regenerated by cbindgen on every cargo build. Source of truth for every type and function signature.