Chunk Access Pipeline

When a Zarr chunk is read from a Z3FDB store, the library executes three steps for every Part (one per add_part() call):

1. Intersection — compute the overlap between the requested chunk’s bounding box and the Part’s bounding box. Parts with no overlap are skipped immediately.

2. FDB retrieve — issue a sub-request to FDB for exactly the fields inside the intersection. Fields outside it are never fetched.

3. Buffer fill — decode each returned GRIB field to float32 and write it into the correct position in the flat chunk buffer (row-major / C order).

The examples below use the two-part view from Dimension Mapping and Data Model: Part A covers surface parameters (sfc, 2 params, axis 1 = [0, 1]) and Part B covers pressure-level parameters (pl, 4 params, axis 1 = [2, 5]). Both parts share 4 date × time values on axis 0.

WHOLE_AXIS chunking

With WHOLE_AXIS on all axes there is exactly one chunk covering the entire array. Both parts contribute to it, each populating a disjoint rectangular region of the buffer.

Chunk (0, 0) bounding box: axis0 = [0, 3], axis1 = [0, 5]

           0   1   2   3   4   5
         ┌───┬───┬───┬───┬───┬───┐
      0  │ A │ A │ B │ B │ B │ B │
         ├───┼───┼───┼───┼───┼───┤
      1  │ A │ A │ B │ B │ B │ B │
         ├───┼───┼───┼───┼───┼───┤
      2  │ A │ A │ B │ B │ B │ B │
         ├───┼───┼───┼───┼───┼───┤
      3  │ A │ A │ B │ B │ B │ B │
         └───┴───┴───┴───┴───┴───┘

Intersection A:  axis0 = [0, 3], axis1 = [0, 1]  (left two columns)
Intersection B:  axis0 = [0, 3], axis1 = [2, 5]  (right four columns)

Buffer extent: [4, 6]

Part A — partAxisOffset = [0, 0], bufferOffset = [0, 0]
          The intersection starts at A's local origin and at the buffer corner.

Part B — partAxisOffset = [0, 0], bufferOffset = [0, 2]
          The intersection starts at B's local origin but at buffer column 2,
          because B begins at global index 2 on axis 1.

SINGLE_VALUE chunking

With SINGLE_VALUE every chunk holds exactly one field. Accessing chunk (1, 2) targets one cell that falls entirely inside Part B.

Chunk (1, 2) bounding box: axis0 = [1, 1], axis1 = [2, 2]

           0   1   2   3   4   5
         ┌───┬───┬───┬───┬───┬───┐
      0  │   │   │   │   │   │   │
         ├───┼───┼───┼───┼───┼───┤
      1  │   │   │ ■ │   │   │   │   ← chunk (1, 2)
         ├───┼───┼───┼───┼───┼───┤
      2  │   │   │   │   │   │   │
         ├───┼───┼───┼───┼───┼───┤
      3  │   │   │   │   │   │   │
         └───┴───┴───┴───┴───┴───┘

Intersection with Part A: empty — skipped.
Intersection with Part B: axis0 = [1, 1], axis1 = [2, 2]

Part B — partAxisOffset = [1, 0], bufferOffset = [0, 0], bufferExtent = [1, 1]

    axis 0: partAxisOffset = 1 because the intersection starts at
            date×time index 1 within Part B's local axis.
    axis 1: partAxisOffset = 0 because global param index 2 is the
            first value in Part B's local param axis (B starts at
            global index 2).
    bufferOffset = [0, 0] because the intersection coincides with
            the chunk's own lower-left corner.

FDB returns one field. Within Part B, axis.index(key) = [1, 0]:

    axis 0: local = 1 − 1 = 0,   bufPos = 0 + 0 = 0
    axis 1: local = 0 − 0 = 0,   bufPos = 0 + 0 = 0
    → written to buffer slot (0, 0)

FixedSizeChunking — cross-part chunk example

With FixedSizeChunk(2) on axis 0 and FixedSizeChunk(3) on axis 1, the chunk grid is 2 × 2. Chunk (0, 0) covers two date×time steps and the first three param slots, which straddles the boundary between Part A and Part B.

Chunking:  axis0 = FixedSizeChunk(2),  axis1 = FixedSizeChunk(3)

Chunk (0, 0) bounding box: axis0 = [0, 1], axis1 = [0, 2]

           0   1   2   3   4   5
         ┌───┬───┬───┬───┬───┬───┐
      0  │ A │ A │ B │   │   │   │  ← rows covered by chunk (0, 0)
         ├───┼───┼───┼───┼───┼───┤
      1  │ A │ A │ B │   │   │   │
         ├───┼───┼───┼───┼───┼───┤
      2  │   │   │   │   │   │   │
         ├───┼───┼───┼───┼───┼───┤
      3  │   │   │   │   │   │   │
         └───┴───┴───┴───┴───┴───┘
         └─  (0,0)  ─┘

Intersection A:  axis0 = [0, 1], axis1 = [0, 1]   (left 2 columns)
Intersection B:  axis0 = [0, 1], axis1 = [2, 2]   (third column only)

Buffer extent: [2, 3]

Part A — partAxisOffset = [0, 0], bufferOffset = [0, 0]
          Intersection starts at A's local origin and at the buffer corner.

Part B — partAxisOffset = [0, 0], bufferOffset = [0, 2]
          B's local axis1 starts at global index 2, so global [2, 2]
          maps to local [0, 0]. The intersection lands at buffer column 2
          because 2 − 0 (chunk lower bound) = 2.

Buffer layout (2 rows × 3 columns):

           0   1   2
         ┌───┬───┬───┐
      0  │ A │ A │ B │
         ├───┼───┼───┤
      1  │ A │ A │ B │
         └───┴───┴───┘

FDB issues two sub-requests — one for Part A, one for Part B. Each field is placed using the buffer-position formula. For a field returned by Part A with axis.index(key) = [1, 1] (second date×time, second sfc param):

axis 0: local = 1 − 0 = 1,   bufPos = 1 + 0 = 1
axis 1: local = 1 − 0 = 1,   bufPos = 1 + 0 = 1
→ written to buffer slot (1, 1)

For a field returned by Part B with axis.index(key) = [0, 0] (first date×time, first pl param — which is global param index 2):

axis 0: local = 0 − 0 = 0,   bufPos = 0 + 0 = 0
axis 1: local = 0 − 0 = 0,   bufPos = 0 + 2 = 2
→ written to buffer slot (0, 2)

See also

Buffer Layout and Position Formula for the general buffer-position formula and how the flat buffer index is computed from the per-axis positions.