Robot data, from recordings to training batches.

Spiral models cameras, lidar, state, actions, annotations, and calibration as related collections on a shared, versioned timeline. Keep the native recordings; declare the episode window; let Spiral plan the reads and materialize the batch.

David Butterfield / UW · NOAA PMEL

Native source readers

Spiral can use MCAP message indexes and video sample structure to seek the topic, time range, compressed samples, and bytes a request needs.

Episode-based sampling

Streams keep independent rates and cardinalities. Episode queries declare windows, alignment, resampling, and missing-data policy.

Calibration and lineage

Versioned transforms and end-to-end lineage connect each sample and model output to the coordinate state that produced it.

01 · Source and sample

Recordings are sources; episodes are derived.

A robot does not produce one tidy row per instant. Cameras, lidar, state, actions, events, and annotations all have different rates and shapes. Spiral keeps them as related collections instead of denormalizing them at ingest, then composes only the streams a sample asks for.

Close view of camera and lidar sensors on an autonomous vehicle — A sensor rig is a relationship among observations, clocks, poses, and calibration. Spiral's episode model keeps those relationships addressable instead of hiding them in a dataloader.

sessions
└── episode_0417
    ├── camera.front[]    30 Hz
    ├── lidar.roof[]      10 Hz
    ├── vehicle.state[]  100 Hz
    ├── actions[]         20 Hz
    └── events[]          sparse

Filtering episode metadata need not read a frame tensor. The query chooses the collections and representation it needs.

Open the physical-AI data model demo

02 · Native planning

A time window spans multiple source formats.

A six-second episode slice can cross MCAP chunks, video codec dependencies, point-cloud messages, and scalar state. Spiral's source readers plan each native structure; the episode layer composes the outputs into one sample.

The distinction matters: MCAP indexes find topic and message ranges, while an MP4 sample table locates compressed video bytes and the codec determines which reference frames must be decoded. One timestamp does not erase those physical differences.

Source	Spiral reads	Why it matters
MCAP	Summary + message indexes, topic/time ranges, chunks	Avoid replaying unrelated topics or the whole recording
H.264 / MP4	Sample tables, decode closure, coalesced byte ranges	Fetch and decode the prerequisites of selected frames
Episode plan	Clock, alignment, missing policy, requested outputs	Compose native reads into one reproducible sample

Trace the video half of the plan

03 · Sample semantics

Alignment is part of the sample definition.

In Spiral, nearest(20 ms), pad, and mark missing describe different samples. Clock choice, tolerance, resampling, and missing-data policy belong beside the query that produced the batch, not in undocumented dataloader cleanup.

camera.front · episode 0417t = 06.94s

Alignment windowtolerance ± 20 ms

Camera

30 Hz

LiDAR

10 Hz

State

100 Hz

Action

20 Hz

Window center

Tolerance

Missing

Camera+03 ms

LiDAR−14 ms

State+01 ms

Action+09 ms

Sensor array on an autonomous vehicle with a projected observation overlay

projection · calibration v7

04 · Calibration and lineage

Calibration is versioned with the observations.

Spiral versions the project state that gives a sample meaning. A sample manifest can name the source streams, project revision, and calibration used to produce a projection. Automatic lineage propagation makes the downstream products of a changed calibration visible to the project.

05 · Access patterns

Replay and training have different access patterns.

Spiral keeps native sources as the system of record and lets the requested sample drive the plan. Sequential replay rewards contiguous reads; shuffled training windows reward selective ranges, bounded video decode closures, and layouts matched to the batch.

Sequential replay

One source · increasing time

Long contiguous ranges, decoder stays warm, nearly every output is useful.

Shuffled training

Many sources · sparse windows

Indexed ranges, bounded decode closures, selected outputs materialized as tensors.

06 · From event to batch

From an event predicate to a training batch.

Spiral filters events and annotations, expands the temporal window, resolves the requested streams, and materializes the sample. The result remains traceable to the recording, clock, calibration, annotation set, and project revision that produced it.

// Query event-aligned episodes
project.asof(revision)
  .episodes.where(event == "hard_brake")
  .window(before="2s", after="4s")
  .align(camera, lidar, state,
         tolerance="20ms",
         missing="mark")
  .to_tensor(batch=64)

sample manifest

source: run-0417.mcap · camera-02.mp4
ranges: MCAP chunks 31–34 · MP4 samples 884–962
alignment: sensor_time · nearest 20ms · mark missing
calibration: rig/extrinsics@v7
annotations: failure-events@2026-07-08
project: revision 7d2f…a91c

Talk to us