Skip to content
Workflows / Investigate

Find every failure like this one.

Point your agent at one failure, and it searches every recording for the moments that match it.

Works with MCAP, LeRobot, MP4, and Parquet

Investigation branch6 commits
QueryLeft-hand slip after stable contact
slip_distance > 15.0 mmcommanded_release = falsesimilarity > .82
2,684matches
02signals saved
01results table
slip_candidatesbranch · investigation/left-hand-drops
The current way

You can replay any one log, but you cannot search all of them.

Fleets record everything, then people scrub logs one at a time. A fault can appear seconds after its cause, and each new question starts another preprocessing script.

“manually analyzing large-scale failure datasets is impractical and does not scale”
Gupta et al. · Failure taxonomies
“try rummaging through multiple mcap files by hand to see where a fault happened”
Hacker News comment
Your agent built on SpiralGrasp failure 01422,684 moments · Open
Regina G.

Investigate this moment: stable grasp, wrist correction, then drop.

episode_7f31c2 · t+6.2s–8.8swrist_camera · gripper_ft · robot_state · task_logOpen moment
t+7.8shover or drag the timeline
contactstable loadobject motion
Your agent

I can reuse three signals: contact loss, commanded release, and object position. I will test two new signals on a sample of 12,400 moments, then scan the full corpus.

Signal planreuse → derive → query
Use existing signals3 ready
contact_lossv3
commanded_releasev2
object_positionv5
Compute new signalssample · 12.4k moments
contact_durationderived
slip_distancederived
Scan the corpus2,684 matches
contact_duration ≥ 500 ms ∧ slip_distance > 15 mm ∧ ¬commanded_release
slip_candidates · 2,684 momentsCommitted to investigation/left-hand-dropsRefine the query
Your agent

The query found 2,684 moments. I saved them as a table on the branch investigation/left-hand-drops. I then compared 87 signals that are not in the query. Four signals have large effects and statistically significant associations with the matches.

Correlated signalsmatches n=2,684 · comparison n=837,316
correction_latency_ms118 ms vs 72 msKS D=.42q<.001
gripper_fw = 2.4.142% vs 15%OR 2.8 [2.3, 3.4]q<.001
wrist_yaw_error7.1° vs 3.4°KS D=.31q<.001
low_light31% vs 18%OR 1.6 [1.3, 2.0]q<.001
Your agent

Firmware 2.4.1 and slow wrist corrections stand out the most. Low light shows up too, but the firmware rollout can explain it. This is correlation, so the next step is to compare firmware versions directly or use matched controls.

Stratify by firmwareFind matched controlsOpen in viewer
What you keep

A debugging session ends as a branch.

The query, the two new signals, and the 2,684 matches are commits on investigation/left-hand-drops. A teammate can run the query again on next week’s episodes, or merge the branch into main.

Investigation branch6 commits
Queryleft-hand slip
Signals2 new, 3 reused
Tableslip_candidates
Result2,684 moments
Beside your tools

Search every recording, then open the match in your viewer.

Spiral imports your recordings as tables, and a vector index on an embedding column finds similar moments. A search can start from a frame, a timestamp, a signal, or a plain description that your agent turns into a query.

Your recordingsyour buckets
recordingss3://fleet-a/*.mcapgs://lab-video/*.mp4hf://lerobot/*
tablesfleet_a.episodeslab_video.clipslerobot.episodes
The same scan, in code

The session resolves to a query you can inspect.

Change a predicate, run it again on another branch, or read the exact SQL behind a match.

SQL · slip_candidates

Docs
-- spiral sql --project robotics --branch investigation/left-hand-drops
CREATE TABLE slip_candidates AS
WITH seed AS (
  SELECT wrist_embedding FROM episodes
  WHERE episode_id = 'episode_7f31c2' AND moment = 7.8
)
SELECT e.episode_id, e.moment, e.slip_distance
FROM episodes AS e, seed AS s
WHERE e.commanded_release = false
  AND e.contact_duration >= 500
  AND e.slip_distance > 15.0
  AND "spql.tensor.cosine_similarity"(
    e.wrist_embedding, s.wrist_embedding
  ) > 0.82
ORDER BY e.slip_distance DESC;

2,684 matching moments one commit on investigation/left-hand-drops

faqs

Before you search the corpus.

MCAP, LeRobot, MP4, and Parquet. One query reads all of them, because the images, arrays, and tables sit side by side.

Not all of it. MP4 files can stay in your bucket, and a query reads only the byte ranges that it needs. MCAP and LeRobot recordings import into tables, which can live there too.

A refresh computes the saved signals over the new rows. You can start it yourself or from an Automation. A pinned result does not change.

No. Keep your viewer for playback. Spiral finds the moments worth opening in it.

Yes. An agent can list projects, branches, and tables through Spiral’s MCP server. It runs SQL and commits through the spiral CLI, on a branch of its own.

Continue the loop

Enrichments add new columns to the saved moments.

EnrichAdd columns without rewriting the table.CurateCurate and test a training set live, and reproduce it forever.TrainTrain on any mix of your data.

How it works:Multimodal engineand Reversibility & isolation