Skip to content
Workflows / Curate

Curate and test a training set live, and reproduce it forever.

Write the set as rules over your data. Commit it, tag it, and every training run and evaluation reads exactly those rows.

Rules, a training set, and a tag

Training setfailed_grasps · branch curate/drops
contact_duration≥ 500 ms
commanded_release= false
slip_distance> 15.0 mm
left_hand= true
Selectioncandidates9,374similar rows added+38eval overlap09,412rows at tag v12
Why curation

Three groups published what curation bought them.

+50 ptsSuccess-rate gain from fine-tuning on a curated set one-fifth the sizeHugging Face · LeRobot+78%Shirts per hour after weighting productive moments in WARP-RMYu et al. · WARP-RM6×Less compute to train a language model that matches a baseline built on the same open dataDatologyAI · Curation Studio
One population

A training set is four rules, not a folder of copied clips.

Population840,000moments across every recorded episodeBuilt on Spiral · Spiral Robotics
01Write each rule as a condition on a column.A rule is one more kind of Enrichment: a true or false value for each row. In Spiral Robotics, an app built on Spiral, the rules form one query that you can run again on new rows.
Rules9,374 candidates
contact_duration≥ 500 ms
commanded_release= false
slip_distance> 15.0 mm
left_hand= true
02Look at the distribution.Move a threshold, and the count updates. Near the line, a recovery and a drop can look alike, so look there first.
Distribution · slip_distance9,374 candidates
03Find more rows like the ones you kept.A vector search over the same rows finds near misses that the rules did not catch.
Similar rowsepisode_7f31c2 · t+7.8s
nearest moments
Search results214 similar rows38 not yet in the setadded to the selection
04Commit the set to a branch.Spiral writes the set to a table and commits it. Tag the commit for training.
Training setbranch curate/drops
9,412rowsfailed_grasps · tag v12commit 01a1a71d
Review the change

Review the rows before training reads them.

Spiral lists the rows that each commit adds and removes, back to any earlier commit of the set.

Changed rowstag v11 → tag v12

+ episode_9a04e1 · t+3.2s

− episode_2c77d0 · t+11.4s

+ episode_51b3f8 · t+0.8s

+614added−2,289removed9,412rows
The same set, in code

Read the set at its tag.

Every consumer resolves tag v12 to commit 01a1a71d. A check is a query with an expected result.

Python · failed_grasps at tag v12

Docs
import spiral
from spiral import column, eq, literal
from spiral.services.catalog import wire

client = spiral.Client()
tag = wire.CatalogTagTarget(tag="v12")
v12 = client.catalog.resolve_commit(
    "robotics",
    wire.CatalogResolveCommitRequest(
        selector=wire.CatalogCommitSelector(tag=tag)
    ),
)
robotics = client.project("robotics")
at_v12 = robotics.snapshot_at(v12.commit_id)
drops = at_v12.table("failed_grasps")

# A check is a query that must match no rows.
released = drops.scan(
    ["episode_id"],
    filter=eq(column("commanded_release"), literal(True)),
)
assert released.to_table().num_rows == 0

print(drops.to_arrow(["episode_id"]).num_rows)  # 9_412

tag v12 · commit 01a1a71d · 9,412 rows

faqs

Before you tag the set.

New rows arrive in new commits, and a tag keeps its exact rows. Apply the saved rules to the new rows, commit, and tag the new commit.

Yes. Each researcher works on a branch and compares the rows that each change adds and removes. A reviewed change gets a new tag, and the old tag does not move.

A tag fixes the evaluation set’s rows. A query in CI checks that the training set and the evaluation set share no rows.

Continue the loop

Training reads the tested set at its tag.

InvestigateFind every failure like this one.EnrichAdd columns without rewriting the table.TrainTrain on any mix of your data.

How it works:Reversibility & isolationand Multimodal engine