The frontier file format.
Most stacks keep Parquet for scans, Lance or Arrow for random access, and a separate copy for the GPU. Spiral builds all of its tables on Vortex, an open format that is faster than both Parquet and Lance on scans and random access, and can decompress at more than 1 TB/s on modern GPUs. Vortex is a Linux Foundation project licensed under Apache-2.0, donated by Spiral in 2025.
Why Vortex
- 100× faster hot random access than Parquet, 3× faster than Lance, and 5× faster than Arrow
- 5× faster scans and 2× faster writes than Parquet
- 4× faster GPU analytics than Parquet, in cuDF
- 1.6× faster ClickBench than Parquet, on the same DataFusion engine
- About half the size of Lance, and 1.1× the size of Parquet with zstd
Vortex leads on five of six measures.
| benchmark | Vortex | Parquet | Lance | Arrow |
|---|---|---|---|---|
| Scan throughput× Parquet, higher is better | 5.15× | 1.00× | 0.95× | — |
| Write throughput× Parquet, higher is better | 2.04× | 1.00× | 0.73× | — |
| ClickBench on DataFusionms, lower is better | 42.15 | 65.57 | 566.25 | — |
| GPU analytics in cuDFms, lower is better | 17.00 | 65.88 | — | — |
| Random access, NVMe, hotms, lower is better | 0.69 | 70.54 | 2.12 | 3.32 |
| Size× Parquet with zstd, lower is better | 1.14× | 1.00× | 2.11× | 5.95× |
The figures are from bench.vortex.dev, read on Nov 2, 2026. The GPU row is Spiral’s measurement of NDS-H SF10 in cuDF on a GH200. A dash marks a missing figure: cuDF does not support Lance, and Arrow was not run. On Spark with Iceberg, TPC-H at scale factor 100 runs nearly 30% faster on Vortex than on Parquet (Vortex on Ice).
See the full benchmark pageVortex in the wild.
Engines add Vortex support, and teams write about why they use it.
Announcing Vortex Support in DuckDBVortex ships as a DuckDB core extension.
Support the Vortex file formatClickHouse merged read and write support in October 2026.
Why We Built Vector Lakebase: Rethinking Unstructured Data Architecture for AIIn Zilliz’s test on S3, a point read fetched 0.07 MB from Vortex and 9.4 MB from Parquet.