Blog

How choosing to store data in columns instead of rows dramatically changes the way in which a database system works.

With the release of DuckDB 2.0, we look at how the Vortex core extension speeds up queries by computing over compressed data, and how to read and write Vortex files from DuckDB.

Min-max statistics are aggregates computed at write time. Treated that way, they can answer queries, not just skip data.

GPU decompression at 2 TB/s with Fused Dynamic Dispatch. How Vortex doubles throughput by fusing CUDA kernels.

We optimized Vortex so it can stream data from object storage into GPU memory fast enough to saturate a 100 Gbps network link. The first step to get this performance is to make sure each byte is only copied once between the kernel and the GPU.

Spiral runs geometry operations directly on columns of coordinates, which makes our polygon benchmarks 5× faster overall.

How Vortex keeps logical types, physical encodings, and storage layouts open to extension so one columnar format can fit any shape of data.

A composable query engine deserves a composable storage layer. Build the format your workload needs without rebuilding the storage stack.

How Spiral helps your clean PyTorch code avoid a video preprocessing footgun.

Every workaround data teams invented to go faster is actually a defense against irreversibility.

Two teams independently built SQL LIKE over FSST-compressed strings: Vortex prefilters candidates, TU Munich compiles the pattern. One experiment across seven x86 generations maps where each wins, where they cross, and why the two hot loops belong in one system.

Evaluating SQL LIKE directly on FSST-compressed strings: a symbol-aware automaton and a SIMD prefilter let Vortex match without decompressing the column first.

We examine each layer of H.264 video in an MP4 file and follow one frame query from timestamps to tensors.

How Vortex uses BtrBlocks-style codec selection to beat Parquet+ZSTD on both size and speed.

The fastest file format is now free and open source, forever.

Using Vortex to accelerate Apache Iceberg queries up to 4x

Preparing to launch Vortex 1.0 and the 2025.05 Edition

Tips and lessons learned from building systems directly against object stores

Rust defines the cast from a special float value to an integer, so our Rust port of ALP compresses 20% to 50% faster than C++.

How push-down compute helps Vortex run faster and decompress less

Waiting sucks. Let's use zone maps to move fast and read less.

How Vortex tackles data layout challenges across analytics and machine learning workflows.

The rationale behind the Vortex logical type system.

Random access string compression with FSST and Rust.

On a journey to make your data more fun sized.

Decoding > 100 billion integers per second with scalar Rust.