Skip to content

Native columnar keyed shuffle

Applies to: keyed operators reached through a shuffle

A keyed exchange splits each Arrow batch by the projected key's Flink BinaryRow hash, Murmur key-group mix, and channel mapping, so a keyed operator's input stays columnar across the shuffle instead of transposing to rows and back. Matching the host key-group layout now lets native raw keyed state rescale safely while the hot map stays in Rust.

The fully-columnar windowed pipeline (source → watermark → shuffle → window) measured 1.91x vs Flink, where the row-fed window was 1.21x.