Strings cross the entry transpose with one copy, and the lookup join writes rows straight into Arrow¶
Applies to: the RowData→Arrow entry transpose and the synchronous lookup join
Entry transpose: one copy instead of two¶
The entry transpose's VarChar writer called StringData.toBytes() — a fresh byte[] copy — only
for Arrow's setSafe to copy those same bytes again. A single-segment heap BinaryStringData —
what every string coming out of Flink's row formats already is — now feeds its segment straight
into the Arrow buffer, halving the copies and deleting the per-string garbage on the
string-dominated entry transposes (q9/q18/q19/q20/q21/q22).
A still-lazy BinaryStringData — what a rowwise source delivers through RowRowConverter, holding
only a java.lang.String — additionally skips Flink's char-at-a-time StringUtf8Utils
materialization: the JDK's intrinsified String.getBytes(UTF_8) produces the identical bytes
(Flink's encoder documents JDK-equivalent output and delegates its edge cases to it) at
near-memcpy speed for ASCII. A 2026-07-12 profile had this char loop at ~9% of q20's whole job.
Measured: with the batched BinaryRow keys above, the string-heavy stateless q22 gained +25% native throughput (1.23x → 1.53x) on the 2M-event generator rung.
Lookup join: collect straight into the Arrow builders¶
The sync lookup join stopped defensively copying every looked-up row (RowDataSerializer.copy,
~27% of q13's lookup path) plus buffering them in a list. The collector now writes each row's
fields into the Arrow builders at collect time, while the runner's reused row object is still
valid — removing both the copy and the intermediate list.