Skip to content

Connectors

Apache Doris has an optional experimental adapter for Flight/Thrift Arrow reads, including experimental incremental scans, and Arrow Stream Load writes, with the released connector retaining enumeration and transaction handling. Its validation is in progress; see its coverage page for fallback boundaries.

Kafka is the only connector StreamFusion considers production-grade today, across the native formats listed below. It's covered in depth in this section.

Parquet and ORC (stock Flink filesystem reads plus native sink encoding to any filesystem Flink supports), Delta Lake (path-based append and merge-on-read sinks), and Apache Paimon (streaming sources and append-only and primary-key table sinks, including fixed, dynamic, and postpone buckets, with Paimon's own compaction) also have native paths, but they have not seen the same production hardening as Kafka — treat them as experimental.

Apache Fluss is an experimental Java Arrow transport for append-only and primary-key log reads and append-only production, enabled by default when installed. Its connector-owned Rust helper handles scalar bucket-key routing and batch statistics. Primary-key writes stay on Flink.

Formats

A format decodes or encodes the bytes on the wire; it's independent of which connector carries them (though not every connector supports every format — see each connector's page). StreamFusion ships a native implementation for:

Each optional format is its own streamfusion-* Maven artifact and native library, mirroring Flink's own connector/format module split — installing a format you don't use never pulls its native code into a connector you do, and each library exports only its own class's native entry points, so the JVM binds every core method to the core library alone. A missing format module is always a normal planner fallback to Flink's own decode/encode path, never a linkage failure. See Deployment for which JARs a given connector+format combination needs.