Memory backend

Status: Native, default

Every native stateful operator — aggregates, joins, dedup, Top-N, window operators, and the rest — holds its state as in-process, JVM-heap-resident structures unless state.backend.type selects something else. There is no separate configuration to opt into this backend; it is what StreamFusion runs on out of the box, and what every operator falls back to when a query condition or build configuration keeps it off the RocksDB backend.

Checkpointing model

State is checkpointed as the backend-independent canonical state format: on each barrier, an operator serializes its live state by key group into versioned, bounded managed-state chunks. There is no incremental upload and no manifest diffing — every checkpoint is a complete snapshot of the operator's current state, the same shape regardless of how much changed since the last barrier. Snapshots from the earlier nanosecond timestamp layout are rejected; see the timestamp layout upgrade.

This is the simplest possible durability story, and it is fast for the common case: no on-disk table, no compaction, no point-read join on the hot path. The tradeoff is checkpoint size and duration scale with total state size rather than with the delta, which matters once state grows past what comfortably re-serializes and uploads every barrier.

Restore

A memory-backend checkpoint or canonical savepoint can restore on either memory or the RocksDB backend. A RocksDB restore imports the logical partitions into its snapshot store on the first run; subsequent checkpoints use the native RocksDB lifecycle.

When to reach for the alternative

The memory backend is the right default for most jobs. Consider the RocksDB backend when checkpoint size or duration dominated by large keyed state becomes a problem — see Benchmarks for measured memory-vs-RocksDB throughput across the Nexmark queries, and Configuration for how backend selection and other runtime flags are set.