Skip to content

Upstream test builds

This optimization reduces CI work and waiting time; it does not change query throughput or native admission. See the upstream suite guide for commands and coverage requirements.

Each workflow builds the Flink planner and current StreamFusion payloads once per Flink line, then transfers those artifacts to the suite jobs. The shared artifact excludes Cargo intermediates and test evidence. Maven caches the isolated repository actually used by the harness. Connector jobs compile only their additional fixtures, while each runtime suite is split into four jobs balanced by historical class durations. Full test selection, parameter counts and native/fallback execution contracts remain blocking.

Before this change, the September 25 Flink 1.18 inventory job took approximately 101 minutes, including 85 minutes in the test invocation. The September 26 Flink 2.2 runtime CI job took approximately 50 minutes, including 31 minutes in the test invocation. These are CI timings from different runs, not engine benchmarks or a controlled version comparison. A measured end-to-end improvement must include build preparation, artifact transfer and runner queues; balanced historical class durations alone are not a measured CI speedup.

The Paimon 1.0 compatibility workflow builds release native libraries during source/sink parity and reuses them for the upstream SQL suite in the same job. Previously it compiled both debug and release libraries: the baseline run took 110.5 minutes, including approximately 31 minutes of debug compilation and 49 minutes of release compilation. The upstream phase uses one JVM. An initial two-JVM trial finished in approximately 67 minutes but hit an upstream wait timeout in PartialUpdateITCase.testForeignKeyJoin (241 of 242 tests passed), so CI retains serial execution to avoid the additional contention. Configurable forks retain distinct identities and separate evidence directories. See the Paimon guide for configuration and reuse prerequisites. These changes remove duplicate compilation; a passing hosted post-change speedup has not yet been measured.