A test suite that mails a 36 MB report every night is a suite nobody reads. This release makes the nightly run say only what changed, and say when it cannot be trusted.
What runs
- Four projects in sequence at three in the morning: the core, the scripting layer, the shared Studio library and the document engine. Reports stay on disk; mail goes out only when the counts differ, with a weekday heartbeat so silence is known to be silence.
- Engines probed first Each database engine is reached over TCP before anything runs. A crash-looping Vertica container once produced 1,232 errors across 389 classes against a baseline of two; the gate exists because of that night.
- Drivers in the module The thirteen test JDBC drivers for the ten non-embedded engines moved into this module, so the scripting project no longer skips 1,088 of its 1,377 multi-database tests.
What is reported
- Test by test The diff compares class and name, not per-engine invocations, so a template run on fourteen engines is one fact. Outcomes are new, fixed, moved, standing and expected.
- Known failures as a ledger A per-project file separates explained red from unexplained; an entry that now passes is flagged stale. Failures are bucketed by message against infrastructure rules, each carrying the incident that created it.
- Invalid outranks regression A foreign build or a changed jar under the running JVM marks the run invalid; every classpath jar is opened before discovery. The editor's automatic build was the intruder at one minute past three.
The first run under the new code reported one new failure, two fixed, two standing and six expected. That is a mail someone reads.