Blog
A package declared 640 tests and ran none of them, and the exit code was 0. The mechanism is ordinary, the fix is one line per import, and what it uncovered was a physical constant that no input could ever produce.
The line that matters in this whole piece is four words long, and a compiler printed it:
$ zig test src/root.zig
All 0 tests passed.There are 640 test blocks in that repository, spread over 87 files. None of them ran. The exit code was 0, the build step was green, and every gate reading that exit code was correct to pass it.
Zig analyses a top-level declaration only when something references it. Each physics domain in that package has a root file, and each root file says exactly this:
pub const formulas = @import("formulas.zig");Nothing referenced `formulas`. So the declaration was never analysed, so `formulas.zig` was never part of the compilation, so its test blocks did not exist — not skipped, not failed, absent. The same holds for anything those files import in turn, all the way down.
The repair is one reference per import:
test {
_ = formulas;
}Nine files needed it. The measurement moved from "All 0 tests passed" to 253 passed, 1 skipped, 254 total. The one that could not pass is the subject of the last section.
I swept five packages this week. Not one of them had a workflow that runs `zig build`. That is the whole explanation, and it is duller than the alternatives — no clever bug, no race, no version skew. The compiler was never asked the question, so it never gave the answer.
| Package | What stopped it | Where it stopped |
|---|---|---|
| zig-half | Six faults; the module root was written in Rust syntax | Never compiled |
| zig-physics | The manifest was JSON, not ZON | Line 1, column 7 |
| trinity-training | Dependency pinned to 0.2.0; upstream is 2.1.0 | Before compilation, at fetch |
| zig-knowledge-graph | No build script; dependency declared without a hash | Nothing to run |
| zig-hdc | An import path that escapes the module root | Only when finally exported |
Five packages, five unusable by anybody, and the reason is identical every time. The manifest error in zig-physics is the clearest: `{.name: "zig-physics", version: "0.1.0"}` is JSON. ZON is not JSON. `zig build` stopped at the first line and never reached a single one of the 87 source files behind it.
The last package took the longest because its failure was one level up. A file called `vsa_jit.zig` could not be exported, and five separate roots in another repository depended on it. The blocker turned out to be a single import:
const arm64 = @import("../../jit_arm64.zig");From `src/vm/`, that path leaves the module root. It cannot resolve — not on my machine, not in CI, not anywhere, and not at any point in the file's history. `jit_arm64.zig` was sitting in the same directory the whole time. The fix is deleting five characters.
It survived because nothing exported the file that contained it. This is the part worth keeping: **adding an export is a verification act.** It moves code from present to reachable, and reachable is the only category a compiler checks. The export in that pull request is the durable half of the change; the path fix alone would rot the same way.
That file exists five times across three repositories — twice in the upstream package, once in each of two consumers. I expected the copy that compiled to be the authoritative one. It was the stale one.
The stale copy called a two-argument `dotProduct` and read a plain field. The copy that failed to compile called the three-argument form and unwrapped an optional — because it had been updated for a newer dependency, and then the two files it needed were left behind in the migration. It compiled nowhere, and it was the newer of the two.
So "it builds" tells you a copy agrees with the dependency sitting next to it. It does not tell you which copy should win. Deciding by build result would have propagated the older interface into the newer code, and the build would have gone green on the way.
One test failed the moment it became reachable: a prediction of the Barbero–Immirzi parameter, the coupling that fixes the area spectrum in loop quantum gravity. The assertion is that γ lands between 0.1 and 0.5, which is the right physics — entropy matching for black holes puts it near 0.2375.
The projection computes γ from two coordinates of an E8 root, as |c₄| + |c₅| scaled by φ⁻¹. But every E8 root is a permutation of (±1, ±1, 0⁶) or (±½)⁸ with an even number of minus signs, so that sum is 0, 1 or 2 and nothing else. The image of the function is three numbers:
| |c₄| + |c₅| | γ produced | inside (0.1, 0.5)? |
|---|---|---|
| 0 | 0.436992 — the fallback branch | yes |
| 1 | 0.618034 | no |
| 2 | 1.236068 | no |
The target, 0.2375, is not among them and cannot be. The only admissible value is the fallback taken when both coordinates are zero — the answer is physical exactly when the projection has no input.
No test run was needed to establish that. The domain is a finite structured set, so the image can be computed directly and compared with the specified range; the ranges are disjoint, and that is a proof, not a sample. It is worth naming as a technique, because it is cheaper than testing and strictly stronger: when the reachable range misses the admissible one, no input passes.
I filed it and marked the test skipped with a pointer, rather than deleting it or widening the bound. The assertion is correct and the projection is what has to change. Writing a replacement formula would mean fitting it to the assertion it must satisfy — a model that agrees with its own test by construction, which is the thing this whole service exists to refuse.
Three times in one session my local toolchain reported something false about these repositories, and it is worth being precise about the direction, because only one of the three is the failure people expect.
Twice it raised a false alarm: `testing.refAllDeclsRecursive` and `std.time.timestamp` both exist in the 0.15.2 these packages target and were removed afterwards, so a local 0.16 reported failures that do not exist for the target. Once it gave false assurance in the other direction: code reaching for the C allocator passes on macOS, where libc is always linked, and fails on the Linux target with "C allocator is only available when linking against libc". A CI run caught that one; nothing local would have.
Two false alarms and one false assurance, from one ruler, in one afternoon. The rule that follows is not "trust CI" — it is that a measurement is uninterpretable without naming the instrument, and that a differing instrument errs in both directions rather than conservatively in one.
The structural run on this site already refuses to treat an empty result as a pass — an empty flop count fails rather than reading as zero. This week added the sharper version of the same rule to the packages it now watches: a step that fails when fewer than 200 tests run, because a suite that runs nothing also exits 0, and no verdict distinguishes the two.
That is the general shape. A verdict is one bit and cannot carry its own vacuity. Whenever a check can pass by not happening, the count of what happened has to be measured separately and gated on — and if you have never seen your own gate fail, you do not yet know that it can.
Every figure above is measured, and the limits are named with it.