Blog
[numbers are from docs/reports/t27b-bench-2026-10.md and the receipts it cites; not rerun for this publication] Five backends, one spec source: the whole-corpus differential run compares t27b against the reference test by test - 0 of 8,344 tests disagree - while Zig, clang and gcc pipelines disagree with themselves across optimization levels. The article also states what the zero does not prove: no independent front end exists, the fuzzer still finds 7 disagreements per 1,000 programs, and only 12% of the code t27b runs is generated.

Our earlier article, [One spec, five targets: the t27 backend shootout](/blog/one-spec-five-targets-the-t27-backend-shootout/), ranked five backends on speed and size. This article is deliberately not that. Speed and size answer "which build should you ship." A different question matters more when the thing you build is a compiler: how do you know its output is correct? This piece is about verification - what the same five-backend setup can prove about trust, and what it cannot.
The t27 repository keeps one correctness budget, stated as a number: the number of tests where two backends disagree. For the newest backend, t27b, the budget is spent at exactly zero.
The whole-corpus differential run compares t27b against the reference pipeline (t27c's generated Zig, built in Debug mode and run natively). Over 1,315 spec files that both sides ran, the verdicts were compared test by test, by name: 0 of 8,344 tests disagree. The same zero holds when the reference itself is rebuilt for aarch64 and executed under emulation.
That single number is the article's thesis. A compiler project that cannot state its disagreement count as one figure does not yet have a verification story; it has a test suite. A test suite tells you the code runs. A differential budget tells you the code agrees with something else that was built a different way.
The budget only means something because the other backends spend it. In the same run:
./scripts/tri test
./target/release/t27c suite --repo-root .
Zig's own ReleaseFast build disagrees with its Debug build on 7 files (11 tests). The generated C, compiled by clang at -O0 versus -O2, differs on 39 files (107 tests); gcc at the two optimization levels differs on 4 files. Those disagreements exist because the generated C contains undefined behavior - the verdict depends on the optimization level - and because ReleaseFast optimizes differently from Debug.
So the picture is not "t27b is perfect." It is: one backend agrees with its reference on every single test, while the mainstream pipelines cannot even agree with themselves across optimization levels. Exact agreement with a reference is worth more than approximate agreement with a family of your own builds.
Agreement at the file level ("the pipeline passed") is weak. A file can pass while one test inside it silently returns the wrong verdict. So the corpus runner compares at the test level, and it does so over the entire spec corpus, not a sample: 1,819 .t27 files under specs/, the same set t27b corpus specs walks, with nothing left out for being slow or broken.
The invocation:
./target/release/t27b test specs/a/b_c.t27 --overflow trap
shows the unit of comparison: one spec, its tests run in the JIT with overflow checks that trap. At corpus scale the same command becomes a walk over every file, and --reference adds the other side:
./target/release/t27b corpus specs --blockers --reference ./target/release/t27c
With --reference, every file also goes through the reference path (t27c test-report, the Zig backend), and where both sides ran a file's tests, the runner compares verdicts test by test and counts files where a test differs, or where only one side has it. The files the reference itself fails are left out of the denominator, so the comparison is honest about what was actually compared: at this tree, 1,311 files pass with at least one test in the reference, and t27b passes 1,296 of them (98.9%).
The smallest disagreement the differential ever catches is instructive, and it is not in t27b. specs/a/b_c.t27 makes the generated C emit bool dummy(void) { true; } with no return: it passes under gcc -O0 and clang -O2, and aborts under gcc -O2 and clang -O0. Four build configurations, one function, four verdicts. That is the failure mode per-test diffing exists to catch, and the reason the budget is counted per test, not per file.
This is the section that matters most, and the one a marketing page would omit.
The 0-of-8,344 result is not independent verification. The repository ran a Diverse Double-Compiling check - build the self-compile three ways, compare stage-2 outputs - and the signed receipt records the verdict honestly: DDC_BACKEND_AGREEMENT, not DDC_INDEPENDENT, with filter C4 FALSE. All three routes share one hand-written front end. The reference that t27b agrees with on 8,344 tests is produced by the same front end that t27b mounts. Agreement here proves the backends implement the same front end the same way. It cannot catch a bug that lives in the shared front end.
The other checks have the same shape:
t27c gen plus Zig..t27; the rest, including the front end and the hand-written test suite, is ordinary Rust compiled by rustc. t27b is not self-hosted, so its "self" is not actually under test.No check in the current setup compares t27b against an independently written t27 front end. That is the honest boundary: the verification is unusually *visible* - signed receipts for every commit, a per-test differential over the whole corpus - but almost all of it is internal consistency, not independence.
Verification at this level is not free, and the shootout article's speed numbers are the price tag. The reference pipeline compiles a spec in 3.05 seconds (Zig Debug, with LLVM and a link step); t27b does the same work, tests included, in 4.8 ms per spec. Its whole binary is 3.08 MB, with no assembler and no linker.
The trade cuts the other way too. The negative result is stated as plainly as the positive one: over the tier-1 benchmarks, t27b executes 3.4x the instructions of gcc -O2 and produces code 1.8x the size. Its generated code sits at gcc -O0 quality, behind every optimizing pipeline in the comparison. Coverage also trails the reference it mirrors: 15 files the reference passes do not pass in t27b.
Most projects would bury one of these halves. The shape of the trade is the finding: a backend can be near-zero on the disagreement budget and simultaneously last on code quality, because agreement with a reference and quality of output are orthogonal axes. A fast, small, exactly-agreeing baseline compiler is a useful artifact even when its optimized output loses - provided you say which axis each claim lives on.
There is also a shipping boundary, stated so nobody over-reads the result: t27b emits object files only (ELF64 or Mach-O), it does not link, and it refuses module-level variables in an object file. It is a test runner and a code generator, not yet a producer of runnable executables.
The honest list, so the next reader does not have to rediscover it:
@cos, anytype, std.*, a frame-size limit). The 98.9% figure is coverage of the reference-passing set, not of the corpus.The conclusion is not "therefore distrust t27b." It is that a verification claim is only as good as its stated boundary, and every number above is published with the command that produced it. That habit - receipt, command, count - is portable to any compiler project, whatever its backends.
Every claim above, with the file and the command that backs it:
| Claim | Source (file) | Command |
|---|---|---|
| 0 of 8,344 tests disagree, t27b vs reference, 1,315 files | docs/reports/t27b-bench-2026-10.md (disagreements table, "Reading these numbers") | ./target/release/t27b corpus specs --blockers --reference ./target/release/t27c |
| Zig Debug vs ReleaseFast: 7 files / 11 tests; clang -O0/-O2: 39 files / 107 tests; gcc -O0/-O2: 4 files | same report, disagreements table | ./scripts/tri test |
| specs/a/b_c.t27 missing-return example | same report, "The C pipelines disagree with themselves" | ./target/release/t27b test specs/a/b_c.t27 --overflow trap |
| Corpus 1,819 files; t27b passes 1,296/1,311 (98.9%) | same report, "Denominator" and coverage tables | ./target/release/t27b corpus specs --blockers --reference ./target/release/t27c |
| DDC_BACKEND_AGREEMENT, not DDC_INDEPENDENT; filter C4 FALSE | same report, trust table (receipt #7706) | signed DDC receipt (t27b lab, commit 0e8564505) |
| Fuzzer: 7 disagreements per 1,000 programs, seed 1791653965 | same report, trust table (receipt #6442) | random-program fuzzer, t27b lab run |
| 12% self-generated share | same report, trust table | line counts: gen/rust/tri/t27b vs cli/t27b/src + bootstrap/src/compiler.rs |
| 4.8 ms/spec vs 3.05 s zig Debug; 3.08 MB binary | same report, compile-speed table and footprint table | ./target/release/t27b test <spec> --time / binary size |
| 3.4x gcc -O2 instructions, 1.8x size | same report, axis C sections and limitations list | qemu instruction-count plugin runs |
| Object only (ELF64/Mach-O), no link, no module-level vars | same report, "no ahead-of-time executable"; cli/t27b/src/codegen.rs ("no writable data section") | ./target/release/t27b --help |
Speed and size context: [One spec, five targets: the t27 backend shootout](/blog/one-spec-five-targets-the-t27-backend-shootout/). Everything else about the project: [t27.ai](https://t27.ai).
Work with me
I audit RTL and build independent, bit-exact models, then take the result through synthesis and, when useful, onto an Artix-7 board. The first conformance module is free.