T27.AI

Blog

Thirty epochs exposed a failure-rate blind spot

2026-08-20 · 6 min read

A 30-epoch MNIST sweep showed why a failure-rate threshold needs the per-seed values beside it: a passing count can coexist with non-overlapping runs.

[measured — MNIST training sweep] A thirty-epoch run was designed to test whether a failure-rate summary stayed informative as training continued. It did not: the per-seed values exposed a gap that the rate alone hides.

At 30 epochs, the five-seed slice recorded TNF4 at 0/5 failures with 97.82 ± 0.15 mean accuracy. The same slice recorded fp6 e2m3 at 4/5 failures and fp6 e3m2 at 2/5 failures. These are measurements from the MNIST experiment, not a claim about a general format ranking.

A threshold can pass while the runs do not agree

Using a 60% threshold, fp6 e3m2 passes 3/5 runs. But its highest run reaches 71.9 while TNF4’s worst reaches 97.6: the two distributions do not overlap, leaving a 25.7-point gap. A rate counts line crossings; it does not describe a distribution that is uniformly lower.

Longer training did not add the expected drift

For TNF4, the reported mean moved from 96.76 at 3 epochs to 97.68 at 10 epochs and 97.82 ± 0.15 at 30 epochs. Across eight configurations, the aggregate failure counts were TNF4 0/40, fp6 e2m3 29/40, and fp6 e3m2 24/40.

The per-seed list is the presentation here that cannot hide which runs moved.

What this does not establish

• This is a measured MNIST training sweep, not a measurement on the ALINX AX7203 FPGA and not a silicon result.
• It does not establish a universal accuracy or failure-rate ranking for TNF4, fp6 e2m3, or fp6 e3m2.
• It does not show that a 60% threshold is the right threshold for another task, seed set, or training recipe.
• The experiment does not establish transfer beyond this MNIST configuration; the per-seed list improves the audit trail but does not remove that scope limit.

The useful result is methodological: keep the failure rate for its narrow question, but print every seed beside it. That small addition turns a passing count into a checkable picture of the runs.