Back to three values
You will learn
How a threshold turns a neuron's sum back into a trit, and why each edge of the dead band needs its own test.
A neuron in a ternary network adds up products of trits, and the sum can be any small integer. quantize in activation_quantizer.t27 squeezes it back: above the threshold it returns 2 (P), below minus the threshold 0 (N), and in between 1 (Z). The on_comb at the end makes it combinational: no register, the output follows the inputs. The browser skips all 7 tests because its runner does not know assert_eq yet; the native t27c runs all 7, all pass, none vacuous. The recording changes > to >= on line 6, so a sum equal to the threshold already counts as P. Exactly one test fails, quantize_boundary_hi, which asks quantize(10, 10) for Z. Every byte in the recording was printed by the command; only the typing is staged.
Try it
In the recording, find the changed line and the test that fails; then in the spec frame find quantize_boundary_lo and work out which one-character change on line 7 it would catch.
t27c on the t27c lab (Railway), spec at t27 5ff0ec512: 7 tests pass natively; one > in the quantizer becoming >= fails exactly one test, quantize_boundary_hi; git restores the spec.
specs/ternary/activation_quantizer.t27
module ActivationQuantizer;
// Ternary activation quantizer for the BitNet engine: re-ternarize a signed
// accumulator to a packed trit {N=0b00, Z=0b01, P=0b10} using a symmetric
// threshold. v > +t -> P, v < -t -> N, else Z.
pub fn quantize(v: i16, threshold: i16) -> u8 {
if (v > threshold) { return 2; }
if (v < -threshold) { return 0; }
return 1;
}
test quantize_positive { assert_eq(quantize(100, 10), 2); }
test quantize_negative { assert_eq(quantize(-100, 10), 0); }
test quantize_zero_band { assert_eq(quantize(5, 10), 1); }
test quantize_boundary_hi { assert_eq(quantize(10, 10), 1); }
test quantize_boundary_lo { assert_eq(quantize(-10, 10), 1); }
test quantize_just_over { assert_eq(quantize(11, 10), 2); }
test quantize_just_under { assert_eq(quantize(-11, 10), 0); }
// W696: the hardware boundary, DERIVED -- not chosen.
//
// T187 measured an exact equivalence over 617 specs: a module gets a data
// port iff the spec declares `on_comb` or `on_clock`. Without one the
// compiler emits `NO DATA PORTS -- this module cannot move a value across
// its boundary`, and synthesis optimises the whole thing away.
//
// The standing rule is that the default must NOT be guessed. Here no guess
// was made: `t27c entry-points` found exactly ONE function in this spec that
// takes a parameter, returns a value, has a body, and whose types all have a
// known width. With one candidate the choice is forced, so this forwards and
// invents nothing. 11 of 387 port-less specs qualified.
fn on_comb(v: i16, threshold: i16) -> u8 { return quantize(v, threshold); }
endmodule
All lessons
Module 1 · Lab: our own research
A number format of our own, an honest scoreboard, and a model's tables multiplied on the board.
Module 2 · AI numbers: the MX block
How AI chips keep weights in a few bits: one shared scale per block, the scale byte itself, and what one outlier does to its neighbours.
Module 3 · Ternary weights
Weights that are only minus, zero or plus a scale, the five rules a ternary alphabet must pass, and a test pass that checked nothing.
Module 4 · The Ternary Network Float
A rule the compiler enforces before any test runs, and a 17-bit float whose exponent is four balanced trits.
Module 5 · Arithmetic on signed numbers
Multiply two signed numbers, add them when their signs differ, and do both at once in a multiply-accumulate.
Module 6 · Parts of a neuron
A ReLU that bends at zero, a power of two for softmax, and an argmax that names the answer.
Module 7 · Learning from a mistake
A loss that prices a wrong guess in bits, one step that moves a weight against its gradient, and the hidden layer that XOR needs.
Module 8 · BitNet: ternary networks
A threshold that squeezes a sum back to three values, one neuron that becomes a different function when its weights change, and a neuron that reads its inputs 27 trits at a time.
Module 9 · The ternary MAC as a chip
The 27-trit dot product as wires with no register, the same sum added into a register on every clock, and a small whole network to close the course.