Same neuron, new weights
You will learn
How one ternary neuron computes a majority vote with one set of weights and a different function with another.
maj3 in bitnet_majority.t27 packs three trits with pack3, takes dot27 against w_all_p, an all-P weight chunk, and quantizes at 0: the result is the sign of a + b + c. weighted_vote is the same neuron with weights pack3(2, 2, 0) and gives the sign of a + b - c, so the weights pick the function. The browser skips all 12 tests because it does not know assert_eq yet; the native t27c runs all 12, all pass, none vacuous. The recording lowers w_all_p by 16, which turns the weight on lane 2 from P to Z, so maj3 no longer sees c. Exactly one test fails, maj_p_n_n: +1 - 1 - 1 is N, but +1 - 1 is Z. Note that wv_p_z_p passes 0, which is N, as its middle input. Every byte in the recording was printed by the command; only the typing is staged.
Try it
In the recording, find the expected and the actual value of maj_p_n_n; then in the spec frame find wv_p_z_p and work out what weighted_vote(2, 1, 2) would return.
t27c on the t27c lab (Railway), spec at t27 5ff0ec512: 12 tests pass natively; one maj3 weight moved from P to Z fails exactly one test, maj_p_n_n; git restores the spec.
specs/ternary/bitnet_majority.t27
module BitnetMajority;
// Sign-only ternary multiply of two packed trits {N=0b00, Z=0b01, P=0b10}.
fn tmul(ta: u8, tb: u8) -> i8 {
if (ta == 1) { return 0; }
if (tb == 1) { return 0; }
if (ta == tb) { return 1; }
return -1;
}
// 27-trit ternary dot product of two 54-bit packed vectors.
fn dot27(a: u64, b: u64) -> i16 {
var acc : i16 = 0;
var i : u32 = 0;
while (i < 27) {
var ta : u8 = ((a >> (i << 1)) & 3) as u8;
var tb : u8 = ((b >> (i << 1)) & 3) as u8;
acc = acc + tmul(ta, tb) as i16;
i = i + 1;
}
return acc;
}
// Ternary sign/threshold quantizer: v > +t -> P, v < -t -> N, else Z.
fn quantize(v: i16, threshold: i16) -> u8 {
if (v > threshold) { return 2; }
if (v < -threshold) { return 0; }
return 1;
}
// Pack 3 trits into the low 3 lanes of a chunk; the rest are Z.
fn pack3(t0: u8, t1: u8, t2: u8) -> u64 {
var z : u64 = 6004799503160661;
var cleared : u64 = z & 18446744073709551552;
return cleared | (t0 as u64) | ((t1 as u64) << 2) | ((t2 as u64) << 4);
}
// Ternary majority of three trits = sign of (a + b + c), realized as a spec-
// first single ternary neuron: pack the inputs, dot with an all-+1 weight chunk
// (so the dot product is exactly a + b + c over the three active lanes), then
// quantize at threshold 0. A recognizable named function computed by the
// BitNet stack. Inputs/output are packed trits {N=0, Z=1, P=2}.
pub fn maj3(a: u8, b: u8, c: u8) -> u8 {
var chunk : u64 = pack3(a, b, c);
var w_all_p : u64 = 12009599006321322;
return quantize(dot27(chunk, w_all_p), 0);
}
// The SAME single neuron computes a DIFFERENT named function when the weights
// change -- the essence of a trained model. With per-input weights [+1, +1, -1]
// it computes sign(a + b - c) (weight P contributes +input, weight N contributes
// -input). Demonstrates that weights define the function, not the topology.
pub fn weighted_vote(a: u8, b: u8, c: u8) -> u8 {
var chunk : u64 = pack3(a, b, c);
var weights : u64 = pack3(2, 2, 0);
return quantize(dot27(chunk, weights), 0);
}
test maj_p_p_n { assert_eq(maj3(2, 2, 0), 2); }
test maj_p_n_z { assert_eq(maj3(2, 0, 1), 1); }
test maj_n_n_p { assert_eq(maj3(0, 0, 2), 0); }
test maj_p_p_p { assert_eq(maj3(2, 2, 2), 2); }
test maj_z_z_z { assert_eq(maj3(1, 1, 1), 1); }
test maj_n_n_n { assert_eq(maj3(0, 0, 0), 0); }
test maj_p_n_n { assert_eq(maj3(2, 0, 0), 0); }
// weighted_vote = sign(a + b - c): flipping the 3rd weight to -1 changes the function.
test wv_p_p_p { assert_eq(weighted_vote(2, 2, 2), 2); }
test wv_p_z_p { assert_eq(weighted_vote(2, 0, 2), 0); }
test wv_z_z_z { assert_eq(weighted_vote(1, 1, 1), 1); }
test wv_p_n_n { assert_eq(weighted_vote(2, 0, 0), 2); }
test wv_z_z_p { assert_eq(weighted_vote(1, 1, 2), 0); }
endmodule
All lessons
Module 1 · Lab: our own research
A number format of our own, an honest scoreboard, and a model's tables multiplied on the board.
Module 2 · AI numbers: the MX block
How AI chips keep weights in a few bits: one shared scale per block, the scale byte itself, and what one outlier does to its neighbours.
Module 3 · Ternary weights
Weights that are only minus, zero or plus a scale, the five rules a ternary alphabet must pass, and a test pass that checked nothing.
Module 4 · The Ternary Network Float
A rule the compiler enforces before any test runs, and a 17-bit float whose exponent is four balanced trits.
Module 5 · Arithmetic on signed numbers
Multiply two signed numbers, add them when their signs differ, and do both at once in a multiply-accumulate.
Module 6 · Parts of a neuron
A ReLU that bends at zero, a power of two for softmax, and an argmax that names the answer.
Module 7 · Learning from a mistake
A loss that prices a wrong guess in bits, one step that moves a weight against its gradient, and the hidden layer that XOR needs.
Module 8 · BitNet: ternary networks
A threshold that squeezes a sum back to three values, one neuron that becomes a different function when its weights change, and a neuron that reads its inputs 27 trits at a time.
Module 9 · The ternary MAC as a chip
The 27-trit dot product as wires with no register, the same sum added into a register on every clock, and a small whole network to close the course.