Two to the x
You will learn
How exp2 splits x into a whole part and a fraction, and why its tests only reach the whole part.
Recording pending: it waits on tri test and tri mutate plant from gHashTag/t27#7400, the two commands the recording runs, and until then the widget below is a placeholder that shows no run. The header of gft_exp2.t27 calls exp2 the missing building block for a GF-T softmax. on_comb splits x into a whole part k and a fraction f: k sets the offset, k + 40, and pow2_frac turns f into the mantissa. The header claims at most 1 ULP of error, measured in the prototype, but all 4 tests use whole inputs, 0, 1.0, -1.0 and 2.0, so f is 0 and pow2_frac returns 0 whatever its coefficients. The browser skips all 4 tests because its runner does not know assert_eq yet; the native t27c runs all 4, all pass, none vacuous. The recording drops the minus sign of k for negative x, and exactly one test fails, em1: 2^-1 comes out as 2.0, not 0.5. Every byte in the recording was printed by the command; only the typing is staged.
Try it
In the recording, find the changed line and the value em1 expects; then in the spec frame find pow2_frac and work out what it returns when f is 0.
Recording pending: waits on tri test and tri mutate plant from gHashTag/t27#7400. Until then this page is a placeholder and shows no run.
specs/ternary/gft_exp2.t27
module GftExp2;
// #1764 + GF-T: a GF-T exp2 primitive -- 2^x for a signed GF-T16 input x, result
// a positive GF-T16. This is the missing building block for a GF-T softmax
// (softmax(l) = 2^l / sum 2^l, in base-2). Approach, all integer / hardware-shaped:
// 1. |x| -> Q16.16 fixed-point (positive shifts only -> no signed-shift ambiguity).
// 2. split into integer k = floor(x) and fraction f in [0,1) (sign-aware floor).
// 3. 2^x = 2^k * 2^f. 2^k is an exact GF-T offset (k+40); 2^f-1 gives the mantissa
// via a Q16 quartic with rounded Horner shifts (<=1 ULP vs the exact mantissa).
// 4. clamp/saturate the offset to [1,80]; sign is always + (2^x > 0).
// Accuracy: <=1 ULP vs the true round_to_GFT(2^x) over the logit range (measured in
// the prototype). Bit-exact to the committed integer oracle over the vector file.
//
// Input: x signed GF-T16 (u32). Output: 2^x as a positive GF-T16 (u32).
// mant(f) = round(512*(2^(f/65536) - 1)), f in [0,65536), via a Q16 quartic with
// rounded Horner shifts. Coefficients fit + local-searched to <=1 ULP.
fn pow2_frac(f: i32) -> i32 {
var p : i32 = 6;
p = (((p * f) + 32768) >> 16) + 29;
p = (((p * f) + 32768) >> 16) + 123;
p = (((p * f) + 32768) >> 16) + 354;
p = ((p * f) + 32768) >> 16;
if (p > 511) { p = 511; }
return p;
}
fn on_comb(x: u32) -> u32 {
if (x == 0) { return 20480; } // 2^0 = +1.0
var neg : i32 = 0;
if ((x >> 16) == 1) { neg = 1; } // sign bit set -> x is negative
var off_in : i32 = ((x >> 9) & 127) as i32;
var mant_in : i32 = (x & 511) as i32;
// absurd-magnitude guard (keeps intermediates in i32; |x| >= 2^8 -> sat/underflow).
if (off_in >= 48) {
if (neg == 1) { return 512; } // underflow -> smallest normal (2^-39)
return ((80 << 9) | 511) as u32; // saturate -> max magnitude
}
// |x| in Q16.16 (positive shifts only).
var num : i32 = 512 + mant_in;
var sh : i32 = off_in - 33;
var mq : i32 = num;
if (sh >= 0) { mq = num << sh; } else { mq = num >> (0 - sh); }
var ki : i32 = mq >> 16;
var ff : i32 = mq & 65535;
// sign-aware floor: k = floor(x), f = x - k in [0,1).
var k : i32 = ki;
var f : i32 = ff;
if (neg == 1) {
if (ff == 0) { k = 0 - ki; f = 0; }
else { k = 0 - (ki + 1); f = 65536 - ff; }
}
var mant : i32 = pow2_frac(f);
var off : i32 = k + 40;
if (off < 1) { return 512; } // underflow -> smallest normal
if (off > 80) { return ((80 << 9) | 511) as u32; }
return ((off << 9) | mant) as u32;
}
// 2^0 = 1.0 (0x5000).
test e0 { assert_eq(on_comb(0), 20480); }
// 2^(+1.0) = 2.0 : x=+1.0=0x5000|... actually x encodes the exponent value 1.0 -> 20480.
test e1 { assert_eq(on_comb(20480), 20992); } // 2.0 = offset41
// 2^(-1.0) = 0.5 = offset39 (0x4e00).
test em1 { assert_eq(on_comb(86016), 19968); } // 0x4e00
// 2^(+2.0) = 4.0 = offset42 (0x5400).
test e2 { assert_eq(on_comb(20992), 21504); } // 0x5400
endmodule
All lessons
Module 1 · Lab: our own research
A number format of our own, an honest scoreboard, and a model's tables multiplied on the board.
Module 2 · AI numbers: the MX block
How AI chips keep weights in a few bits: one shared scale per block, the scale byte itself, and what one outlier does to its neighbours.
Module 3 · Ternary weights
Weights that are only minus, zero or plus a scale, the five rules a ternary alphabet must pass, and a test pass that checked nothing.
Module 4 · The Ternary Network Float
A rule the compiler enforces before any test runs, and a 17-bit float whose exponent is four balanced trits.
Module 5 · Arithmetic on signed numbers
Multiply two signed numbers, add them when their signs differ, and do both at once in a multiply-accumulate.
Module 6 · Parts of a neuron
A ReLU that bends at zero, a power of two for softmax, and an argmax that names the answer.
Module 7 · Learning from a mistake
A loss that prices a wrong guess in bits, one step that moves a weight against its gradient, and the hidden layer that XOR needs.
Module 8 · BitNet: ternary networks
A threshold that squeezes a sum back to three values, one neuron that becomes a different function when its weights change, and a neuron that reads its inputs 27 trits at a time.
Module 9 · The ternary MAC as a chip
The 27-trit dot product as wires with no register, the same sum added into a register on every clock, and a small whole network to close the course.