Five rules for a weight alphabet

You will learn

What values a ternary weight may take, how five measured rules leave one formula, and what a sixth rule removes from it.

Try it

Run the tests; then change MAX_TRITS to 3 in the source and see which two checks fail. In the spec frame, trits size a float exponent, not a weight: find exp_values of tnf4 and the test that says which offset is not finite.

specs/numeric/tnf4.t27

// SPDX-License-Identifier: Apache-2.0
// tnf4.t27 -- TNF4: Ternary Network Float, 4-bit class (bottom rung).
// Balanced-ternary exponent = 2 trits (3^2 = 9 offsets, exponent +-4, ~2.4
// decades), 1-bit mantissa, NO regime decode.
//   layout: [ sign(1) | E = 2 balanced-ternary trits | M = 1 bit ]
//
// Positioning (the 4-bit rung is unique): the format-to-beat here is NOT tekum
// but **BitNet-1.58 ternary WEIGHTS** {-1,0,+1} -- a weight quantizer, not a real
// format. TNF4 is a genuine 4-bit REAL number (magnitude + ~2.4-decade range),
// so it plays the ACTIVATION/value role next to BitNet's weight role, exactly as
// tri_compute_bitnet.t27 attests (ternary weights x GF/TF activations). Against
// the binary 4-bit leader MXFP4 it trades block-scale range for a native ternary
// exponent and no block-decode. phi^2 + 1/phi^2 = 3 | TRINITY

module triformat_tnf4 {
    use base::types;

    const EXP_TRITS: u32 = 2;
    const MANT_BITS: u32 = 1;
    const EXP_OFFSET: u32 = 4;    // (3^2 - 1) / 2
    const OFFSET_MAX: u32 = 8;    // 3^2 - 1 (reserved special row)

    // 2-trit exponent field -> unsigned offset in [0,8].
    fn exp_offset(t0: u32, t1: u32) -> u32 {
        return t0 + (3 * t1);
    }

    fn is_finite(offset: u32) -> bool {
        return offset != OFFSET_MAX;
    }

    fn exp_values() -> u32 {
        return 9;
    }

    // ---- Tests / invariants ----
    test offset_range {
        assert(exp_offset(0, 0) == 0, "min offset (exponent -4)");
        assert(exp_offset(2, 2) == 8, "max offset (reserved special)");
        assert(exp_offset(1, 1) == 4, "center offset = exponent 0 (unity)");
    }
    test finiteness {
        assert(is_finite(4) == true, "unity finite");
        assert(is_finite(8) == false, "offset 8 special");
        assert(exp_values() == 9, "3^2 exponent values (~2.4 decades) in 2 trits");
    }
}

Open the lesson's spec in the player ↗

All lessons

• Module 1 · Lab: our own research

  A number format of our own, an honest scoreboard, and a model's tables multiplied on the board.

  1. 01 T27: a number format of our own
  2. 02 An honest scoreboard
  3. 03 A model's tables on the board

• Module 2 · AI numbers: the MX block

  How AI chips keep weights in a few bits: one shared scale per block, the scale byte itself, and what one outlier does to its neighbours.

  1. 04 One scale for a block of weights
  2. 05 The scale byte on a real machine
  3. 06 One outlier, many zeros

• Module 3 · Ternary weights

  Weights that are only minus, zero or plus a scale, the five rules a ternary alphabet must pass, and a test pass that checked nothing.

  1. 07 Weights of minus phi, zero and plus phi
  2. 08 A pass that checked nothing
  3. 09 Five rules for a weight alphabet

• Module 4 · The Ternary Network Float

  A rule the compiler enforces before any test runs, and a 17-bit float whose exponent is four balanced trits.

  1. 10 Blocked is not failed
  2. 11 A float whose exponent is four trits
  3. 12 Negate twice, get the number back

• Module 5 · Arithmetic on signed numbers

  Multiply two signed numbers, add them when their signs differ, and do both at once in a multiply-accumulate.

  1. 13 A sign is one bit
  2. 14 Opposite signs subtract
  3. 15 Multiply, then add

• Module 6 · Parts of a neuron

  A ReLU that bends at zero, a power of two for softmax, and an argmax that names the answer.

  1. 16 A bend at zero
  2. 17 Two to the x
  3. 18 Pick the largest

• Module 7 · Learning from a mistake

  A loss that prices a wrong guess in bits, one step that moves a weight against its gradient, and the hidden layer that XOR needs.

  1. 19 A loss in bits
  2. 20 One step downhill
  3. 21 XOR needs a bend

• Module 8 · BitNet: ternary networks

  A threshold that squeezes a sum back to three values, one neuron that becomes a different function when its weights change, and a neuron that reads its inputs 27 trits at a time.

  1. 22 Back to three values
  2. 23 Same neuron, new weights
  3. 24 A neuron in chunks

• Module 9 · The ternary MAC as a chip

  The 27-trit dot product as wires with no register, the same sum added into a register on every clock, and a small whole network to close the course.

  1. 25 A dot product made of wires
  2. 26 Add it up on every clock
  3. 27 A whole network