t27.aiРусский

A model's tables on the board

You will learn

How the host feeds the board over a serial bridge, why the window size matters, and what the board computed.

The board does the arithmetic; the host sends it work over a USB serial bridge. The board's answers wait in the bridge chip's small receive buffer, and when the host keeps more requests in flight than their answers fit, answers are lost. A smaller window removed the loss and did not slow the run. Over that link the board multiplied every ternary weight matrix of a small language model, and every row matched the reference bit for bit. The text itself is generated on the Mac, which does everything except the ternary dot products: the board is the calculator, not the whole model.

Try it

Drag the window until the bucket overflows and note the largest window that fits; then compare the two real runs. In the spec frame, weighted applies a weight by its sign alone; find apply_alpha, where each layer multiplies once, and the test that shows it rounds down.

Open the interactive lesson →

The bucket under the tap: a 512-byte UART buffer and 19-byte answers
The bucket under the tap: a 512-byte UART buffer and 19-byte answers ↗

Drag the number of requests in flight and watch the answers fill a CP2102N's 512-byte receive buffer, then read the measured runs on the AX7203 board.

specs/igla/race/unit_weights_node.t27

// SPDX-License-Identifier: Apache-2.0
// t27/specs/igla/race/unit_weights_node.t27
// The CONTROL for T93: a ternary node on the unit alphabet {-1,0,+1}, built to
// be measured beside specs/igla/race/ternary_node.t27.
//
// WHY THIS EXISTS.  T93 measured the phi node at ZERO DSPs and stated, in the
// same breath, that the figure is a measurement of ONE SIDE:
//
//     "If a {-1,0,+1} node with a per-layer scale synthesised to zero DSPs at
//      comparable accuracy, the closure argument would be decorative rather than
//      load-bearing. That comparison has NOT been run here."
//
// This file is the other side. It is written to make the comparison FAIR, not to
// make the phi node win: same interface, same slicer, same accumulator width,
// same fault handling. The ONLY difference is the one the argument is about.
//
// THE DIFFERENCE, stated precisely.  With weights in {-1,0,+1} the gain of a
// layer is 1. Multiplying a value by 1 tells you nothing about the layer, so the
// layer's magnitude has to come from somewhere else, and every published ternary
// method puts it in a LEARNED REAL SCALE alpha_L applied once per layer:
//
//     y_L = alpha_L * sum_i ( w_i * x_i ),   w_i in {-1,0,+1}
//
// The sum is cheap -- a sign select and an adder tree, exactly as cheap as the
// phi node's. The cost is `alpha_L *`, and that is a MULTIPLY. On this fabric a
// multiply is a DSP48 or a LUT-built multiplier, and it is per layer, not per
// weight, so it is small -- but it is not zero, and zero is what the phi node
// measures.
//
// FAIRNESS CAVEAT, stated BEFORE the measurement rather than after.  This
// compares AREA at equal structure. It does NOT compare ACCURACY. A unit-alphabet
// network with a trained alpha_L may reach accuracy a phi network does not, or
// vice versa; nothing here measures that, and the article's own perplexity tables
// are the place that argument belongs. **An area comparison at unequal accuracy
// is not a verdict.**
//
// alpha is carried as Q8.8 fixed point (256 == 1.0), the smallest representation
// that makes the multiply real rather than a shift.
//
// phi^2 + 1/phi^2 = 3 | TRINITY

module IglaRaceUnitWeightsNode {
    use base::types;

    // ---- The unit alphabet: same codes, different meaning ----
    // Identical wire encoding to ternary_node.t27 -- the codes are not the
    // subject of the comparison, the interpretation is.
    const SYM_ZERO  : u8 = 0;   // 0
    const SYM_POS   : u8 = 1;   // +1   (ternary_node reads this as +phi)
    const SYM_NEG   : u8 = 2;   // -1   (ternary_node reads this as -phi)
    const SYM_ERROR : u8 = 3;

    // ---- The learned per-layer scale, Q8.8 ----
    const ALPHA_ONE   : i32 = 256;   // 1.0
    const ALPHA_SHIFT : i32 = 8;
    // A representative trained value: 1.375 = 352/256. Chosen non-trivial on
    // purpose -- a power of two would collapse the multiply to a shift and
    // quietly make the control cheaper than any real network.
    const ALPHA_TRAINED : i32 = 352;

    // ---- Stage 1: receive.  Identical to ternary_node's slicer. ----
    fn rx_slice(hi: u8, lo: u8) -> u8 {
        if (hi == 1) {
            if (lo == 1) { return SYM_POS; } else { return SYM_ERROR; }
        } else {
            if (lo == 1) { return SYM_ZERO; } else { return SYM_NEG; }
        }
    }

    fn rx_is_fault(hi: u8, lo: u8) -> bool {
        return (rx_slice(hi, lo) == SYM_ERROR);
    }

    // ---- Stage 2: weight application is a SIGN SELECT ----
    // This is exactly as cheap as the phi node's Fibonacci step. The unit
    // alphabet is not expensive here, and saying otherwise would be a strawman.
    fn weighted(sym: u8, act: i32) -> i32 {
        if (sym == SYM_POS) { return act; } else {
        if (sym == SYM_NEG) { return (0 - act); } else {
                              return 0;
        } }
    }

    // ---- Stage 3: accumulate.  Integer addition, same as the phi node. ----
    fn acc_add(acc: i32, contrib: i32) -> i32 {
        return (acc + contrib);
    }

    // ---- Stage 4: THE DIFFERENCE -- apply the learned scale ----
    // y = (acc * alpha) >> 8.  This is the multiply the phi node does not have,
    // because phi^k = F_k*phi + F_{k-1} is a pair of integers that is READ.
    fn apply_alpha(acc: i32, alpha: i32) -> i32 {
        return ((acc * alpha) >> ALPHA_SHIFT);
    }

    // ---- End to end: one MAC step plus the layer scale ----
    fn node_step(hi: u8, lo: u8, act: i32, acc: i32) -> i32 {
        return acc_add(acc, weighted(rx_slice(hi, lo), act));
    }

    fn layer_output(acc: i32) -> i32 {
        return apply_alpha(acc, ALPHA_TRAINED);
    }

    // ---- Combinational port surface (T81) ----
    // One step THROUGH the scale, so the multiply is on the measured path.
    fn on_comb(hi: u8, lo: u8, act: i32, acc: i32) -> i32 {
        return apply_alpha(node_step(hi, lo, act, acc), ALPHA_TRAINED);
    }

    // ========================================================================
    // Tests
    // ========================================================================

    // ---- The slicer, identical to the phi node's ----

    test rx_both_high_is_pos
        given s = rx_slice(1, 1)
        then s == 1

    test rx_lo_only_is_zero
        given s = rx_slice(0, 1)
        then s == 0

    test rx_neither_is_neg
        given s = rx_slice(0, 0)
        then s == 2

    test rx_crossed_is_a_fault
        given f = rx_is_fault(1, 0)
        then f == true

    // ---- Weight application is a sign select ----

    test pos_weight_passes_the_activation
        given r = weighted(1, 7)
        then r == 7

    test neg_weight_negates_the_activation
        given r = weighted(2, 7)
        then r == -7

    test zero_weight_contributes_nothing
        given r = weighted(0, 7)
        then r == 0

    // ---- Accumulation ----

    test accumulate_adds
        given r = acc_add(100, 23)
        then r == 123

    test accumulate_handles_negatives
        given r = acc_add(100, -23)
        then r == 77

    // ---- The scale: this is what the phi node does not need ----

    // alpha = 1.0 is the identity: (100 * 256) >> 8 == 100
    test alpha_one_is_the_identity
        given r = apply_alpha(100, 256)
        then r == 100

    // alpha = 1.375: (100 * 352) >> 8 == 35200 >> 8 == 137
    test alpha_trained_scales
        given r = apply_alpha(100, 352)
        then r == 137

    // alpha applied to zero is zero
    test alpha_on_zero
        given r = apply_alpha(0, 352)
        then r == 0

    // The scale ROUNDS DOWN -- a real cost the phi node also does not pay,
    // because its inter-layer gain is exact.
    test alpha_truncates
        given r = apply_alpha(1, 352)
        then r == 1

    // ---- End to end ----

    test end_to_end_pos
        given r = node_step(1, 1, 10, 0)
        then r == 10

    test end_to_end_neg
        given r = node_step(0, 0, 10, 0)
        then r == -10

    test end_to_end_zero_is_a_skip
        given r = node_step(0, 1, 10, 42)
        then r == 42

    // (0 + 100) * 352 >> 8 == 137
    test layer_output_applies_the_scale
        given r = layer_output(100)
        then r == 137

    // ---- Port surface: the multiply is on the measured path ----

    // node_step(1,1,100,0) == 100, then * 352 >> 8 == 137
    test comb_surface_runs_step_and_scale
        given r = on_comb(1, 1, 100, 0)
        then r == 137

    test comb_surface_zero_weight_still_scales
        given r = on_comb(0, 1, 100, 100)
        then r == 137

    // ========================================================================
    // Invariants
    // ========================================================================

    invariant codes_match_the_phi_node
        assert SYM_POS == 1

    invariant neg_code_matches
        assert SYM_NEG == 2

    invariant zero_code_matches
        assert SYM_ZERO == 0

    invariant fault_is_outside_the_alphabet
        assert SYM_ERROR > SYM_NEG

    // Q8.8: unity is 1 << 8.
    invariant alpha_unity_is_one_shifted
        assert ALPHA_ONE == 256

    invariant alpha_shift_matches_unity
        assert ALPHA_SHIFT == 8

    // The trained alpha must NOT be a power of two, or the multiply collapses to
    // a shift and the control becomes cheaper than any real network -- which
    // would make the comparison flattering rather than fair.
    invariant trained_alpha_is_not_a_power_of_two
        assert ALPHA_TRAINED != 256

    invariant trained_alpha_is_not_two
        assert ALPHA_TRAINED != 512

    // alpha > unity, so the scale is a genuine gain rather than an attenuation
    // that a shift could approximate.
    invariant trained_alpha_exceeds_unity
        assert ALPHA_TRAINED > ALPHA_ONE
}

Open the lesson's spec in the player ↗

All lessons