t27.aiРусский

A tiny layer

You will learn

How several neurons reading one input describe it with a few trits.

A layer is several neurons that read the same input. Three detectors, vertical, horizontal and diagonal, turn a 3 x 3 picture into three output trits: a vertical bar gives +1, 0, 0. Stack layers and you have a network; the lesson spec is a 4-neuron layer in one combinational module, and the ai-numbers course takes it to a whole network on the board.

Try it

Draw a picture that makes two neurons fire at once; then one that makes all three answer -1.

Open the interactive lesson →

A layer of three ternary neurons
A layer of three ternary neurons ↗

Three neurons share one input: a vertical, a horizontal and a diagonal detector. Draw on the input and read three output trits.

specs/ternary/comb_bitnet_layer.t27

module CombBitnetLayer;
// #1764: a full BitNet LAYER (4 neurons) in one combinational synthesizable
// module. One 27-trit activation vector `a` feeds four neurons, each with its
// own fixed 27-trit weight vector (trained weights baked in as consts -- the
// realistic inference case), producing four trit activations packed into the
// `result` output (2 bits per trit: neuron i in result[2i+1:2i]).
//
// This is the next architectural level above the single neuron: weights are
// constants (a real trained layer), the activation is the only input port, and
// each neuron is quantize(dot27(w_i, a)) -- a weighted ternary sum then sign.
// It synthesizes to ~4x the single-neuron LUT cost, still purely combinational.
//
// Weights here are the three canonical vectors (all +1 / all -1 / all 0) plus a
// repeat, so the layer's response to any activation is hand-checkable.

// packed-trit constants: all +1, all -1, all 0.
const W_P : u64 = 12009599006321322;
const W_N : u64 = 0;
const W_Z : u64 = 6004799503160661;

fn tmul(ta: u8, tb: u8) -> i8 {
    if (ta == 1) { return 0; }
    if (tb == 1) { return 0; }
    if (ta == tb) { return 1; }
    return -1;
}
fn tp(a: u64, b: u64, i: u32) -> i8 {
    return tmul(((a >> (i << 1)) & 3) as u8, ((b >> (i << 1)) & 3) as u8);
}
fn dot27(a: u64, b: u64) -> i8 {
    return tp(a,b,0) + tp(a,b,1) + tp(a,b,2) + tp(a,b,3) + tp(a,b,4)
         + tp(a,b,5) + tp(a,b,6) + tp(a,b,7) + tp(a,b,8) + tp(a,b,9)
         + tp(a,b,10) + tp(a,b,11) + tp(a,b,12) + tp(a,b,13) + tp(a,b,14)
         + tp(a,b,15) + tp(a,b,16) + tp(a,b,17) + tp(a,b,18) + tp(a,b,19)
         + tp(a,b,20) + tp(a,b,21) + tp(a,b,22) + tp(a,b,23) + tp(a,b,24)
         + tp(a,b,25) + tp(a,b,26);
}
fn quantize(v: i8) -> u8 {
    if (v > 0) { return 2; }
    if (v < 0) { return 0; }
    return 1;
}
// One neuron: weighted ternary sum of weight w and activation a, then sign.
fn neuron(w: u64, a: u64) -> u8 {
    return quantize(dot27(w, a));
}

// The layer: 4 neurons over the shared activation `a`, trits packed 2 bits each.
fn on_comb(a: u64) -> u8 {
    return (neuron(W_P, a))
         | (neuron(W_N, a) << 2)
         | (neuron(W_Z, a) << 4)
         | (neuron(W_P, a) << 6);
}

// a = all +1: dots are +27, -27, 0, +27 -> trits P,N,Z,P = 2|0<<2|1<<4|2<<6 = 146
test layer_allp { assert_eq(on_comb(12009599006321322), 146); }
// a = all -1: dots are -27, +27, 0, -27 -> trits N,P,Z,N = 0|2<<2|1<<4|0<<6 = 24
test layer_alln { assert_eq(on_comb(0), 24); }
// a = all 0: every dot is 0 -> all Z = 1|1<<2|1<<4|1<<6 = 85
test layer_allz { assert_eq(on_comb(6004799503160661), 85); }
endmodule

Open the lesson's spec in the player ↗

All lessons