Blog
Neural networks in t27: 27 lessons, and a widget in each one that runs a specs/ml spec
[every widget runs fn bodies of a gHashTag/t27 specs/ml spec compiled to wasm; lesson 10 is average pooling until t27#10612 merges; the fixtures are the specs' own, nothing is trained on real data] Course 10 opens the parts the project's agents think with: ReLU, GELU and softmax, embeddings and dropout, pooling and batch norm, MSE, cross-entropy and Huber, SGD, RMSProp, Adam and LAMB, backpropagation, a 2-8-1 network that learns XOR in the browser, a recurrent cell, LSTM gates, attention, positional encoding, an encoder block and PPO.
Course 10 of t27.ai is out: Neural networks in t27, from one neuron to a transformer block. The agents of this project think with neural networks, and the parts those networks are made of already live in gHashTag/t27 as specs with numeric tests: activations, layers, losses, optimizers, recurrent cells, attention, a transformer block. The course opens them one by one. It has 27 lessons in 9 modules, each lesson opens with a question a curious person would ask, and its widget answers it with numbers that are the spec's own fn bodies, compiled to WebAssembly and run in your browser.
Nine modules, twenty-seven widgets
| Module | Widgets | Specs in specs/ml |
|---|---|---|
| One neuron | nn-relu, nn-sigmoid, nn-tanh | relu_activation, sigmoid_activation, tanh_activation |
| New bends | nn-gelu, nn-silu, nn-elu | gelu_activation, silu_swish_activation, elu_activation |
| A layer | nn-softmax, nn-embedding, nn-dropout | softmax, embedding_layer, dropout_layer |
| Pictures | nn-conv, nn-maxpool, nn-batchnorm | avgpool2d_layer, maxpool2d_layer, batchnorm_layer |
| The error | nn-mse, nn-bce, nn-huber | mse_loss, binary_crossentropy_loss, huber_loss |
| Downhill | nn-sgd, nn-rmsprop, nn-adam | sgd, rmsprop, adam |
| Learning | nn-backprop, nn-xor, nn-lamb | dense_layer, mlp, lamb |
| Memory and attention | nn-rnn, nn-lstm, nn-attention | rnn_cell, lstm_single, attention_mechanism |
| The transformer and the reward | nn-positions, nn-encoder, nn-ppo | positional_encoding, encoder_block, ppo_clip_loss |
The pictures are what the numbers look like: activation curves you drag along with their slopes, softmax bars that sharpen as you cool the temperature, pooling windows as heat-maps, a weight walking down a bowl under SGD, RMSProp and Adam, two heads of attention as grids, and positions as the hands of four clocks.
A network that learns in your browser
Lesson 20 trains a network. mlp.t27 holds a 2-8-1 network of ReLUs and a test that it learns XOR in 2000 epochs at lr 0.05. Compiled to wasm, the same xor(epochs, which) fn runs in the widget, from scratch for every point of its loss curve. The loss falls from 0.8896 in epoch 1 to 1.8e-13, and the four answers come out within 4e-7 of 0, 1, 1 and 0. The figure below is drawn by that compiled fn.
How the specs got into the browser
No widget copies a formula. scripts/t27-logic.mjs of 999-multibots-telegraf runs t27c gen on a spec, builds the Zig with Zig 0.16 for wasm32 and writes a small module with one function per spec fn; scripts/widget-logic.mjs of the site puts it beside the widget with a byte-identical copy of the spec and stops the build when the two drift apart. The specs/ml fns take an index as usize, which t27-logic.mjs learned to lower in 999#4017; after that, 27 specs gave the 27 widgets their logic without a line rewritten. The site's spec reader learned to read an epsilon written as 1e-8 in the same change.
What it does not show
- Lesson 10 is average pooling over 2 x 2 windows, not a convolution: the sliding conv2d spec, t27#10612, is still open, and the lesson says so.
- The XOR grid shows the four inputs the spec exposes, not a decision region over the plane: the spec has no fn for a point between the corners.
- The fixtures are the specs' own small ones, 3 inputs, 4 tokens, 3 positions: nothing here was trained on real data.
Try it
- Course: t27.ai/learn/neural-networks/
- t27.ai/widgets/nn-softmax/, t27.ai/widgets/nn-xor/, t27.ai/widgets/nn-attention/, t27.ai/widgets/nn-positions/
- Specs: gHashTag/t27 specs/ml/
What this does not settle
- Lesson 10 averages each 2 x 2 window; a convolution with a learned kernel waits for t27#10612.
- The XOR widget shows the four inputs the spec exposes, not a decision region over the plane.
- Every fixture is a spec's own small one; no widget runs a model trained on real data.
Receipts
Work with me
Need an FPGA/RTL problem taken to measured hardware?
I work contract and part-time on hardware-AI, FPGA/RTL and ML systems — from specification and open toolchains to reproducible measurements.