t27.aiРусский

A model's tables on the board

You will learn

How the host feeds the board over a serial bridge, why the window size matters, and what the board computed.

The board does the arithmetic; the host sends it work over a USB serial bridge. The board's answers wait in the bridge chip's small receive buffer, and when the host keeps more requests in flight than their answers fit, answers are lost. A smaller window removed the loss and did not slow the run. Over that link the board multiplied every ternary weight matrix of a small language model, and every row matched the reference bit for bit. The text itself is generated on the Mac, which does everything except the ternary dot products: the board is the calculator, not the whole model.

Try it

Drag the window until the bucket overflows and note the largest window that fits; then compare the two real runs.

Open the interactive lesson →

The bucket under the tap: a 512-byte UART buffer and 19-byte answers
The bucket under the tap: a 512-byte UART buffer and 19-byte answers ↗

All lessons