One step downhill

You will learn

How one SGD step moves a weight against its gradient, and why a test with two equal inputs cannot tell them apart.

gft_sgd_step.t27 moves a weight against its gradient: w' = w - eta*g, where eta is the learning rate. Its 3 tests are step, with w = 1.0, g = 0.5 and eta = 0.5; nograd, where g = 0 leaves w alone; and ascend, where g = -1.0 and eta = 1.0 push w up to 2.0. The browser skips all 3 tests because its runner does not know assert_eq yet; the native t27c runs all 3, all pass, none vacuous. The recording runs t27c test-report, then a one-line sed that multiplies g by itself instead of by eta, and t27c test-report again. In step g equals eta, and in nograd both products are 0. So exactly one test fails, ascend, where the weight falls to 0 instead of rising to 2.0. git checkout puts the line back, and sha256sum prints the same hash as before the edit. Every byte in the recording was printed by the command; only the typing is staged.

Open the interactive lesson →