Trinity S³AI
Gradient descent: how a network walks downhill
Set the learning rate and momentum and watch a weight walk from 0 to the bottom of (x - 3)^2. Plain steps creep, momentum overshoots and rings, and a rate of 1 bounces forever.
Trinity S³AI
Set the learning rate and momentum and watch a weight walk from 0 to the bottom of (x - 3)^2. Plain steps creep, momentum overshoots and rings, and a rate of 1 bounces forever.