RMSProp: a step that does not care how big the gradient is

Feed one weight a constant gradient of 0.001, 0.5 or 1000: RMSProp divides by the root of its running mean square, so all three walk the same path. Change the decay and the path bends.

This page sets no cookie and sends nothing anywhere.

Where the numbers come from

gHashTag/t27 specs/ml/optimizer/rmsprop.t27 (t27 master 239b4b059): every value on the page is its fn bodies, compiled to wasm (logic.js)
Course 10 of t27.ai, neural-networks (gHashTag/trinity#1695): lesson 17 opens this widget

This widget's .t27 source · All widgets