Batch norm: features on very different scales, made comparable

A batch of 4 samples whose 3 features live on scales 1, 10 and 1: batch norm gives each feature mean 0 and variance 1. Then set gamma and the running statistics for inference.

This page sets no cookie and sends nothing anywhere.

Where the numbers come from

gHashTag/t27 specs/ml/layers/batchnorm_layer.t27 (t27 master 239b4b059): every value on the page is its fn bodies, compiled to wasm (logic.js)
Course 10 of t27.ai, neural-networks (gHashTag/trinity#1695): lesson 12 opens this widget

This widget's .t27 source · All widgets