Mean squared error
You will learn
How MSE turns every miss into one number, why it chases large errors, and where its gradient points.
A loss turns all the misses of a model into one number to push down. Mean squared error averages the squared misses: predictions 1.5, 2 and 2 against targets 1, 2 and 3 cost (0.25 + 0 + 1) / 3 = 0.4167. Squaring makes a miss of 2 cost 4 times a miss of 1, so MSE chases large errors. Its gradient for each prediction is 2 (p - t) / n, here 0.333, 0 and -0.667: each prediction is pushed back toward its target. As a function of one prediction the loss is a parabola, a bowl with one bottom. The spec mse_loss.t27 computes the loss and the gradient.
Try it
Move prediction 2 onto its target and read the loss; then drag along the bowl and find its bottom.
Move three predictions against the targets 1, 2 and 3: the loss is the mean of the squared misses, its bowl is a parabola, and each gradient points its prediction back at the target.
specs/ml/loss/mse_loss.t27
// SPDX-License-Identifier: Apache-2.0
// t27/specs/
// Penalizes large errors more heavily | φ² + 1/φ² = 3 | TRINITY
// Restored 2026-10-05 from trinity specs/algo/mse_loss.tri: the port kept only
// the first parameter and returned void, so forward computed a value and
// threw it away, and every test asserted `result != undefined`.
module MseLoss;
use base::types;
use math::constants;
// ═══════════════════════════════════════════════════════════
// 2. Types
// ═══════════════════════════════════════════════════════════
// reduction: "mean", "sum" or "none" (from the .tri). forward below is the
// .tri's mean; the field is carried, not yet read.
pub const MSELossConfig = struct {
reduction : []const u8,
};
// ═══════════════════════════════════════════════════════════
// 3. Core Functions
// ═══════════════════════════════════════════════════════════
// forward(predictions, targets) -> mean((predictions - targets)^2); 0 for n = 0
fn forward(predictions: []const f32, targets: []const f32) -> f32 {
if (predictions.len == 0) {
return 0.0;
}
var sum_loss : f32 = 0.0;
var i : usize = 0;
while (i < predictions.len) {
const diff = predictions[i] - targets[i];
sum_loss = sum_loss + diff * diff;
i = i + 1;
}
return sum_loss / (predictions.len as f32);
}
// backward: grad_input[i] = 2 (predictions[i] - targets[i]) / n
fn backward(predictions: []const f32, targets: []const f32, grad_input: []f32) -> void {
if (predictions.len == 0) {
return;
}
const scale = 2.0 / (predictions.len as f32);
var i : usize = 0;
while (i < predictions.len) {
grad_input[i] = scale * (predictions[i] - targets[i]);
i = i + 1;
}
}
// ═══════════════════════════════════════════════════════════
// TDD: Tests
// ═══════════════════════════════════════════════════════════
fn mse_at(p0: f32, p1: f32, p2: f32) -> f32 {
const predictions : [3]f32 = [p0, p1, p2];
const targets : [3]f32 = [1.0, 2.0, 3.0];
return forward(&predictions, &targets);
}
fn mse_grad_at(p0: f32, p1: f32, p2: f32, index: usize) -> f32 {
const predictions : [3]f32 = [p0, p1, p2];
const targets : [3]f32 = [1.0, 2.0, 3.0];
var grads : [3]f32 = [0.0] ** 3;
backward(&predictions, &targets, &grads);
return grads[index];
}
fn mse_empty() -> f32 {
const none : [0]f32 = [];
return forward(&none, &none);
}
test forward_zero_on_exact_match
given result = mse_at(1.0, 2.0, 3.0)
when also = mse_empty()
then result == 0.0 and also == 0.0
// errors (0.5, -1, 2): (0.25 + 1 + 4) / 3 = 1.75
test forward_basic_case
given result = mse_at(1.5, 1.0, 5.0)
when flipped = mse_at(0.5, 3.0, 1.0)
then @abs(result - 1.75) < 0.000001 and @abs(flipped - result) < 0.000001
test backward_matches_finite_difference
given slope = (mse_at(1.5, 1.001, 5.0) - mse_at(1.5, 0.999, 5.0)) / 0.002
when result = mse_grad_at(1.5, 1.0, 5.0, 1)
then @abs(result - slope) < 0.001 and @abs(result - (-2.0 / 3.0)) < 0.000001
// ═══════════════════════════════════════════════════════════
// TDD: Invariants
// ═══════════════════════════════════════════════════════════
invariant mse_zero_at_target
assert mse_at(1.0, 2.0, 3.0) == 0.0
All lessons
Module 1 · One neuron
A neuron sums its inputs and bends the sum: ReLU, the sigmoid and tanh, and where each one stops passing a gradient back.
Module 2 · New bends
The smooth curves of current models: GELU with its dip, Swish with its knob, and ELU with a floor instead of a cliff.
Module 3 · A layer
What sits around the neurons: softmax turns scores into a choice, an embedding turns a word into numbers, dropout switches units off.
Module 4 · Pictures
Reading a picture in patches: a window averaged over its patch, max pooling, and batch normalisation of features on different scales.
Module 5 · The error
How wrong a guess is, as one number: mean squared error, cross-entropy for probabilities, and the Huber loss for outliers.
Module 6 · Downhill
Walking down the loss: gradient descent with momentum, RMSProp that ignores the gradient's size, and Adam that combines them.
Module 7 · Learning
Where the gradients come from and what they achieve: backpropagation through a layer, a network that learns XOR, and LAMB for large batches.
Module 8 · Memory and attention
Networks that read a sequence: a recurrent cell and its fading echo, the gates of an LSTM, and attention over every position at once.
Module 9 · The transformer and the reward
The parts of a transformer and of an agent: positions as clocks, one encoder block, and PPO, which learns from reward without lurching.