Home / Maths for AI / Symbols / Machine learning notation

Loss function

Say it: "the loss"

What it means

A number measuring how wrong a prediction is; zero for perfect. J(θ)J(\theta) or L\mathcal{L} often denote the average loss over the dataset.

Example

MSE=1n∑i=1n(yi−y^i)2\text{MSE} = \frac{1}{n}\sum_{i=1}^{n}(y_i - \hat{y}_i)^2

Mean squared error, for regression.

In code

import numpy as np
y, y_hat = np.array([1, 0, 1]), np.array([0.9, 0.2, 0.6])
print("MSE", round(np.mean((y - y_hat) ** 2), 3), "log loss", round(-np.mean(y * np.log(y_hat) + (1 - y) * np.log(1 - y_hat)), 3))

Output

MSE 0.07 log loss 0.28

Where it appears in machine learning

Training is minimising the loss; choosing the loss decides what "good" means.

Related symbols

Learn it properly in the free lesson Optimisation, with practice questions that check your answers.