Home / Maths for AI / Symbols / Probability and statistics

Likelihood

Say it: "the likelihood of theta"

What it means

How probable the observed data is, as a function of the parameters: L(θ)=P(data∣θ)\mathcal{L}(\theta) = P(\text{data} \mid \theta). Training by maximum likelihood picks arg⁡max⁡θL(θ)\arg\max_\theta \mathcal{L}(\theta), usually via the log-likelihood.

Example

7 heads in 10 flips: L(p)∝p7(1−p)3, maximised at p=0.77 \text{ heads in } 10 \text{ flips: } \mathcal{L}(p) \propto p^7 (1 - p)^3, \text{ maximised at } p = 0.7

In code

import numpy as np
p = np.linspace(0.01, 0.99, 99)
L = p ** 7 * (1 - p) ** 3
print(round(p[L.argmax()], 2))

Output

0.7

Where it appears in machine learning

Minimising cross-entropy loss is the same as maximising the likelihood of the labels.

Related symbols

Learn it properly in the free lesson Likelihood, MLE & MAP, with practice questions that check your answers.