Home / Maths for AI / Symbols / Probability and statistics

Covariance matrix

Say it: "capital sigma"

What it means

For several variables, the matrix of covariances: Σij\Sigma_{ij} measures how variables ii and jj move together (diagonal entries are the variances). Not to be confused with the sum sign ∑\sum.

Example

Σ=(Var(x)Cov(x,y)Cov(x,y)Var(y))\Sigma = \begin{pmatrix} \mathrm{Var}(x) & \mathrm{Cov}(x, y) \\ \mathrm{Cov}(x, y) & \mathrm{Var}(y) \end{pmatrix}

In code

import numpy as np
rng = np.random.default_rng(0)
x = rng.normal(size=1000)
y = 2 * x + rng.normal(size=1000)
print(np.cov(x, y).round(2))

Output

[[0.96 1.97]
 [1.97 5.1 ]]

Where it appears in machine learning

PCA diagonalises it; the multivariate Gaussian N(μ,Σ)\mathcal{N}(\mu, \Sigma) is described by it.

Related symbols

Learn it properly in the free lesson Eigenvectors, Covariance & SVD, with practice questions that check your answers.