Home / Maths for AI / Symbols / Machine learning notation

Eta (learning rate)

Say it: "eta"

What it means

The usual symbol for the learning rate: the size of each gradient-descent step, as in θ←θ−η∇L\theta \leftarrow \theta - \eta \nabla L. Sometimes written α\alpha.

Example

θnew=4−0.1×10=3\theta_{\text{new}} = 4 - 0.1 \times 10 = 3

With gradient 10 and learning rate 0.1.

In code

w, lr = 4.0, 0.1
for step in range(5):
    grad = 2 * w          # loss w**2
    w -= lr * grad
    print(step, round(w, 4))

Output

0 3.2
1 2.56
2 2.048
3 1.6384
4 1.3107

Where it appears in machine learning

Too large and training diverges; too small and it crawls. The most important setting to tune.

Related symbols

Learn it properly in the free lesson Optimisation, with practice questions that check your answers.