Eta (learning rate)
Say it: "eta"
What it means
The usual symbol for the learning rate: the size of each gradient-descent step, as in . Sometimes written .
Example
With gradient 10 and learning rate 0.1.
In code
w, lr = 4.0, 0.1 for step in range(5): grad = 2 * w # loss w**2 w -= lr * grad print(step, round(w, 4))
Output
0 3.2 1 2.56 2 2.048 3 1.6384 4 1.3107
Where it appears in machine learning
Too large and training diverges; too small and it crawls. The most important setting to tune.
Related symbols
Learn it properly in the free lesson Optimisation, with practice questions that check your answers.