Softmax
Say it: "softmax"
What it means
Turns a list of scores into probabilities that add up to 1: .
Example
The largest score gets the largest share.
In code
import numpy as np z = np.array([2.0, 1.0, 0.0]) p = np.exp(z - z.max()) / np.exp(z - z.max()).sum() print(p.round(3), p.sum())
Output
[0.665 0.245 0.09 ] 0.9999999999999999
Where it appears in machine learning
The output layer of every classifier with several classes, and the attention weights in transformers.
Related symbols
Learn it properly in the free lesson Numerical Stability, with practice questions that check your answers.