Home / Maths for AI / Symbols / Calculus

Gradient (nabla)

Say it: "grad f", "del f", "nabla f"

What it means

The vector of all partial derivatives: ∇f=(∂f∂x1,…,∂f∂xn)\nabla f = \left(\frac{\partial f}{\partial x_1}, \dots, \frac{\partial f}{\partial x_n}\right). It points in the direction of steepest increase.

Example

f(x,y)=x2+y2  ⇒  ∇f=(2x,2y)f(x, y) = x^2 + y^2 \;\Rightarrow\; \nabla f = (2x, 2y)

At (3, 4) the gradient is (6, 8).

In code

import numpy as np
x = np.array([3.0, 4.0])
grad = 2 * x
print(grad, "step:", x - 0.1 * grad)

Output

[6. 8.] step: [2.4 3.2]

Where it appears in machine learning

Gradient descent: θ←θ−η∇θL\theta \leftarrow \theta - \eta \nabla_\theta L. Every neural network is trained this way.

Related symbols

Learn it properly in the free lesson Gradients, with practice questions that check your answers.