Gradient Ascent & Descent

How machines learn

An interactive lesson with 6 steps, about 5 minutes: you predict, drag and solve, with instant feedback on every answer. It runs in your browser and needs JavaScript.

What you’ll learn

• Gradient ascent: $\mathbf{x} \leftarrow \mathbf{x} + \eta\nabla f$. Descent uses $-\nabla f$.
• It finds the nearest peak (or valley), which may only be local.
• Step size matters. Gradient descent is how neural networks are trained.

Steps

1. Climbing in the fog
2. Which summit?
3. One step by hand
4. Too big a step
5. Three steps down
6. Training a neural network