Functions from Kinks

How ReLUs build any shape

An interactive lesson with 6 steps, about 5 minutes: you predict, drag and solve, with instant feedback on every answer. It runs in your browser and needs JavaScript.

What you’ll learn

• Each ReLU neuron adds a kink; output weights set the change of slope there.
• Sums of kinks (tents) can approximate any curve: universal approximation.
• Depth composes functions, so the number of pieces can grow exponentially.

Steps

1. One kink
2. Build a tent
3. Count the kinks
4. Depth multiplies
5. Pieces of a line
6. Tents of tents