Backpropagation

Definition

Backpropagation computes gradients of the loss with respect to every weight using the chain rule.

It is how every neural network learns.

Diagram

Flowchart

Example: One-weight numerical

x, y, w, lr = 2, 8, 1, 0.1
for _ in range(3):
    pred = w * x
    grad = 2 * (pred - y) * x
    w -= lr * grad
print(round(w, 3))

Output: 3.976

Key points

• Gradients flow from output to input
• Frameworks do it automatically (autograd)

Interview questions

More AI topics