What does backpropagation compute in a neural network?

Why

A network is a chain of simple operations, each with a known derivative. The Deep Learning book by Goodfellow, Bengio and Courville describes back-propagation as an algorithm that computes the chain rule with a specific, highly efficient order of operations. In the forward pass the network computes predictions and the loss and stores intermediate values.

In the backward pass it starts from the loss and works back layer by layer, multiplying local derivatives, so each weight learns how much a small change in it would change the loss. Backpropagation itself does not learn anything. It only supplies gradients, and an optimizer such as SGD or Adam turns them into updates.

In PyTorch, autograd records operations on tensors as a graph, and calling loss.backward() fills each parameter's .grad attribute. The PyTorch tutorial points out a practical consequence: gradients accumulate across backward calls, so they must be zeroed each iteration, which is what optimizer.zero_grad() does. Gradients also explain many training failures.

Repeated multiplication through many layers can shrink them toward zero or blow them up, which is why activation choice, initialization, normalization and residual connections matter.