What does gradient clipping solve and what does it not solve?

Why

In a recurrent model of longitudinal biomarker trajectories, backpropagation through many steps can yield a few updates with enormous gradient norms. Norm clipping rescales gradients above a threshold, reducing the size of those excursions. Inspect the unclipped norm and fraction of steps clipped, rather than assuming every update should hit the bound.

Combine the intervention with reasonable learning rates, input scales and numerical checks. PyTorch clip_grad_norm_ returns the total norm. error_if_nonfinite can flag NaN and infinity instead of silently masking them. Overly aggressive clipping makes training slow or prevents useful learning, and clipping gradients does not guarantee stable parameter updates for every optimizer.

It also cannot rescue a model trained on biologically confounded labels. Evaluate held-out performance after stabilizing the optimizer.