What does a minimal PyTorch training loop need, and what do model.train() and model.eval() change?

Why

The PyTorch optimization tutorial lays out the loop in three moves inside each batch: optimizer.zero_grad() to reset gradients, because they add up by default, loss.backward() to compute new ones, and optimizer.step() to update parameters. Around that sits a loop over a DataLoader that shuffles training data each epoch. Mode switches are a frequent bug.

Dropout zeroes random units during training and, per its documentation, acts as the identity in evaluation. Batch normalization uses the current batch's statistics in training and stored running averages in evaluation. Forgetting model.eval() before validation makes predictions random from run to run and often worse.

Forgetting model.train() after validation silently trains with dropout off. The PyTorch autograd notes stress that eval mode is completely separate from no-grad mode. Wrap validation and inference in torch.no_grad() or torch.inference_mode() to skip building the graph, which saves memory and time.

Two more habits make a loop trustworthy. Move both data and model to the same device, and log training and validation loss every epoch with the random seed recorded. Calling loss.item() for logging avoids holding the graph in memory between steps.