Question 1 of 6 Fundamentals Concept
Why do neural networks need a separate validation set?
Short answer
Training data updates model parameters, validation data guides selection and tuning, and a held-out test set estimates final generalization after those decisions.
Why
A model predicting drug response may appear accurate when repeated measurements from the same patient are split across training and validation. That is leakage: the model can recognize patient-specific patterns without generalizing to new patients. Split by the real unit of deployment, which could be patient, donor, site or study, and perform normalization and feature selection within each training fold.
Use validation results for learning rate, architecture and early stopping, then freeze the selection strategy before final testing. Reusing a test set repeatedly makes it a de facto validation set. When possible, evaluate a time- or site-separated cohort to test transfer.
State whether the benchmark asks about new specimens from known people or truly unseen individuals, because these are distinct generalization targets.
Common mistakes
- Randomly splitting rows when measurements from the same biological entity occur in both partitions
The interviewer might ask next
How should the split change when training on multiple specimens from the same donor?