thebench.bio #1 Ecosystem for the Bio-Tech Industry
All Machine Learning questions

Question 2 of 9 Intermediate Scenario

How would you split data when samples come from the same donor?

Short answer

Keep every sample from a donor in the same partition, including technical replicates and repeated visits, and stratify at the donor level where feasible.

Why

The statistical unit is often the person or donor rather than each assay measurement. When training contains one visit and testing contains another from the same donor, the model can exploit person-specific molecular signatures and produce optimistic accuracy. Use an explicit donor ID and group-aware splitting such as GroupKFold, then verify there is no overlap between train and test groups.

If centers are a major deployment shift, reserve entire centers or periods for an additional external-style test. Compute feature normalization, feature selection and hyperparameter tuning inside training folds, not once across the whole cohort. Check outcome balance at the group level and report group counts rather than just sample counts.

A grouped split can have higher variance with few donors, which should be acknowledged.

Common mistakes

  • Splitting rows randomly without checking donor, family, patient or experimental-batch overlap