Why might cross-validation performance not predict performance at a new hospital?

Why

Good internal cross-validation estimates performance for splits that reflect the source data and the chosen sampling plan, not every future clinical site. Another hospital may have a different disease spectrum, referral policy, scanner, sequencing platform, missing-data pattern or label definition. Investigate domain shift by comparing feature distributions, measurement units, prevalence, subgroup metrics and calibration before and after deployment.

Preserve a genuinely external cohort and evaluate sensitivity, specificity and probability calibration according to the clinical use case. Grouping by donor is necessary where donors repeat, but it does not replace a site-held-out evaluation. An AUROC that is stable while calibration deteriorates can still yield unsuitable treatment thresholds.

Clearly separate technical drift from biological population differences.