When is a simpler baseline model preferable to a complex model?

Why

A logistic regression model can be a more responsible baseline than a large neural network for a small clinical cohort with structured predictors. Compare both under the same realistic splits, preprocessing and tuned decision thresholds, and report confidence intervals around relevant measures. Consider explainability for the target audience, maintenance needs, failure detection, computation, fairness and sensitivity to covariate shift.

A complex model deserves adoption only when its improvement is clinically meaningful, reproducible and generalizes beyond development. Baseline selection is not anti-innovation: it supplies a robust comparator and exposes whether extra complexity earns its cost. Avoid choosing a model solely on a single leaderboard metric or claiming simple models are universally more calibrated.