thebench.bio #1 Ecosystem for the Bio-Tech Industry
All Model Evaluation questions

Question 8 of 20 Fundamentals Concept

What do R-squared, mean absolute error and root mean squared error tell you about a regression model?

Short answer

R-squared is the share of variance in the outcome the model explains, relative to always predicting the mean. MAE is the average size of errors in the outcome's units. RMSE is also in those units but weights large errors more heavily, so a gap between RMSE and MAE signals a few big misses.

Why

The scikit-learn guide defines R-squared as the proportion of variance explained, with a best value of 1.0, and notes that a constant model predicting the mean scores 0 and that R-squared can be negative because a model can be arbitrarily worse. It also warns that because variance depends on the dataset, R-squared may not be comparable across datasets. A model predicting binding affinity can score R-squared 0.6 on a diverse compound library and 0.2 on a narrow series with the same absolute errors, simply because the narrow series varies less.

MAE and RMSE answer the practical question of how wrong the predictions are, in the outcome's units. If a solubility model has MAE 0.5 log units and RMSE 1.1, most predictions are close but some are far off, and those outliers deserve a look. Choose the metric that matches the cost: RMSE when large errors are disproportionately harmful, MAE when every unit of error costs the same.

All three summarize average behavior and hide where errors occur, so plot predicted against observed values and residuals against key variables. Report them on held-out data, because training-set R-squared always rises as features are added.

A strong answer covers

  • R-squared is variance explained against a mean-only baseline
  • R-squared can be negative and depends on outcome spread
  • MAE is average absolute error in outcome units
  • RMSE penalizes large errors more
  • Plot residuals and report on held-out data

Common mistakes

  • Comparing R-squared between datasets with very different outcome ranges
  • Reporting training-set R-squared as model performance
  • Assuming R-squared cannot fall below zero