thebench.bio #1 Ecosystem for the Bio-Tech Industry
All Model Evaluation questions

Question 10 of 20 Fundamentals Interpretation

How should you read a ROC curve and its area under the curve?

Short answer

A ROC curve plots sensitivity against 1 minus specificity as the threshold moves. The AUC is the probability that a randomly chosen positive case gets a higher score than a randomly chosen negative case, so 0.5 is chance and 1.0 is perfect ranking. It summarizes ranking across all thresholds, not performance at the one you will use.

Why

Each point on the curve is one threshold. Moving the threshold down catches more positives but also flags more negatives, so the curve climbs from the bottom left to the top right. A diagonal line is a model no better than a coin.

Hajian-Tilaki's review gives the most useful interpretation of the area: the probability that a randomly chosen diseased subject is ranked as more likely diseased than a randomly chosen non-diseased subject. An AUC of 0.80 therefore means the model orders a random positive and negative pair correctly 80 percent of the time. That framing explains its limits.

AUC is about ranking, so it is unchanged if every predicted probability is halved, which means it says nothing about calibration. It averages over thresholds that will never be used, so two models with equal AUC can differ greatly in the region that matters, for example at high specificity for a screening test. It is insensitive to prevalence, which sounds like an advantage but hides how many false positives each true positive costs when positives are rare.

Read the curve's shape near the operating point, report AUC with a confidence interval, and add calibration and threshold-specific metrics.

A strong answer covers

  • Sensitivity against 1 minus specificity across thresholds
  • AUC as probability of correct pair ranking
  • 0.5 is chance, 1.0 perfect
  • Says nothing about calibration
  • Look at the operating region, add CI and threshold metrics

Common mistakes

  • Reading an AUC of 0.8 as 80 percent accuracy
  • Choosing between models with equal AUC without looking at the part of the curve near the intended threshold