thebench.bio #1 Ecosystem for the Bio-Tech Industry
All Statistics questions

Question 11 of 20 Fundamentals Concept

What are type I and type II errors, and how does statistical power connect them?

Short answer

A type I error is concluding there is an effect when there is none, with probability alpha. A type II error is missing a real effect, with probability beta. Power is 1 minus beta, the chance of detecting an effect of a stated size, and it rises with sample size and effect size.

Why

Every test makes a decision under uncertainty, so two kinds of mistake are possible. Rejecting a true null hypothesis is a false positive, and its probability is the significance level alpha, often set at 0.05. Failing to reject a false null is a false negative, with probability beta.

Power, 1 minus beta, is the probability that the study detects an effect of a specified size if that effect is real. Banerjee and colleagues note that many studies set alpha at 0.05 and beta at 0.20, giving 80 percent power. The two errors trade against each other.

Lowering alpha to 0.01 to avoid false positives makes false negatives more likely unless the sample grows. Power depends on four things: the effect size you care about, the variability of the measurement, the sample size and alpha. A small pilot of four animals per group may have little chance of detecting even a meaningful difference, so a non-significant result from it is weak evidence of no effect.

Low power also inflates the effects that do reach significance, because only the luckiest estimates cross the threshold. Power is a property of the design for a stated effect size, computed before the study, not a property of an observed result.

A strong answer covers

  • Type I is a false positive, probability alpha
  • Type II is a false negative, probability beta
  • Power equals 1 minus beta for a stated effect size
  • Power rises with n, effect size and lower variability
  • Non-significance in a low-power study is weak evidence

Common mistakes

  • Reading a non-significant result as proof that the treatment has no effect
  • Calculating power from the observed effect after the study to explain a null result
  • Believing alpha is the probability that a significant finding is false