thebench.bio #1 Ecosystem for the Bio-Tech Industry
All Statistics questions

Question 14 of 20 Intermediate Practical

How would you estimate the sample size for a two-group comparison of a continuous outcome?

Short answer

Fix four inputs before collecting data: the smallest difference worth detecting, the expected standard deviation, alpha and power. Then n per group is about 2 times (1.96 + 0.84) squared times SD squared divided by the difference squared for 5 percent two-sided alpha and 80 percent power, inflated for expected dropout.

Why

Sample size is a design decision that links the scientific question to the risk of a wrong answer. The difference to detect should be the smallest effect that would matter, not the effect you hope for. The SD comes from pilot data or earlier studies.

Suresh and Chandrashekara give the normal-approximation formula for comparing two means and recommend adding 10 to 20 percent for withdrawals and missing data. Software such as statsmodels TTestIndPower solves the exact t-test version from the standardized effect, the difference divided by the SD.

How to work it through

  1. Agree with the scientists on the primary outcome and the smallest difference that would change a decision.
  2. Find a credible SD from a pilot, historical controls or the literature, and note its uncertainty.
  3. Choose alpha and power, typically 0.05 two-sided and 80 or 90 percent.
  4. Compute n per group with the formula or with power software for the planned test.
  5. Inflate for expected dropout by dividing by 1 minus the dropout fraction.
  6. Repeat the calculation over a plausible range of SDs and differences and record the assumptions in the protocol.

An example

A study compares a biomarker between treated and control animals. A difference of 8 units matters and the historical SD is 12, so the standardized effect is 0.67. The formula gives 2 times 2.80 squared times 144 divided by 64, about 35.3, so 36 per group.

The exact t-test calculation in statsmodels gives 36.3, so 37 per group. With 10 percent expected loss, 37 divided by 0.9 is 41.1, so the plan is 42 per group. If the SD were 15 instead, the requirement would rise by more than half.

Where the answer stops holding

The result is only as good as the SD estimate, and pilot SDs from a few animals are themselves very uncertain. The formula assumes independent observations and a roughly normal outcome, so clustered designs, repeated measures and skewed data need other methods. Sample size for a confirmatory study must be fixed in advance.

Recomputing it after looking at the data invalidates the error rates.

A strong answer covers

  • Smallest important difference, not the hoped-for one
  • SD from pilot or historical data
  • Alpha and power fixed in advance
  • Inflate for dropout
  • Sensitivity of n to the SD assumption

Common mistakes

  • Choosing the difference to detect so that the required sample size fits the budget
  • Using an SD from technical replicates when the unit of analysis is the animal
  • Forgetting that n scales with the square of the SD divided by the difference