Question 1 of 9 Fundamentals Concept
How is a confidence interval different from a p-value?
Short answer
A confidence interval describes a range of effect values compatible with the observed data under the method's assumptions. A p-value summarizes how incompatible the data are with a specified null model. They answer different questions. Neither tells you the probability that the hypothesis is true.
Why
A confidence interval starts with an effect estimate and conveys its sampling uncertainty. For instance, an estimated treatment benefit of four points with a 95% interval from one to seven points tells you both the central estimate and the range of values consistent with that interval procedure and its assumptions. A p-value instead evaluates the data against a specified null model, often no effect.
A small p-value is not a measure of how big, useful or biologically important the effect is. The same dataset may yield a narrow interval for a trivial effect, or a wide interval compatible with both benefit and harm. For many conventional two-sided tests of a single parameter, a 95% confidence interval excluding the null corresponds to p below 0.05, but that correspondence is not universal.
Always report the effect size, uncertainty, experimental design and assumptions. Avoid saying a 95% confidence interval gives a 95% probability that this fixed population parameter lies within the specific realized interval.
A strong answer covers
- Avoid interpreting either as a probability that the hypothesis is true