thebench.bio #1 Ecosystem for the Bio-Tech Industry
All Statistics questions

Question 20 of 20 Advanced Concept

When would a hierarchical model be preferable to pooling results across laboratories?

Short answer

Use a hierarchical, or random-effects, model when labs plausibly have genuinely different true values and you want both an overall estimate and how much labs vary. Simple pooling assumes one shared value and understates uncertainty when labs differ.

Why

Imagine eight laboratories measuring the same reference material, each with its own instruments and operators. Complete pooling treats every measurement as coming from one population and ignores lab, so a lab with many replicates dominates and the interval is too narrow if labs truly differ. No pooling analyzes each lab separately and gives noisy estimates for labs with few runs.

A hierarchical model sits between. It assumes each lab has its own true value drawn from a common distribution, estimates the overall mean and the between-lab variance, and pulls each lab's estimate toward the overall mean in proportion to how little data that lab has. The Cochrane Handbook describes the same choice in meta-analysis.

A fixed-effect analysis assumes every study estimates one underlying effect, while random-effects assumes different but related effects that follow a distribution, and gives relatively more weight to small studies. The between-lab variance is often the real answer, since it says how reproducible the method is. A prediction interval then gives the range expected for a new lab.

Limits matter. With only three or four labs the between-lab variance is poorly estimated, the distributional assumption is hard to check, and a systematic difference with a known cause, such as an instrument model, belongs in the model as a covariate.

A strong answer covers

  • Complete pooling, no pooling and partial pooling
  • Random effects assume related but different true values
  • Between-lab variance as a reproducibility measure
  • Shrinkage strongest for labs with little data
  • Few labs make variance estimates unstable

Common mistakes

  • Pooling all replicates and reporting a confidence interval that ignores lab-to-lab variation
  • Fitting a random effect for lab with only two or three labs and trusting its variance estimate
  • Using a random effect to absorb a known systematic cause that should be a covariate