thebench.bio #1 Ecosystem for the Bio-Tech Industry
All Research Reproducibility questions

Question 6 of 6 Advanced Concept

Why do documented exclusions matter when interpreting an experiment?

Short answer

Exclusions change the analyzed sample and can bias effects. Prespecified, transparent criteria help distinguish quality control from outcome-driven selection.

Why

Before collecting or analyzing outcomes, define measurable exclusion conditions such as sample authentication failure, prespecified contamination thresholds or missing critical measurements. Track every excluded unit, its reason and the timing of the decision. Compare group counts and relevant covariates before and after exclusions and include a sensitivity analysis retaining questionable observations where scientifically appropriate.

A decision to discard a control sample because it weakens significance changes the inference and damages trust. At the same time, keeping a genuinely contaminated sample can also mislead. Explain the effect of exclusions on the estimand and generalizability, and publish a transparent flow diagram or log.

A documented exclusion is not automatically justified. Readers need to judge whether the rule and its application were independent of desired results.

Common mistakes

  • Removing inconvenient observations without a consistent rule or recording how sample selection changed