Question 6 of 9 Advanced Troubleshooting
How would you approach multiple comparisons in an omics experiment?
Short answer
Testing thousands of genes creates many opportunities for false positives. Define the hypothesis family, estimate effects and adjust for multiplicity, often using the Benjamini-Hochberg false discovery rate approach in discovery analyses. Then check biological plausibility and replication instead of treating adjusted p-values as proof.
Why
If you test 20,000 genes at an unadjusted threshold of 0.05 under a simplified all-null model, you would expect roughly 1,000 nominal positives even without true biological effects. Omics analyses therefore need a multiplicity strategy defined before interpreting hits. First specify the primary contrasts, experimental unit, quality controls, independent biological replicates and model assumptions.
In a discovery setting, Benjamini-Hochberg adjustment is often used to control the expected fraction of false discoveries among rejected hypotheses under its conditions. Confirmatory settings may require stronger family-wise error control. Report adjusted p-values or q-value definitions, estimated log fold changes, uncertainty, filtering criteria and independent validation.
When counts and dispersion are modeled, use appropriate tools rather than thousands of naive t-tests. Correlated features and dependent hypotheses affect assumptions, so an FDR level is not a guarantee that any given gene call is correct. Avoid changing filtering or thresholds after seeing which genes look attractive without transparent disclosure.
Common mistakes
- Confusing a 5% FDR with a 5% probability that each reported gene is a false positive, or calling technical repeats biological replicates
The interviewer might ask next
When would you prefer family-wise error control over false discovery rate control?