Question 5 of 6 Advanced Scenario
How would you preregister a sensitivity analysis?
Short answer
Specify the question, primary analysis, plausible alternatives, exclusion rules and decision criteria before examining outcomes. Preserve a timestamped public or access-controlled record.
Why
Write down the estimand, target population, outcome, hypothesis and main statistical model. Then list a bounded set of justified alternatives such as different covariate adjustments, transformations, missing-value procedures or quality thresholds, and describe how they will be summarized. Specify exclusion criteria and justify them independently of results.
State what constitutes substantive stability: sign, effect magnitude, confidence intervals or scientific relevance, not simply whether all p-values are below 0.05. Deposit the analysis plan in a timestamped registration platform and record amendments with reasons and dates. Exploratory analyses are valuable but should be visibly labeled as post hoc.
For example, a precommitted set of three RNA-seq filtering thresholds is more interpretable than choosing whichever threshold yields the most discoveries.
Common mistakes
- Preregistering a vague promise to try several models without defining the actual choices
The interviewer might ask next
How would you report a justified change to a preregistered plan after inspecting the data?