Study Analysis Plan
Lead Author(s): Jeff Martin, MD
When Written
Written before the data are analyzed
Study Analysis Plan Content
- Detailed description of the techniques to be used to analyze data, step by step
- Forms the basis of Statistical Analysis section in manuscripts
- Parameters/rules/logic to guide key decisions:
- Which variables will be assessed for confounding and interaction?
- What p value will be used to guide reporting of interaction?
- What is a meaningful difference between two estimates (e.g., 10%) to determine confounding?
For example, it will outline whether you will use a forward or backward selection process.
- It will state which p value thresholds you will use to decide whether to report interaction or not.
- It will also state the rules you will use to deal with the bias-variance tradeoff of determining what to control for.
- It will contain how you define a meaningful difference between two estimates (e.g., crude and adjusted) to determine confounding.
Required
Required for clinical trial registration
- Can observational work be far behind?
Such a pre-specificed plan is required for registration of clinical trials with the FDA. I have to wonder if observational studies are far behind?
Utility
A plan helps to keep the analysis:
- Focused
- Reproducible
- Honest (avoids p value shopping)
There are many advantages to having such a plan.
- For one, it helps keep you focused in your own work and avoids your wasting a lot of time making things up as you go.
- A well written plan is also one that is reproducible by others.
- Finally, as I mentioned above, a plan helps keep you honest by avoiding p value shopping, by which I mean trying a number of different approaches until you get the answer you like.