Example: Selection Bias - Nuclear Bomb Testing
Lead Author(s): Jeff Martin, MD
Example of Selection Bias: Leukemia and Nuclear Bomb Testing
History Smoky Atomic Test in Nevada
Leukemia Following Smoky Atomic Test in Nevada: (Caldwell et al., JAMA, 1980)
There has been a lot of interest over how much risk there was for cancer in the military observers of the various nuclear bomb tests that have been performed over the years.
- In one such study, there was an attempt to find all observers of the Smoky Atomic test in Nevada.
- In this study, 76% of the total troops were later identified and the occurrence of leukemia was determined.
Analysis of the Survey Respondents
On the surface, 76% seems pretty good. After all, the pre-election poll sampled just 1 in 10,000 voters and got the right answer. But how representative were these 76%?The authors were actually clever enough to empirically look at this.
The authors separated the troops into those who were contacted by the investigators - 82% of the participants- and those who contacted the investigators on their own - 18% of the participants.
- And, as you might have predicted those who contacted the investigators on their own, i.e. self-selection - had a much higher leukemia prevalence, over 4 times higher.
Hence, at the end of the day we really don't know how representative these 76% were and we have to be concerned that they are enriched - by self selection - by those who developed leukemia.
- The only way you could know the truth is if you contacted all 100% or if you got close to 100% and did sensitivity analyses on the remainder.
True Denominator
This is about as good as you can do in descriptive studies because in this example you actually could through military records enumerate the entire denominator.
- Often we don't know the true denominator.
- This is in contrast to, say, elections, where you have the chance to see the actual truth.
Do you think the investigators would have done better by just using those who they contacted? Hard to know. This group may have been depleted of very sick persons (but not yet dead and not in any death registry) or perhaps those who already died of cancer.
References
Caldwell, G. G., Kelley, D. B., & Heath, C. W., Jr. (1980). Leukemia among participants in military maneuvers at a nuclear bomb test. A preliminary report. Jama, 244 (14), 1575-1578