Reporting Power
Lead Author(s): Peter Bacchetti, PhD
Power calculations are widely recognized to be irrelevant in the interpretation of completed studies, because confidence intervals are simpler, more direct, and more reliable. A study provides strong evidence against a hypothesized effect if that effect is outside the confidence interval. Trying to instead reason from P>0.05 and the estimated power to detect the hypothesized effect is convoluted and unrealiable. Some references illustrating the consensus on this topic are:
Cox DR. Planning of Experiments. New York: Wiley, 1958: page 161. “Power . . . is quite irrelevant in the actual analysis of data.”
Tukey JW. Tightening the clinical trial. Controlled Clinical Trials 1993; 14:266-285. Page 281: “power calculations … are essentially meaningless once the experiment has been done.”
Goodman SN, Berlin JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med 1994; 121:200-6.
Hoenig JM, Heisey DM. The abuse of power: the pervasive fallacy of power calculations for data analysis. American Statistician. 2001;55:19-34.
Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG: CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. British Medical Journal 2010, 340:28. Page 8: “There is little merit in a post hoc calculation of statistical power using the results of a trial”
Senn, SJ. Power is indeed irrelevant in interpreting completed studies. BMJ 2002; 325: 1304.
Strangely, guidelines for reporting of randomized clinical trials nevertheless call for reporting of how the sample size was determined, and this has spilled over to all other types of studies. Regardless of how investigators may have actually chosen sample size, this requirement is almost always addressed by providing a power calculation. The requirement is justified as a way to indicate the primary outcome and the original target sample size, but this justification has been challenged on the basis that such information could instead be stated directly (Bacchetti P. Peer review of statistics in medical research - author’s thoughts on power calculations. Br Med J, 325: 492-493, 2002).