Statistical Test Needed

FORUM QUESTION: Julia St George - 22 May 2009 - 12:12

I found that 0 of 63 patients responded to an alternative therapy, but when I submitted the paper for publication, they said I needed a statistical test. Why do I need a statistical test for this, and what is the best way to respond to the reviewers?

FORUM ANSWER: FrankHarrell - 22 May 2009 - 15:11

The bigger question, if you are meaning to address questions of relative efficacy, is what is your control group? Data from a single-arm trial are not useful for establishing efficacy or lack thereof unless results from the comparator are known to a high degree of precision and you have faith that the patients entering your study are very similar to those in the comparator studies. But I sense however that you are only trying to establish that the alternative therapy is not looking good by itself in the types of patients treated. Then a statistical test is not called for but a confidence interval is. Here the 3/N rule comes into play because of the zero. The upper 95% confidence limit for the probability of response is close to 3/63 = 0.048. The Wilson (a method that is generally more accurate than the so -called "exact" binomial method) confidence limits are [0, 0.057] which in this case happens to equal the "exact" limits. You can be, loosely speaking, be 95% confident that the unknown probability of response to alternative therapy in the types of patients you studied is <= than 0.057.