Perfect isn’t quite good enough

*Blog post
Year 2006
Small sample size issues
Author

Steve Simon

Published

December 12, 2006

Someone wanted me to double check their calculations for Fisher’s Exact test. If the control group, 3 out of 10 patients experienced an unfortunate outcome. In the treatment group none did (out of 6). You would think that a perfect result in the treatment group would be compelling, but the one-sided p-value for Fisher’s Exact test is 0.21.

That calculation is a bit disappointing, but perhaps not too surprising. The rule of three states that when you observe zero events in a sample of n patients, then 3/n is an approximate upper 95% confidence interval. So with 0/6, that upper limit would be 3/6 or 50% which is well above the control rate of 30%.<U+FFFD> So perfection just isn’t good enough when you have such small sample sizes.

Related web pages:

Earlier versions are here and here.