Quiz about p-values (created 2010-04-14).
This page is moving to a new website.
In one of my webinars, I offered the following quiz question:
A research paper computes a p-value of 0.45. How would you interpret this p-value?
This is actually a bit of a trick question. It wasn't intended as a trick question; I was asleep at the switch when I wrote this. Still, it's like when my seven year old son makes a mistake--this really represents a learning opportunity.
Recall that a p-value is a measure of evidence AGAINST the null hypothesis. If the p-value is small (typically we say that 0.05 or less is small) then you have lots of evidence against the null hypothesis and you should reject it.
This p-value is large, so we do not reject the null hypothesis. So we can rule out #2 right away. But how about #1? We sometimes say when the p-value is large that we "accept the null hypothesis." But purists will insist on the wording "fail to reject the null hypothesis." A large p-value means little or no evidence against the null hypothesis, but you should not automatically interpret it as lots of evidence for the null hypothesis. It may be that your sample size is so small that you have little evidence for ANY hypothesis. Without some sort of context, such as an a priori sample size justification, we can't really say too much. With that sample size justification, #1 would be a good answer. But without it, it is possible that we can't decide between #1 and #3. If we know that the sample size is much too small then both #3 and #4 are correct.
But really, #6 is indeed the correct answer, though I can't be too upset with people who choose #1 or #4 or even #3 (which sort of makes #5 an attractive choice). Without more context, you cannot say anything with assurance about a large p-value. If I had provided more information, you could have chosen a specific answer, perhaps, but I didn't give you that extra information.
This is an example of how a p-value seems backwards. It is always a measure against the null hypothesis, but never, by itself, a measure for the null hypothesis. Here's how I'll word it in future webinars.
A research paper computes a p-value of 0.45. How would you interpret this p-value?