P.Mean: Quiz about p-values (created 2010-04-14).

In one of my webinars, I offered the following quiz question:

A research paper computes a p-value of 0.45. How would you interpret this p-value?

  1. Strong evidence for the null hypothesis
  2. Strong evidence for the alternative hypothesis
  3. Little or no evidence for the null hypothesis
  4. Little or no evidence for the alternative hypothesis
  5. More than one answer above is correct.
  6. I do not know the answer.

This is actually a bit of a trick question. It wasn't intended as a trick question; I was asleep at the switch when I wrote this. Still, it's like when my seven year old son makes a mistake--this really represents a learning opportunity.

Recall that a p-value is a measure of evidence AGAINST the null hypothesis. If the p-value is small (typically we say that 0.05 or less is small) then you have lots of evidence against the null hypothesis and you should reject it.

This p-value is large, so we do not reject the null hypothesis. So we can rule out #2 right away. But how about #1? We sometimes say when the p-value is large that we "accept the null hypothesis." But purists will insist on the wording "fail to reject the null hypothesis." A large p-value means little or no evidence against the null hypothesis, but you should not automatically interpret it as lots of evidence for the null hypothesis. It may be that your sample size is so small that you have little evidence for ANY hypothesis. Without some sort of context, such as an a priori sample size justification, we can't really say too much. With that sample size justification, #1 would be a good answer. But without it, it is possible that we can't decide between #1 and #3. If we know that the sample size is much too small then both #3 and #4 are correct.

But really, #6 is indeed the correct answer, though I can't be too upset with people who choose #1 or #4 or even #3 (which sort of makes #5 an attractive choice). Without more context, you cannot say anything with assurance about a large p-value. If I had provided more information, you could have chosen a specific answer, perhaps, but I didn't give you that extra information.

This is an example of how a p-value seems backwards. It is always a measure against the null hypothesis, but never, by itself, a measure for the null hypothesis. Here's how I'll word it in future webinars.

A research paper computes a p-value of 0.45. How would you interpret this p-value?

  1. Strong evidence for the null hypothesis
  2. Strong evidence for the alternative hypothesis
  3. Strong evidence against the null hypothesis
  4. Strong evidence against the alternative hypothesis
  5. A large p-value, by itself, does not provide strong evidence for or against any hypothesis.
  6. I do not know the answer.

Creative Commons License This work is licensed under a Creative Commons Attribution 3.0 United States License. This page was written by Steve Simon and was last modified on 2010-04-15. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: P-values.