StATS: Statistical koan #4 (March 26, 2006)
A visit from a thief.
Student Leaf asked Master Stem, "I have heard some of my teachers say 'I accept the null hypothesis' and others say 'I fail to reject the alternative hypothesis.' Why do these words not mean the same thing?"
Master Stem scoffed "Bah, these teachers of yours! They do not understand probability, every single one of them."
Student Leaf asked "But what does probability have to do with it? The null hypothesis is true or it is not. Where is the probability?"
Master Stem replied "All hypotheses have probability if you consider them carefully. Consider a thief who visits your house. You do not see anything unusual during the visit, but after the thief leaves, you find that your jewelry has been stolen."
Student Stem replies "Then the thief must have taken the jewels."
"Are you so sure? The night that the thief visited, was the night that you had a great celebration, and one hundred people came and visited. Only after everyone had left did you notice that the jewels were gone."
"Then the thief probably did not take the jewels, Master Stem. It was probably one of the other guests."
"But you had the police investigate. They searched the houses of all the other guests and these searches did not yield the jewels."
"Then the other guests are innocent. It must have been the thief because only his house could have contained the jewels."
"That is reasonable, perhaps, if you believe that there is no other hiding place for the jewels. But I forgot to mention that the police are very busy and they did not search very carefully in any one house."
"Then I am not sure what to believe, Master Stem."
"You never believe anything with certainty, Student Leaf. A thief visits your house. You have an initial level of mistrust because I have called this person a thief. Think about what this means. This person has been known to steal from others, so you have a high degree of belief that he may have stolen from you as well. Now I provide you with more information. Each piece of information either increases or decreases your degree of belief. But no matter how much data you accumulate, you will never have certainty, just a degree of belief."
"How then, Master Stem, am I able to make any decision, if I never have certainty."
"There is balance, Student Leaf. How much do you lose if the thief is allowed to get away with the crime? What is the consequence of the loss of face when your accusations of thievery are proven false?"
"I do not know if I can assign a value to these things, Master Stem."
"Such foolishness. You want to make very important decisions about these jewels, but not once did you think to assess their value. Were they but a trifle? Then be very careful not to risk your reputation over such a small thing. Were the jewels of great beauty and incomparable value? Then do not let any small thing stand in the way of possibly recovering such a great treasure."
This page was written by Steve Simon while working at Children's Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children's Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Teaching resources.