StATS: Statistical koans (February 27, 2006)

Update: Check out my koan page.

I've been thinking about using a literacy device known as a koan to illustrate important statistical issues. A koan is a story used in Buddhist teaching that involves a seemingly meaningless or contradictory statement. Careful contemplation of this statement leads you to a more fuller understanding of Buddhist teaching. Often these stories end with a student reaching a state of greater enlightenment, so might be related to the concept of experiencing an epiphany. Here's an example:

Nan-in, a Japanese master during the Meiji era (1868-1912), received a university professor who came to inquire about Zen. Nan-in served tea. He poured his visitor's cup full, and then kept on pouring. The professor watched the overflow until he no longer could restrain himself. "It is overfull. No more will go in!" "Like this cup," Nan-in said, "you are full of your own opinions and speculations. How can I show you Zen unless you first empty your cup?" as quoted on the Zen Koans page,

Although koans have an overt link to religion, they have been used by others to illustrate difficult non-religious issues.

In the days when Sussman was a novice Minsky once came to him as he sat hacking at the PDP-6. "What are you doing?", asked Minsky. "I am training a randomly wired neural net to play Tic-Tac-Toe." "Why is the net wired randomly?", asked Minsky. "I do not want it to have any preconceptions of how to play." Minsky shut his eyes. "Why do you close your eyes?", Sussman asked his teacher. "So the room will be empty." At that moment, Sussman was enlightened. as quoted on the Artificial Intelligence Koans page,

Some other examples of non-religious koans appear at The Rootless Root, the Unix Koans of Master Foo, by Eric Steven Raymond.

Could I use the same device to teach Statistics? Probably not, but I'll try anyway. So here is the first ever statistical koan:

Master Stem was teaching the class about the perils of observational data when Student Leaf interrupted. "Master Leaf," the student inquired, "You have taught us that randomized studies are the only true path to knowledge. So why do you now teach us about observational studies?"

Master Stem replied, "Have I been so certain about the true validity of research that uses randomization?"

Student Leaf replied, "Yes, Master Stem. You taught us that randomized studies assure us of a fair and unbiased comparison that cannot be achieved without randomization. By invoking the law of large numbers, you have shown that only randomized studies prevent bias from unknown and unmeasured covariates."

Then Master Stem replied, "Very good, Student Leaf. And this knowledge about the superiority of randomized studies, did it itself emerge from a randomized study?"

If I get inspired I may write a few more statistical koans. What I am hoping is to find a good koan that illustrates the logical flaws of traditional hypothesis testing. If I am successful it will lead to my first Bayesian epiphany.

This page was written by Steve Simon while working at Children's Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children's Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Teaching resources.