I’m on various email discussion groups and every once in a while someone sends out a request that sounds something like this.

I’m teaching a class (or running a journal club or giving a seminar) on research design (or evidence based medicine or statistics) and I’d like to find an example of a research study that use bad statistical analysis.

And there’s always a flood of responses back. But if I were less busy, I’d jump into the conversation and say “Stop! Don’t do it!” Here’s why.

Sharing bad examples have several problems. They encourage black-and-white thinking versus a more nuanced interpretation. They also can breed fear and/or cynicism in your students. The alternative to bad examples are to pick examples based on other criteria than how bad or good the statistical analysis is, or to pair bad examples with good examples to show the right way to do research.

Black-and-white thinking. An important critical thinking skill is deciding when a problem is so serious that you should totally disregard that study.