P.Mean: Review of new James Penston book (created 2011-12-31). News: Sign up for "The Monthly Mean," the newsletter that dares to call itself average, www.pmean.com/news.

Here's an early draft of a book review I've been asked to write about James Penston's newest book, Stats.con - How we've been fooled by statistics-based research in medicine.

If you've ever read a research publication that you didn't like, you'll love this book. It will give you all the ammunition you need to tear the study to shreds. Is it an observational study? Then either selection bias or confounding will invalidate the results. Is it a large scale randomized trial? Then inadequate allocation concealment probably allowed researchers to manipulate the randomization list and create biased groups. Or maybe too many patients were lost to follow-up. Maybe the entry requirements were so unnatural that extrapolation to the real world is impossible.

For both observational and randomized studies, if they used a p-value, well the Bayesians have pretty much destroyed the reputation of the p-value. If the researchers used a Bayesian alternative to the p-value, you're still okay. Bayesian methods are so complicated that something bad must be lurking in the shadows. "Just as the problems associated with frequentist statistics have managed to escape detection in the medical research community because of their complexity, so, too, the mysteries of Bayesian analysis may hide a multitude of sins that will take a long time to surface."

This book will also help you to feel superior to those quantitatively oriented statisticians. We statisticians have been hiding the flaws of our methods and have "skillfully created the illusion that all is well." We are "zealots" practicing "shameless brainwashing." We are keeping doctors from practicing medicine the way they want to. Referring to doctors, Dr. Penston informs us that "their professional freedom is now so severely restricted that their role in medicine is under threat."

This is not the first time that insults have been hurled at statisticians. Statisticians have been compared to terrorists because of their promotion of meta-analysis and to fascists for their promotion of a research hierarchy. Such language is the rallying cry for those who want to undo all the evils caused by Statistics. Those of you who fret about what you would lose, worry not. After all, mcrobiologists identified the microbial causes for many diseases, doctors discovered the benefits of antibiotics, and surgeons the value of effective anesthesia, "all without a statistician in sight."

Dr. Penston throws down the gauntlet on page 17 when he informs us that "epidemiological studients haven't been responsible for the discovery of the causes of many diseases, not have large randomizsed trials produced many drugs that have materially altered the lives of patients." I am not a doctor or an epidemiologist, but it is trivial to find data to dispute these claims. Epidemiologists have uncovered aspirin as the culprit behind many cases of Reye's Syndrome and have revealed multiple modes of transmission for the AIDS virus. The efficacy of folate supplementation for the prevention of neural tube defects and of antiretroviral drugs were all developed with the assistance of clinical trials. And if birth control pills haven't materially altered the lives of patients, then nothing has.

What Dr. Penston fails to address are the importance of negative findings. Antiarrhythmic drugs were thought to be useful but a clinical trial showed that they killed more than they helped. Careful epidemiology has established that there is no link between vaccines and autism. Placebo surgery trials have established the uselessness of

There is an irony in Dr. Penston's arguments that he notes briefly but which he fails to grasp the full significance of. Most of the problems with epidemiological studies and randomized controlled trials were discovered through the use of Statistics. If Statistics is a big con, if they produce unreliable results,

This page was written by Steve Simon and is licensed under the Creative Commons Attribution 3.0 United States License. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Incomplete pages.