This page has moved to my new website.
While surfing the web, I found out about a book, Fiction and Fantasy in Medical Research. The Large-Scale Randomised Trial by James Penston (2003, The London Press, London England. ISBN: 0-9544636-1-7). I picked up a copy but have not read it yet. The blurb on the back cover reads:
"Every day, millions of patients throughout the world take treatment which is based on the results of large-scale randomised trails. But, how much do we really know about these studies? This book exposes the serious flaws in this method of medical research. Although making vast profits for the pharmaceutical industry, large-scale randomized trials do little to improve the lives of patients and are responsible for an enormous waste of scare health care resources."
Wow! That's quite an indictment. When I am done reading the book, I'll post a short review on this page. But Dr. Penston is not the first critic in this area.
The Association for Human Resource Protection printed a stinging critique of randomized clinical trials (RCTs) in the study of psychiatric drugs. An article in the June 2001 issue of the Journal of Clinical Epidemiology ([Medline]), asks whether the RCTs is the gold standard or the golden calf. I myself discuss the strengths and weaknesses of the RCT ([Medline]) and demote it from a gold standard to a silver standard.
Perhaps the sharpest criticisms of RCTs, however, come from proponents of complementary and alternative medicine (CAM). They have considered randomized clinical trials (RCTs) to be "reductionist" because they fail to look at the whole patient and reduce that patient to a single dimension. Mason et al [Medline] give a balanced perspective on this controversy. They point out that:
"..many practitioners argue that research methods dissect their practice in a reductionist manner and fail to take into account complementary medicine's holistic nature."
They argue that RCTs have to be adapted to the special features of CAM. In particular, the tendencies of RCTs and CAM are often in conflict.
Note that these are tendencies. Some RCTs focus on more than one disease, but the tendency is to focus on a single disease. Some types of CAM are standardized, but the tendency is to offer individualized therapies.
It's not just CAM that exhibits these conflicts, though. The Medical Research Council wrote a report in April 2000 ([pdf]) that discusses the evaluation of complex interventions where it is difficult to isolate the individual components of the intervention. They mention several examples.
Does a physiotherapist contribute significantly to the management of knee injuries? This role goes beyond a simple sequence of exercises.
The package of care to treat a knee injury may be quite straightforward and easily definable - and therefore reproducible: “This series of exercise in this order with this frequency for this long, with the following changes at the following stages”. However, the physiotherapist may have, in addition to the exercises, a psychotherapy role in rebuilding the patient's confidence, a training role teaching their spouse how to help with care or rehabilitation, and potentially significant influence via advice on the future health behaviour of the patient. Each of these elements may be an important contribution to the effectiveness of a physiotherapy intervention.
How does a stroke unit improve the quality of care for stroke patients? The concept of a stroke unit is difficult to standardize.
For example, although research suggests that stroke units work, what, exactly, is a stroke unit? What are the active ingredients that make it work? The physical set-up? The mix of care providers? The skills of the providers? The technologies available? The organizational arrangements?
How cognitive behavioral therapy works? This approach is highly individualistic.
Does success depend on the personality of the therapist? The personality, health status, social status, or other characteristic of the patient? The content of the therapy? The way it is delivered? The frequency of contact? The location of contact? The duration and the timing? What other components count?
Rather than arguing that RCTs need to be adapted to the special needs of CAM, perhaps RCTs should be adapted to meet the special needs of many types of medical interventions.
Furthermore, the claim that a practice is holistic should not be used as a blithely disregard evidence from an overly simplistic RCT. Perhaps the RCT can get to the heart of the issue by focusing on a single key dimension to the problem. A fourth grade student evaluated Therapeutic Touch (TT) for a science fair project. This project was highlighted on a Public Broadcasting Service show "Scientific American Frontiers" and published in the April 1, 1998 issue of JAMA ([Medline]) and received a lot of press coverage (CNN has a very nice story).
Therapeutic Touch is a therapy to improve health through the manipulation of the human energy field. There apparently is no physical touching. The official website on therapeutic touch describes it as:
"...an intentionally directed process of energy exchange during which the practitioner uses the hands as a focus to facilitate the healing process. It is a contemporary interpretation of several ancient healing practices. Therapeutic Touch is a scientifically-based practice founded on the premise that the human body, mind, emotions and intuition form a complex, dynamic energy field. The human energy field is governed by pattern and order. In health, the field is balanced, however in disease, the energy is characterized by imbalance and disorder."
Emily Rosa's experiment was very simple, perhaps too simple. If practitioners of Therapeutic Touch are able to manipulate energy fields, they must first be able to detect energy fields. She would hold her hand above either the left or right hand of the practitioner and ask him/her to tell which hand. The choice of hand was randomly determined by a coin flip. A screen with two holes in it prevented the practitioner from seeing what was going on.
Emily Rosa got 21 experienced practitioners to agree to the test. They were right only 44% of the time. Did this simple experiment disprove the healing power of TT? Perhaps not. TT is a complex intervention and this experiment only looked at a single aspect of it.
The experiment does shift the burden of proof, however. Detection of energy fields is a fundamental aspect of TT that all other aspects of this therapy rely on. How can practitioners of TT manipulate energy fields that they cannot even detect? Any further research should be discontinued until practitioners of TT can demonstrate the ability to detect energy fields in a rigorous blinded study.
Larry Sarner (Emily Rosa's step-father) makes much the same point in an article on the Quackwatch web site that responds to criticisms of the Rosa study. In particular, he responds to the criticism of reductionism:
[Critical comment #5] This was not a test of TT, but a parlor game. What the practitioners were required to do during the experiment invalidated its applicability to TT, especially since TT is a holistic process and can't be validly analyzed in parts. Emily's test was not of efficacy or technique (or "healing"), but I of raw ability. It's very much like testing a surgeon to see if he can l tell, without looking, in which hand the scalpel is being held. In any event, there was some movement. Emily presented her hand after each coin flip, which required relative movement between her hands and the subject's. Both subjects and Emily had at least small I movements of their hands during the trials, and some practitioners even wiggled their fingers or hands. Previous descriptions of the sensations of feeling an HEF state that the field itself is constantly in motion, and the literature states that such motion can be easily felt. Significantly, all of Emily's subjects agreed to the protocol and none voiced any concern that the test setup would pose a problem in demonstrating their ability. The argument about TT being "holistic" is a thinly disguised attempt to get back to "outcome" (i.e., clinical) testing, where it is easier to obfuscate, ignore negative results, or explain away nonconforming data. There have been numerous clinical trials on outcomes using TT. The results are highly mixed. Some tests do not have statistically significant results, others revealed slight positive effects (though statistically significant), and several actually reported statistically significant effects, but negative (i.e., the control group did better than the TT group). Holistic practitioners' prejudice against what they call "reductionism" (analyzing things in parts) is not shared by others in scientific medicine.
There is, by the way, a huge financial incentive to demonstrate the ability to detect energy fields. The James Randi Education Foundation offers a one million dollar prize to anyone who can show, under carefully controlled conditions, evidence of any paranormal, supernatural, or occult power or event. James Randi himself says that TT as well as several other alternative medicine therapies (Iridology, Reiki, Homeopathy and Applied Kinesiology) would qualify for the challenge.