StATS: Excluding placebo responders (created 2004-06-25).
I've always been fascinated by the placebo effect and the ethical issues associated with use of placebos in research.
A correspondent in the IRBForum email discussion group asked about the recent efforts of drug companies to identify patients who are likely to show a placebo effect and then exclude them from randomized trials. Here are some excerpts from a June 18 article in the Wall Street Journal (I couldn't get free access to this article on the web).
Many antidepressant trials fail because the placebo effect can be so powerful and highly variable. Somewhere between 30% to 50% of patients in depression trials get better when given fake pills, and that number has increased and become more volatile over time, making it more difficult to prove that a drug works. In comparison, only about half of patients taking antidepressants find their symptoms relieved by 50% or better.
"The placebo response has plagued psychiatric clinical trials," says Steven Paul, head of research and development at Lilly. If drug companies could sort out the placebo responders, it "would greatly help us in the short term in drug development."
But some clinical-trial experts worry that excluding placebo responders to facilitate drug development could lead to the approval of ineffective drugs. Kay Dickersin, a Brown University professor who teaches courses on clinical trials, says such winnowing "allows bias to enter in" and constitutes "a subtle manipulation" of trial results.
The companies dismiss the criticism that they will use the information to gain approval of ineffective drugs. They say the techniques would be used only in early trials, not for trials done for FDA approval.
This is a classic example of the tension between internal validity and external validity. Heterogeneity in the study population leads to a loss in precision. So drug companies, in an effort to show efficacy in the most economical way possible, want a nice homogenous sample.
This isn't just with regards to placebo responders. They want a sample where everyone complies with the medication. After all, it's hard to show efficacy in a population where half the people forget to take their pills.
They also try to exclude troublesome patients, patients with comorbid conditions or who are taking other medications that might interfere with the drug being studied. This often leads, by the way, to an unfair exclusion of elderly patients from studies even though they are often the target market for the drugs being studied. Older patients are almost always going to have several things go wrong with them at once, and they are the ones with frequent flier cards at the local pharmacy.
Once you get a nice squeaky clean sample, all is well and good for that particular comparison, but can you really extrapolate the results to patients that you would normally see in your practice? Do you have the option of saying "I won't treat you because I think that you are unlikely to take the prescriptions as directed"? Can you refuse to treat a patient unless they have only the one illness that you are trying to treat?
It seems to me that you should probably start with a nice clean sample and exclude all those troublemakers. After all, if you can't show an effect in a small trial with a clean sample, what's the point in running a larger pragmatic trial?
Of course, when the small trial with a clean sample shows up significant, you still need to look at the larger trial with a more heterogenous population.
By the way, I get nervous when people try to hitch their wagon to the so called placebo effect. I don't think these drug companies will have much effect in excluding placebo responders. In many studies the placebo effect is intermingled with other effects, such as the tendency for some diseases to resolve spontaneously. What's the old saying--if you don't get treatment for your cold it will last for seven days, but if you treat it aggressively it will be gone in a week.
Other effects like the cyclical nature of some diseases and the regression to the mean would not disappear when placebo responders were excluded. The placebo effect might also just be the greater level of care and attention that patients get in a randomized clinical trial. Just getting someone to visit a doctor regularly will probably help that person's health in a lot of small ways.
I attended a talk about hypnotherapy and I asked the speaker afterwards about the placebo effect. He argued that the effect of hypnotism WAS the placebo effect, because the placebo effect was the mind's ability to control the body. I guess he wasn't familiar with the Hrobjartsson and Gotzsche review in the 2001 NEJM that showed that the placebo effect was, for the most part, no different than an unblinded "no treatment" arm.
This page was written by Steve Simon while working at Children's Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children's Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Exclusions in research or Category: Placebos in research.