StATS: Testing for side effects (December 29, 2004)

Today's USA Today has a nice summary of the recent controversies over drugs that are being removed from the market because of an unacceptable increased risk of side effects. Under special scrutiny these days are a class of drugs known as COX-2 inhibitors. According to MedicineNet,

COX-2 inhibitors are newly developed drugs for inflammation that selectively block the COX-2 enzyme. Blocking this enzyme impedes the production of the chemical messengers (prostaglandins) that cause the pain and swelling of arthritis inflammation. Cox-2 inhibitors are a new class of nonsteroidal anti-inflammatory drugs (NSAIDs). Because they selectively block the COX-2 enzyme and not the COX-1 enzyme, these drugs are uniquely different from traditional NSAIDs.

One particular drug in this class, Vioxx (generic name: Rofecoxib), has been voluntarily removed from the market by its manufacturer, Merck, because this drug was associated with an increase in the risk of serious cardiovascular events. Two other drugs in this class (Celebrex/Celecoxib and Bextra/Valdecoxib) also have data suggesting an increase in the risk of cardiovascular problems, and there are calls to pull these drugs off the market also.

There are a lot of political comments about this issue, and while these are very important, it is also important to recognize that detecting drugs that have an increased risk of side effects is a difficult job.

The first problem is with most of the randomized trials that drug companies use to submit these drugs for government approval. These trials typically do not last long enough, nor have sufficient numbers of patients to get a handle on rare but important side effects.

Sometimes patients who tolerate a drug poorly will drop out of the trial. This causes serious problems unless the dropouts are accounted for properly and analyzed appropriately.

The second problem is that prescribing physicians do not carefully consider the limitations of these randomized trials. Sometimes a drug will be prescribed for a different condition than the one originally submitted, a practice known as off-label prescribing. This is a difficult area, because drug companies cannot advertise or market a drug for a condition other than the one that it originally got approval for. But these companies will often find covert and subtle ways to encourage and promote off-label use. From the perspective of side effects, off-label use leads to a lot of unknowns, because the types of people who have the new condition will be quite different from those in the randomized trials and may have a radically different side effect profile.

Often these trials exclude patients who are more likely to develop side effects. That would be okay if the same exclusion found its way into medical practice, but often this is forgotten or overlooked.

The third problem is that data on the side effects of drugs is inherently ambiguous. If you want to show that a drug is efficacious, you can usually point out two or three outcomes that would definitively indicate whether the drug works or not. But the possible side effects for a drug is usually large and unwieldy. Most statistical approaches work well only when the number of outcomes being evaluated is small and manageable.

Steve Milloy, author of the Junk Science web page, also points out that side effect data is often a secondary endpoint in most clinical trials.

None of the clinical trials giving rise to the questions about Vioxx, Celebrex and Aleve were, after all, specifically designed to test whether the drugs posed a heart attack or stroke risk. The data on Vioxx came from a study of gastrointestinal effects; the Celebrex data came from a cancer prevention study; and the Alleve data came from an Alzheimer’s prevention study. --,2933,142453,00.html

In actuality, there were two trials that indicated problems with Vioxx: the VIGOR trial (VIOXX Gastrointestinal Outcomes Research), published in the November 23, 2000 issue of NEJM and the APPROVe trial (Adenomatous Polyp Prevention on VIOXX), which remains unpublished, but which was stopped early in September 2004. According to a Merck press release, the APPROVe trial

was designed to evaluate the efficacy of VIOXX 25 mg in preventing recurrence of colorectal polyps in patients with a history of colorectal adenomas. In this study, there was an increased relative risk for confirmed cardiovascular events, such as heart attack and stroke, beginning after 18 months of treatment in the patients taking VIOXX compared to those taking placebo. --

For Celebrex/Celccoxib, the APC trial (Adenoma Prevention with Celecoxib), a large colorectal cancer prevention trial, was stopped early on December 17, 2004 because the patients taking Celecoxib had an increased risk of cardiovascular events (Cardiovascular death, myocardial infarction, or stroke). In the placebo group, there were 6 events among the 679 patients. In the Celecoxib 200 mg BID group, there were 15 events among 685 patients. In the Celecoxib 400 mg BID group, there were 19 events among 671 patients. The relative risk for both Celecoxib groups combined versus placebo was 2.8 (95% CI 1.2 to 6.7). Interestingly enough, the Pre-SAP trial (Prevention of Spontaneous Adenomatopus Polyps Trial), another cancer prevention trial, showed no change in the risk of cardiovascular events (Placebo 11/628 events, Celecoxib 400 mg QD 16/933 events, relative risk 1.0, 95% CI 0.5 to 2.1).

When you are evaluating the side effects of a drug, you need to consider the individual values and preferences of the patients. What are the benefits of the drug for this particular patient? For many patients there are good alternative drugs; for others, this may be the only drug that offers relief. What is the patient's risk profile for the potential side effects. A healthy patient who has a vanishingly small chance of seeing a side effect may be willing to tolerate a doubling or tripling of risk. A patient who is already at a high level of risk may not be so willing. A recent FDA guidance on these drugs suggests

Patients who are at a high risk of gastrointestinal (GI) bleeding, have a history of intolerance to non-selective NSAIDs, or are not doing well on non-selective NSAIDs may be appropriate candidates for Cox-2 selective agents. --

These comments are not intended to exonerate the drug companies or the FDA. Some people have suggested that the drug companies have marketed these drugs too aggressively, have not performed the appropriate safety tests for these drugs, and/or have failed to inform the public about what they knew about the risks of these drugs. Others have suggested that FDA has an inherent conflict because of its role in drug approval and that drug safety monitoring should be handled by an independent agency. Whether or not these allegations are true, it is still worth remembering that testing for side effects is very difficult to do.

Further reading

There is way too much on this issue to summarize in a short bibliography, but here are a few useful starting points.

  1. Balancing benefits and harms: the example of non-steroidal anti-inflammatory drugs. Dieppe P, Bartlett C, Davey P, Doyal L, Ebrahim S. Bmj 2004: 329(7456); 31-4. [Medline] [Full text] [PDF]
  2. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ioannidis JP, Evans SJ, Gotzsche PC, O'Neill RT, Altman DG, Schulz K, Moher D. Ann Intern Med 2004: 141(10); 781-8. [Medline] [Abstract] [Full text] [PDF]
  3. Failing the public health--rofecoxib, Merck, and the FDA. Topol EJ. N Engl J Med 2004: 351(17); 1707-9. [Medline] [Full text] [PDF]

This page was written by Steve Simon while working at Children's Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children's Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Measuring benefit/risk