StATS: Number Needed to Treat (created 2000-01-27).
Dear Professor Mean, How are patients and their doctors supposed to decide whether a research finding has practical significance? Why don't the medical journals make things clearer?
You're hoping for clarity from medical profession? These are the folks who take a simple ear ache and call it "otitis media." To them, a runny nose is "rhinorhea" and a tummy ache is "gastrointestinal distress." It's enough to make me produce lacrimal secretions.
In fairness to these folks, though, they do realize that practical interpretation of the medical research is difficult. They are trying to change it. There are two important changes that we are starting to see in medical research papers. First, they have learned that you can't ignore the size of the effect and focus only on the statistical significance. Since confidence intervals provide information about both the size and significance, many journals include them instead of p-values.
A second change is the realization that absolute changes in risk are more important than relative changes in risk. A nurse recently informed me that my snoring (oops! sleep apnea) can triple the risk of a stroke (excuse me, a cerebrovascular event) if left untreated. But how serious is that for someone who is only 42 years old and otherwise in good health? Three times nothing is nothing, and three times something very small is still very small. I decided to get treatment, but it was more for helping me and my wife to sleep better than a concern about stroke.
A good measure of the absolute risk is the number needed to treat (NNT). It is the average number of patients that a doctor would need to treat in order to have one additional event occur. A small value (e.g., NNT=2.7) means that a doctor will see a lot of events in very little time. A large value (e.g., NNT=800) means that the doctor will have to treat a large number of patients in order to see a very few events.
When you are measuring an increase in bad events like side effects that might be associated with a treatment, then the number needed to treat is sometimes described as the number needed to harm (NNH). Often you can quantify the tradeoffs between the benefits and side effects of a treatment by comparing the NNT and NNH values.
Here are some examples of Numbers Needed to Treat, found at the Bandolier web site (http://www.jr2.ox.ac.uk/bandolier/index.html).
Prevention of post-operative vomiting using Droperidol, NNT=4.4. For every four or five surgery patients treated with Droperidol, you will see one less vomiting incident on average.
Prevention of infection from dog bites using antibiotics, NNT=16. For every 16 dog bites treated with antibiotics, you would see one fewer infection on average.
Primary prevention of stroke using a daily low dose of aspirin for one year, NNT=102. For every hundred patient years of treatment with aspirin, you will see one fewer stroke on average.
Notice that this last event is a rate. Assuming that the rates are reasonably homogenous over time, one hundred patient years is equivalent to following ten patients for a decade. Be careful, of course, of rates that are not homogenous over time. If the rates decline the longer you follow your patients, then the number of events you will see for one hundred patients during their first year of therapy would be quite different from the number of events you would see following ten patients for their first decade of therapy.
Here's another example from the British Medical Journal (Freemantle 1999: 318(7200); 1730-1737). Prevention of cardiac death using beta blockers among patients with previous myocardial infarction, NNT=42. For every 42 patients treated for two years with beta blockers, you would see one fewer death. This is superior to treatment with antiplatelet agents (NNT=153), Statins (NNT=94), or Warfarin (NNT=63), but not as effective as thrombolysis and aspirin for 4 weeks (NNT=24).
To compute the NNT, you need to subtract the rate in the treatment group from the rate in the control group and then invert it (divide the difference into 1).
A recently published article on the flu vaccine showed that among the children who received a placebo, 17.9% later had culture confirmed influenza. In the vaccine group, the rate was only 1.3%. This is a 16.6% absolute difference. When you invert this percentage, you get NNT=6. This means that for every six kids who get the vaccine, you will see one less case of flu on average.
The study also looked at the rate of side effects. In the vaccine group, 1.9% developed a fever. Only 0.8% of the controls developed a fever. This is an absolute difference of 1.1%. When you invert this percentage, you get NNH=90. This means that for every 90 kids who get the vaccine, you will see one additional fever on average.
Sometimes the ratio between NNT and NNH can prove informative. For this study, NNH/NNT=90/6=15. This tells you that you should expect to see one additional fever for every fifteen cases of flu prevented.
Although I am not a medical expert, the vaccine looks very promising because you can prevent a lot of flu events and only have to put up with a few additional fevers. In general, it takes medical judgment to assess the trade-offs between the benefits of a treatment and its side effects. The NNT and NNH calculations allow you to assess there trade-offs.
What if the outcome measure is continuous?
To calculate the NNT or NNH, you need to have a distinct event. With a continuous variable, you could define such an event by setting a cut-off. For example, an intervention to improve breastfeeding rates might improve the average duration of breastfeeding by seven weeks. How would you calculate the NNT for this data? Well, you might declare that you are interested in the proportion of mothers who breastfeed for at least 12 weeks. If you had access to the original data, you would find that 54% of women in the control group and 87% in the treatment group breastfed for at least 12 weeks. This would allow you to compute an NNT of 3. For every three mothers given the new intervention, one additional mother would breastfeed beyond 12 weeks.
The choice of 12 weeks is somewhat arbitrary and you would get different results if you chose a different cut-off, such as 24 weeks. You should choose a value that has clinical relevance to your colleagues.
Calculating the NNT or NNH from a continuous measure using a cutoff is usually impossible to do after the fact. So if you are reading someone else's work and they present the data as a mean difference, you cannot calculate NNT or NNH. You would need additional information, such as the proportions that exceed some threshold, or you would have to make some questionable assumptions, such as normality for the outcome measure.
Professor Mean explains that the journals are getting better at presenting the practical implications of the research. In particular, they are presenting the number needed to treat, a measure that helps you better understand the practical significance of your research findings. The number needed to treat is the average number of patients that you will have to treat with a new therapy to see one additional success, on average, compared to the standard therapy.
Additional references that I need to add to my bibliography
This page was written by Steve Simon while working at Children's Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children's Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Measuring benefit and risk