StATS: Clinical importance (created 2005-03-11).
Many journal authors have the bad habit of looking just at the p-value of a
study and ignoring the clinical importance of their findings. If they get a
small p-value, which indicates a statistically significant difference between
the new therapy and the standard therapy, they dance in the streets, they pop
open the champagne bottles, they celebrate wildly, and they publish their
results in an "A" journal. If they get a large p-value, they rend their
clothes, they throw ashes on their heads, they wail and moan, and they
publish their results in a "C" journal.
An article about measurement of fatigue
- Measurement of Fatigue Determining Minimally Important Cllinical
Differences. Schwartz AL, Meek PM, Nail LM, Fargo J, Lundquist M,
Donofrio M, Grainger M, Throckmorton T, Mateo M. Journal of Clinical
Epidemiology 2002: 55(3); 239 - 244.
offers some valuable lessons about clinically relevant differences.
Cancer patients have major problems with fatigue. The only good measure is
a self-report, and this can be measured in several different ways:
- Profile of Mood States, a 65 item scale with a subscale of five items
representing fatigue. Each item is rated from 0 to 4.
- Schwartz Cancer Fatigue Scale, a 28 item scale with four subscales:
physical, emotional, cognitive, and temporal. Each item is rated from
0 to 4.
- General Fatigue Scale a ten item scale with no subscales. Each item is
rated from 1 to 10.
- A ten point single item scale.
The last scale asked the question "what is your level of fatigue today"
with 0 representing "no fatigue" and 10 representing "the greatest possible
fatigue." There's a slight error here, because if you count properly, there
are 11 numbers in the range from 0 to 10.
The researchers measured a group of 103 cancer patients before and after
initiation of chemotherapy. In addition to getting the four scales, the
patients were asked at follow-up whether their fatigue levels had changed and
by how much. Interestingly, 30 subjects reported a decrease in fatigue, but
the average scores on all four scales for these patients did not differ from
their peers who reported no change in fatigue. Those who reported an increase
in fatigue did differ from those reporting no change. What this means is
difficult to interpret, but the authors feel that patients may perceive
increases in fatigue differently than decreases in fatigue.
If you look at the average change in each scale for those patients who
report a small change in fatigue, this represents a minimally important
clinical difference. The numbers don't seem to quite match the tables, but
the authors suggest that a 5.6 unit shift in POMS, 5.0 for SCFS, 9.7 for GFS,
and 2.4 for the single item scale. If you divide each of these values by the
number of items in the scale, you get values that hover around 1.0 for the
first three scales, which is similar to a recently published paper in BMJ.
- Interpreting treatment effects in randomised trials. Guyatt GH,
Juniper E, Walter S, Griffith L, Goldstein R. British Medical Journal 1998:
- Clinically significant changes in pain along the visual analog scale.
Bird SB, Dickson EW. Ann Emerg Med 2001: 38(6); 639-43.
- The visual analog scale for pain: clinical significance in
postoperative patients. Bodian CA, Freedman G, Hossain S, Eisenkraft JB,
Beilin Y. Anesthesiology 2001: 95(6); 1356-61.
- How well is the clinical importance of study results reported? An
assessment of randomized controlled trials. Chan KB, Man-Son-Hing M,
Molnar FJ, Laupacis A. Cmaj 2001: 165(9); 1197-202.
- Setting the minimal metrically detectable change on disability rating
scales. Hebert R, Spiegelhalter DJ, Brayne C. Arch Phys Med Rehabil
1997: 78(12); 1305-8.
- Interpreting thresholds for a clinically significant change in health
status in asthma and COPD. Jones PW. Eur Respir J 2002: 19(3); 398-404.
- Quality of life questionnaires: does statistically significant =
clinically important? Juniper EF. J Allergy Clin Immunol 1998: 102(1);
- Does the clinically significant difference in visual analog scale
pain scores vary with gender, age, or cause of pain? Kelly AM. Acad
Emerg Med 1998: 5(11); 1086-90.
- A proposal to use confidence intervals for visual analog scale data
for pain measurement to determine clinical significance. Mantha S,
Thisted R, Foss J, Ellis JE, Roizen MF. Anesth Analg 1993: 77(5); 1041-7.
- Can there be a more patient-centred approach to determining
clinically important effect sizes for randomized treatment trials?
Naylor CD. J Clin Epidemiol 1994: 47(7); 787-95.
- Determining the minimum clinically significant difference in visual
analog pain score for children. Powell CV, Kelly AM, Williams A. Ann
Emerg Med 2001: 37(1); 28-31.
- Assessing clinically significant change: application to the SCL-90-R.
Schmitz N, Hartkamp N, Franke GH. Psychol Rep 2000: 86(1); 263-74.
- Clinical utility and clinical significance in the assessment and
management of pain in vulnerable infants. Stevens B, Gibbins S. Clin
Perinatol 2002: 29(3); 459-68.
- What is the relationship between the minimally important difference
and health state utility values? The case of the SF-6D. Walters SJ,
Brazier JE. Health Qual Life Outcomes 2003: 1(1); 4.
- Minimum clinically significant VAS differences for simultaneous
(paired) interval serial pain assessments. Yamamoto LG, Nomura JT, Sato
RL, Ahern RM, Snow JL, Kuwaye TT. Am J Emerg Med 2003: 21(3); 176-9.
This page was written by
Steve Simon while working at Children's Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children's Mercy Hospital website. Need more
information? I have a page with general help
resources. You can also browse for pages similar to this one at Category: Clinical importance.