StATS: Calculations involving diagnostic tests using open source abstracts (October 5, 2006)
I spent a few hours reviewing 200+ abstracts published in BiomedCentral that had the words "sensitivity" and "specificity" in the title. There were four which had enough information in the abstract to be used as teaching examples on how to calculate sensitivity, specificity, positive predictive value, and/or negative predictive value.
1. Read the following abstracts. Calculate the values of sensitivity, specificity, positive predictive value, or negative predictive value as requested. If you need a hint, some helpful tables from the text of the articles is included below.
a. Frozen section is superior to imprint cytology for the intra-operative assessment of sentinel lymph node metastasis in Stage I Breast cancer patients. Mori M, Tada K, Ikenaga M, Miyagi Y, Nishimura S, Takahashi K, Makita M, Iwase T, Kasumi F, Koizumi M. World Journal of Surgical Oncology 2006, 4:26 (17 May 2006) [Abstract] [Full Text] [PDF] Background A standard intra-operative procedure for assessing sentinel lymph node metastasis in breast cancer patients has not yet been established. Patients and methods One hundred and thirty-eight patients with stage I breast cancer who underwent sentinel node biopsy using both imprint cytology and frozen section were analyzed. Results Seventeen of the 138 patients had sentinel node involvement. Results of imprint cytology included nine false negative cases (sensitivity,
[[CALCULATE THIS VALUE]]). In contrast, only two cases of false negatives were found on frozen section (sensitivity,
[[CALCULATE THIS VALUE]]). There were two false positive cases identified by imprint cytology (specificity,
[[CALCULATE THIS VALUE]]). On the other hand, frozen section had 100% specificity. Conclusion These findings suggest that frozen section is superior to imprint cytology for the intra-operative determination of sentinel lymph node metastasis in stage I breast cancer patients.
b. The Single Item Literacy Screener: Evaluation of a brief instrument to identify limited reading ability. Morris NS, MacLean CD, Chew LD, Littenberg B. BMC Family Practice 2006, 7:21 (24 March 2006) [Abstract] [Full text] [PDF] Background Reading skills are important for accessing health information, using health care services, managing one's health and achieving desirable health outcomes. Our objective was to assess the diagnostic accuracy of the Single Item Literacy Screener (SILS) to identify limited reading ability, one component of health literacy, as measured by the S-TOFHLA. Methods Cross-sectional interview with 999 adults with diabetes residing in Vermont and bordering states. Participants were randomly recruited from Primary Care practices in the Vermont Diabetes Information System June 2003 – December 2004. The main outcome was limited reading ability. The primary predictor was the SILS. Results Of the 999 persons screened, 169 (17%) had limited reading ability. The sensitivity of the SILS in detecting limited reading ability was 54% [95% CI: 47%, 61%] and the specificity was 83% [95% CI: 81%, 86%] with an area under the Receiver Operating Characteristics Curve (ROC) of 0.73 [95% CI: 0.69, 0.78]. Seven hundred seventy (77%) screened negative on the SILS and 692 of these subjects had adequate reading skills (negative predictive value =
[[CALCULATE THIS VALUE]][95% CI: 0.88, 0.92]). Of the 229 who scored positive on the SILS, 92 had limited reading ability (positive predictive value =
[[CALCULATE THIS VALUE]][95% CI: 0.34, 0.47]). Conclusion The SILS is a simple instrument designed to identify patients with limited reading ability who need help reading health-related materials. The SILS performs moderately well at ruling out limited reading ability in adults and allows providers to target additional assessment of health literacy skills to those most in need. Further study of the use of the SILS in clinical settings and with more diverse populations is warranted.
c. Gait disorders are associated with non-cardiovascular falls in elderly people: a preliminary study. Montero-Odasso M, Schapira M, Duque G, Soriano ER, Kaplan R, Camera LA. BMC Geriatrics 2005, 5:15 (1 December 2005) [Abstract] [Full text] [PDF] Background The association between unexplained falls and cardiovascular causes is increasingly recognized. Neurally mediated cardiovascular disorders and hypotensive syndromes are found in almost 20 percent of the patients with unexplained falls. However, the approach to these patients remains unclear. Gait assessment might be an interesting approach to these patients as clinical observations suggests that those with cardiovascular or hypotensive causes may not manifest obvious gait alterations. Our primary objective is to analyze the association between gait disorders and a non-cardiovascular cause of falls in patients with unexplained falls. A second objective is to test the sensitivity and specificity of a gait assessment approach for detecting non-cardiovascular causes when compared with intrinsic-extrinsic classification. Methods Cross-sectional study performed in a falls clinic at a university hospital in 41 ambulatory elderly participants with unexplained falls. Neurally mediated cardiovascular conditions, neurological diseases, gait and balance problems were assessed. Gait disorder was defined as a gait velocity < 0.8 m/s or Tinetti Gait Score <9. An attributable etiology of the fall was determined in each participant. Comparisons between the gait assessment approach and the attributable etiology regarding a neurally mediated cardiovascular cause were performed. Fisher exact test was used to test the association hypothesis. Sensitivity and specificity of gait assessment approach and intrinsic-extrinsic classification to detect a non-cardiovascular mediated fall was calculated with 95% confidence intervals (CI95%). Results A cardiovascular etiology (orthostatic and postprandial hypotension, vasovagal syndrome and carotid sinus hypersensitivity) was identified in 14% of participants (6/41). Of 35 patients with a gait disorder, 34 had a non-cardiovascular etiology of fall; whereas in 5 out of 6 patients without a gait disorder, a cardiovascular diagnosis was identified (p < 0.001). Sensitivity and specificity of the presence of gait disorder for identifying a non-cardiovascular mediated cause was
[[CALCULATE THIS VALUE]](CI95% = 85–99) and
[[CALCULATE THIS VALUE]](CI95% = 36–99), respectively. Conclusion In community dwelling older persons with unexplained falls, gait disorders were associated with non-cardiovascular diagnosis of falls. Gait assessment was a useful approach for the detection of a non-cardiovascular mediated cause of falls, providing additional value to this assessment.
d. Software PREP-Mt: predictive RNA editor for plant mitochondrial genes. Mower JP. BMC Bioinformatics 2005, 6:96 (12 April 2005) [Abstract] [Full text] [PDF] Background In plants, RNA editing is a process that converts specific cytidines to uridines and uridines to cytidines in transcripts from virtually all mitochondrial protein-coding genes. There are thousands of plant mitochondrial genes in the sequence databases, but sites of RNA editing have not been determined for most. Accurate methods of RNA editing site prediction will be important in filling in this information gap and could reduce or even eliminate the need for experimental determination of editing sites for many sequences. Because RNA editing tends to increase protein conservation across species by "correcting" codons that specify unconserved amino acids, this principle can be used to predict editing sites by identifying positions where an RNA editing event would increase the conservation of a protein to homologues from other plants. PREP-Mt takes this approach to predict editing sites for any protein-coding gene in plant mitochondria. Results To test the general applicability of the PREP-Mt methodology, RNA editing sites were predicted for 370 full-length or nearly full-length DNA sequences and then compared to the known sites of RNA editing for these sequences. Of 60,263 cytidines in this test set, PREP-Mt correctly classified 58,994 as either an edited or unedited site (accuracy = 97.9%). PREP-Mt properly identified 3,038 of the 3,698 known sites of RNA editing (sensitivity =
[[CALCULATE THIS VALUE]]) and 55,956 of the 56,565 known unedited sites (specificity =
[[CALCULATE THIS VALUE]]). Accuracy and sensitivity increased to 98.7% and 94.7%, respectively, after excluding the 489 silent editing sites (which have no effect on protein sequence or function) from the test set. Conclusion These results indicate that PREP-Mt is effective at identifying C to U RNA editing sites in plant mitochondrial protein-coding genes. Thus, PREP-Mt should be useful in predicting protein sequences for use in molecular, biochemical, and phylogenetic analyses. In addition, PREP-Mt could be used to determine functionality of a mitochondrial gene or to identify particular sequences with unusual editing properties. The PREP-Mt methodology should be applicable to any system where RNA editing increases protein conservation across species.
Hint for 1a. Two tables from Frozen section is superior to imprint cytology for the intra-operative assessment of sentinel lymph node metastasis in Stage I Breast cancer patients that might help with your calculations.
Hint for 1b. One table from The Single Item Literacy Screener: Evaluation of a brief instrument to identify limited reading ability that might help with your calculations.
Hint for 1c. One table from Gait disorders are associated with non-cardiovascular falls in elderly people: a preliminary study that might help with your calculations.
Hint for 1d. One table from Software PREP-Mt: predictive RNA editor for plant mitochondrial genes that might help with your calculations.
This page was written by Steve Simon while working at Children's Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children's Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Diagnostic testing.