P.Mean >> Category >> Exclusions in research studies (created 2007-06-18).

These pages discuss the problems with generalizability that occur when researchers include important segments of the population from their research or when research subjects refuse to participate. Articles are arranged by date with the most recent entries at the top. You can find outside resources at the bottom of this page. Other entries about exclusions in research studies can be found in the exclusions in research page at the StATS website.

2011

9. The Monthly Mean: Why is a 20% dropout rate bad? (September-Novmeber 2011) and P.Mean: Why is a 20% dropout rate bad? (created 2011-11-21). Dear Professor Mean, How can we give an evidence based answer about why 20% loss of follow-up in a randomized trial is too much?

2010

8. The Monthly Mean: Is it ethical to exclude non-English speaking patients from a clinical trial? (September/October 2010)

7. P.Mean: Is it ethical to recruit a panhandler that you see on the street into your research study (created 2010-09-01). Someone asked a question about the ethics of approaching a panhandler and sharing information about a research study. I don't know all the details, but apparently, this study was examining veterans of the Iraq war, and this panhandler was holding a sign saying something like please help a veteran of the Iraq war. There was some concern about whether the monetary incentive would be disproportionate for someone who had to beg for a living, or it might be a problem if the panhandler was given money and a flyer about the research study at the same time. I discussed some of my concerns about this study, but it was from the perspective of statistical validity rather than from an ethical perspective.

6. P.Mean: Withdrawing from a study and taking your data with you (created 2010-05-15). Someone asked me what the phrase "you can withdraw from the study at any time" really means. Can a research subject withdraw and take their data with them (that is, ask that their data be expunged from the database)? What if they raise the objection after the data analysis is done, because they don't like the results of the study. Can they ask for their data to be expunged then? What if they raise the objection after the data is published?

2009

5. P.Mean: Sneaking ineligible patients into a clinical trial (created 2009-10-30). There was an interesting article in the New York Times (Chen PW. Bending the Rules of Clinical Trials) that described a terminal cancer patient and the doctor's goal to get them access to a new experimental drug, even though the patient was not eligible for the clinical trial that was studying this drug. It's a difficult situation for doctors. Do you do what's best for the patient in front of you, knowing that the data collected from this patient might corrupt the findings of the clinical trial?

Outside resources:

Mark Zimmerman, Jill I. Mattia, Michael A. Posternak. Are Subjects in Pharmacological Treatment Trials of Depression Representative of Patients in Routine Clinical Practice?. Am J Psychiatry. 2002;159(3):469-473. Abstract: "OBJECTIVE: The methods used to evaluate the efficacy of antidepressants differ from treatment for depression in routine clinical practice. The rigorous inclusion/exclusion criteria used to select subjects for participation in efficacy studies potentially limit the generalizability of these trials' results. It is unknown how much impact these criteria have on the representativeness of subjects in efficacy trials. This study estimated the proportion of depressed patients treated in routine clinical practice who would meet standard inclusion/exclusion criteria for an efficacy trial. METHOD: A total of 803 individuals, aged 16-65 years, who were seen at intake at an outpatient practice underwent a thorough diagnostic evaluation, including the administration of semistructured diagnostic interviews; 346 patients had current major depression. Common inclusion/exclusion criteria used in efficacy studies of antidepressants were applied to the depressed patients to determine how many would have qualified for an efficacy trial. RESULTS: Approximately one-sixth of the 346 depressed patients would have been excluded from an efficacy trial because they had a bipolar or psychotic subtype of depression. The presence of a comorbid anxiety or substance use disorder, insufficient severity of depressive symptoms, or current suicidal ideation would have excluded 86.0% (N=252) of the remaining 293 outpatients with nonpsychotic unipolar major depressive disorder from an antidepressant efficacy trial. CONCLUSIONS: Subjects treated in antidepressant trials represent a minority of patients treated for major depression in routine clinical practice. These results show that antidepressant efficacy trials tend to evaluate a subset of depressed individuals with a specific clinical profile." [Accessed December 9, 2009]. Available at: http://ajp.psychiatryonline.org/cgi/content/abstract/159/3/469.

Alyson Littman, Edward Boyko, Isabel Jacobson, et al. Assessing nonresponse bias at follow-up in a large prospective cohort of relatively young and mobile military service members. BMC Medical Research Methodology. 2010;10(1):99. Abstract: "BACKGROUND: Nonresponse bias in a longitudinal study could affect the magnitude and direction of measures of association. We identified sociodemographic, behavioral, military, and health-related predictors of response to the first follow-up questionnaire in a large military cohort and assessed the extent to which nonresponse biased measures of association. METHODS: Data are from the baseline and first follow-up survey of the Millennium Cohort Study. Seventy-six thousand, seven hundred and seventy-five eligible individuals completed the baseline survey and were presumed alive at the time of follow-up; of these, 54,960 (71.6%) completed the first follow-up survey. Logistic regression models were used to calculate inverse probability weights using propensity scores. RESULTS: Characteristics associated with a greater probability of response included female gender, older age, higher education level, officer rank, active-duty status, and a self-reported history of military exposures. Ever smokers, those with a history of chronic alcohol consumption or a major depressive disorder, and those separated from the military at follow-up had a lower probability of response. Nonresponse to the follow-up questionnaire did not result in appreciable bias; bias was greatest in subgroups with small numbers. CONCLUSIONS: These findings suggest that prospective analyses from this cohort are not substantially biased by non-response at the first follow-up assessment." [Accessed October 25, 2010]. Available at: http://www.biomedcentral.com/1471-2288/10/99.

C Bartlett, L Doyal, S Ebrahim, et al. The causes and effects of socio-demographic exclusions from clinical trials. Excerpt: "The exclusion from trials of people likely to be in need of or to benefit from an intervention could compromise the trials' generalisability. We investigated the exclusion of women, older people and minority ethnic groups, focusing on two drug exemplars, statins and non-steroidal anti-inflammatory drugs (NSAIDs)." [Accessed May 19, 2010]. Available at: http://www.hta.ac.uk/execsumm/summ938.shtml.

Harriette G. C. Van Spall, Andrew Toren, Alex Kiss, Robert A. Fowler. Eligibility Criteria of Randomized Controlled Trials Published in High-Impact General Medical Journals: A Systematic Sampling Review. JAMA. 2007;297(11):1233-1240. Abstract: "Context: Selective eligibility criteria of randomized controlled trials (RCTs) are vital to trial feasibility and internal validity. However, the exclusion of certain patient populations may lead to impaired generalizability of results. Objective: To determine the nature and extent of exclusion criteria among RCTs published in major medical journals and the contribution of exclusion criteria to the representation of certain patient populations. Data Sources and Study Selection: The MEDLINE database was searched for RCTs published between 1994 and 2006 in certain general medical journals with a high impact factor. Of 4827 articles, 283 were selected using a series technique. Data Extraction: Trial characteristics and the details regarding exclusions were extracted independently. All exclusion criteria were graded independently and in duplicate as either strongly justified, potentially justified, or poorly justified according to previously developed and pilot-tested guidelines. Data Synthesis: Common medical conditions formed the basis for exclusion in 81.3% of trials. Patients were excluded due to age in 72.1% of all trials (60.1% in pediatric populations and 38.5% in older adults). Individuals receiving commonly prescribed medications were excluded in 54.1% of trials. Conditions related to female sex were grounds for exclusion in 39.2% of trials. Of all exclusion criteria, only 47.2% were graded as strongly justified in the context of the specific RCT. Exclusion criteria were not reported in 12.0% of trials. Multivariable analyses revealed independent associations between the total number of exclusion criteria and drug intervention trials (risk ratio, 1.35; 95% confidence interval, 1.11-1.65; P = .003) and between the total number of exclusion criteria and multicenter trials (risk ratio, 1.26; 95% confidence interval, 1.06-1.52; P = .009). Industry-sponsored trials were more likely to exclude individuals due to concomitant medication use, medical comorbidities, and age. Drug intervention trials were more likely to exclude individuals due to concomitant medication use, medical comorbidities, female sex, and socioeconomic status. Among such trials, justification for exclusions related to concomitant medication use and comorbidities were more likely to be poorly justified. Conclusions: The RCTs published in major medical journals do not always clearly report exclusion criteria. Women, children, the elderly, and those with common medical conditions are frequently excluded from RCTs. Trials with multiple centers and those involving drug interventions are most likely to have extensive exclusions. Such exclusions may impair the generalizability of RCT results. These findings highlight a need for careful consideration and transparent reporting and justification of exclusion criteria in clinical trials." [Accessed January 15, 2010]. Available at: http://jama.ama-assn.org/cgi/content/abstract/297/11/1233.

David Tuller. Following Trail of Lost AIDS Patients in Africa. The New York Times. 2010. Excerpt: "At Peter’s clinic, as elsewhere in Africa, patients who have not come for their medications in recent months are considered to have defaulted from treatment. As a “defaulter tracer,” Peter tries to track them down, find out what’s gone wrong and get them back into treatment, if possible. Epidemiologists refer to such patients as “lost to follow-up,” and their increasing numbers in sub-Saharan Africa are causing concern among providers of H.I.V. and AIDS care. Interruptions in treatment lead to viral strains that are resistant to the cheapest medications, and to higher rates of illness and death." [Accessed October 27, 2010]. Available at: http://www.nytimes.com/2010/10/26/health/26cases.html?_r=1&nl=health&emc=healthupdateema2.

U.S. Food and Drug Administration. Guidance for Sponsors, Clinical Investigators, and IRBs: Data Retention When Subjects Withdraw from FDA-Regulated Clinical Trials. Excerpt: "This guidance is intended for sponsors, clinical investigators and institutional review boards (IRBs). It describes the Food and Drug Administration's (FDA) longstanding policy that already-accrued data, relating to individuals who cease participating in a study, are to be maintained as part of the study data. This pertains to data from individuals who decide to discontinue participation in a study, who are withdrawn by their legally authorized representative, as applicable, or who are discontinued from participation by the clinical investigator. This policy is supported by the statutes and regulations administered by FDA as well as ethical and quality standards applicable to clinical research. Maintenance of these records includes, as with all study records, safeguarding the privacy and confidentiality of the subject's information." [Accessed September 22, 2010]. Available at: http://www.fda.gov/downloads/RegulatoryInformation/Guidances/UCM126489.pdf.

Office for Human Research Protections. Guidance on Withdrawal of Subjects from Research: Data Retention and Other Related Issues. Excerpt: "This document applies to non-exempt human subjects research conducted or supported by HHS. It clarifies that when a subject chooses to withdraw from (i.e., discontinue his or her participation in) an ongoing research study, or when an investigator terminates a subject's participation in such a research study without regard to the subject's consent, the investigator may retain and analyze already collected data relating to that subject, even if that data includes identifiable private information about the subject. For HHS-conducted or supported research that is regulated by the Food and Drug Administration (FDA), FDA's guidance on this issue also should be consulted." [Accessed September 22, 2010]. Available at: http://www.hhs.gov/ohrp/policy/subjectwithdrawal.html.

Samuel Brilleman, Nancy Pachana, Annette Dobson. The impact of attrition on the representativeness of cohort studies of older people. BMC Medical Research Methodology. 2010;10(1):71. Abstract: "BACKGROUND: There are well-established risk factors, such as lower education, for attrition of study participants. Consequently, the representativeness of the cohort in a longitudinal study may deteriorate over time. and the results may become less generalisable to the target population. Death is a common form of attrition in cohort studies of older people. The aim of this paper is to examine the effects of death and other forms of attrition on risk factor prevalence in the study cohort and the target population over time. METHODS: Differential associations between a risk factor and death and non-death attrition are considered under various hypothetical conditions. Empirical data from the Australian Longitudinal Study on Women's Health (ALSWH) for participants born in 1921-26 are used to identify associations which occur in practice, and national cross-sectional data from Australian Censuses and National Health Surveys are used to illustrate the evolution of bias over approximately ten years. RESULTS: The hypothetical situations illustrate how death and other attrition can theoretically affect changes in bias over time. Between 1996 and 2008, 28.4% of ALSWH participants died, 16.5% withdrew and 10.4% were lost to follow up. There were differential associations with various risk factors, for example, non-English speaking country of birth was associated with non-death attrition but not death whereas being underweight (body mass index < 18.5) was associated with death but not other forms of attrition. Compared to national data, underrepresentation of women with non-English speaking country of birth increased from 3.9% to 7.2% and over-representation of current and ex-smoking increased from 2.6% to 5.8%. CONCLUSIONS: Deaths occur in both the target population and study cohort, while other forms of attrition occur only in the study cohort. Therefore non-death attrition may cause greater bias than death in longitudinal studies. This is an important issue for studies of older people where attrition due to death and other health related causes is likely. However although more than a quarter of the oldest participants in the ALSWH died in the 12 years following recruitment, differences from the national population changed only slightly." [Accessed August 11, 2010]. Available at: http://www.biomedcentral.com/1471-2288/10/71.

Richard Feinman. Intention-to-treat. What is the question?. Nutrition & Metabolism. 2009;6(1):1. Abstract: "It has become commonplace for Randomized Controlled Trials (RCTs) to be analyzed according to Intention-to-Treat (ITT) principles in which data from all subjects are used regardless of the subjects' adherence to protocol. While ITT analyses can provide useful information in some cases, they do not answer the question that motivates many RCTs, namely, whether the treatments differ in efficacy. ITT tends to reduce information by combining two questions, whether the intervention is effective and whether, as implemented, it has good compliance. Because these questions may be separate there is a risk of misuse. Two examples are presented that demonstrate this potential for abuse: a study on the effectiveness of vitamin E in reducing cardiovascular risk and comparisons of low fat and low carbohydrate diets. In the first case, a treatment that is demonstrably effective is described as without merit. In the second, ITT describes as the same, two diets that actually have different outcomes. These misuses of ITT are not atypical and are not technical problems in statistics but have real consequences for scientific principles and health recommendations. ITT analyses may answer the question of what happens when treatments are recommended but are inappropriate where separate information on adherence and performance is available. It is proposed that results of RCTs, or any experimental study, be reported, not in terms of the analyses that were performed, but rather in terms of the questions that the analyses can answer properly." [Accessed February 24, 2009]. Available at: http://www.nutritionandmetabolism.com/content/6/1/1.

MediciGlobal. L2FU - Lost to Follow Up. Excerpt: "Patient drop outs in a clinical trial costs your company money. It can cost you the integrity of your study too! If it's important to recover patients lost from your clinical trial, you've come to the right place. Here, you'll read how L2FU's services can help you and how to begin finding patients today!" [Accessed July 26, 2010]. Available at: http://www.l2fu.com.

Journal article: Elie A Akl, Matthias Briel, John J You, Francois Lamontagne, Azim Gangji, Tali Cukierman-Yaffe, Mohamad Alshurafa, Xin Sun, Kara A Nerenberg, et al. LOST to follow-up Information in Trials (LOST-IT): a protocol on the potential impact Trials. 2009;10:40. Abstract: "BACKGROUND: Incomplete ascertainment of outcomes in randomized controlled trials (RCTs) is likely to bias final study results if reasons for unavailability of patient data are associated with the outcome of interest. The primary objective of this study is to assess the potential impact of loss to follow-up on the estimates of treatment effect. The secondary objectives are to describe, for published RCTs, (1) the reporting of loss to follow-up information, (2) the analytic methods used for handling loss to follow-up information, and (3) the extent of reported loss to follow-up. METHODS: We will conduct a systematic review of reports of RCTs recently published in five top general medical journals. Eligible RCTs will demonstrate statistically significant effect estimates with respect to primary outcomes that are patient-important and expressed as binary data. Teams of 2 reviewers will independently determine eligibility and extract relevant information from each eligible trial using standardized, pre-piloted forms. To assess the potential impact of loss to follow-up on the estimates of treatment effect we will, for varying assumptions about the outcomes of participants lost to follow-up (LTFU), calculate (1) the percentage of RCTs that lose statistical significance and (2) the mean change in effect estimate across RCTs. The different assumptions we will test are the following: (1) none of the LTFU participants had the event; (2) all LTFU participants had the event; (3) all LTFU participants in the treatment group had the event; none of those in the control group had it (worst case scenario); (4) the event incidence among LTFU participants (relative to observed participants) increased, with a higher relative increase in the intervention group; and (5) the event incidence among LTFU participants (relative to observed participants) increased in the intervention group and decreased in the control group. DISCUSSION: We aim to make our objectives and methods transparent. The results of this study may have important implications for both clinical trialists and users of the medical literature." [Accessed on November 21, 2011]. http://www.trialsjournal.com/content/10/1/40.

Chronic Disease Prevention and Control Research Center at Baylor College of Medicine. Major Deficiencies in the Design and Funding of Clinical Trials: A Report to the Nation Improving on How Human Studies Are Conducted. Excerpt: "Clinical trials are a critical resource for the discovery of new, life-saving drugs and for developing better prevention and diagnostic screening methods. Today's most effective prevention and treatment modalities are based on previous clinical trial results. But while the need for clinical research is undisputed, how clinical trials are now conducted remains problematic. Increasing research finds major deficiencies in the way clinical trials are designed, carried out and funded in the U.S. with serious implications for the outcomes of medical research studies. Of key significance for the future of scientific innovation is the exclusion or underrepresentation of women, older people, minorities, disabled persons, and rural populations in the vast majority of the research studies conducted in the U.S. Without adequate representation of all patient populations, researchers cannot learn about potential differences among groups and cannot ensure the generalization of results." [Accessed December 9, 2009]. Available at: http://www.bcm.edu/edict/PDF/EDICT_Project_White_Paper.pdf.

Cora Craig, Christine Cameron, Joe Griffiths, et al. Non-response bias in physical activity trend estimates. BMC Public Health. 2009;9(1):425. Abstract: "BACKGROUND: Increases in reported leisure time physical activity (PA) and obesity have been observed in several countries. One hypothesis for these apparently contradictory trends is differential bias in estimates over time. The purpose of this short report is to examine the potential impact of changes in response rates over time on the prevalence of adequate PA in Canadian adults. METHODS: Participants were recruited in representative national telephone surveys of PA from 1995-2007. Differences in PA prevalence estimates between participants and those hard to reach were assessed using Student's t tests adjusted for multiple comparisons. RESULTS: The number of telephone calls required to reach and speak with someone in the household increased over time, as did the percentage of selected participants who initially refused during the first interview attempt. A higher prevalence of adequate PA was observed with 5-9 attempts to reach anyone in the household in 1999-2002, but this was not significant after adjustment for multiple comparisons. CONCLUSIONS: No significant impact on PA trend estimates was observed due to differential non response rates. It is important for health policy makers to understand potential biases and how these may affect secular trends in all aspects of the energy balance equation." [Accessed November 29, 2009]. Available at: http://www.biomedcentral.com/1471-2458/9/425.

Tara Parker-Poe. Older Cancer Patients Often Excluded From Research. Excerpt: "The majority of people diagnosed with cancer are over 65, but most major cancer studies exclude them, leaving a wide gap in knowledge about how best to treat older patients. The knowledge gap about older cancer patients was highlighted recently by researchers from Barcelona who studied the role that age played in the prognosis of 224 cancer patients." [Accessed December 9, 2009]. Available at: http://well.blogs.nytimes.com/2008/11/17/older-cancer-patients-excluded-from-research/.

Irving Kuo. Randomization May Not be Valid in Tests of Psychotherapy vs Medications. Description: In a study comparing various combinations of medication and/or cognitive behavioral therapy for treating depression, only 1% of all patients surveyed found all seven arms of the study acceptable. This leads to serious problems with volunteer bias. [Accessed December 9, 2009]. Available at: http://www.medscape.com/viewarticle/564001.

Nicole Huang, Shu-Fang Shih, Hsing-Yi Chang, Yiing-Jenq Chou. Record linkage research and informed consent: who consents?. BMC Health Services Research. 2007;7(1):18. Abstract: "BACKGROUND: Linking computerized health insurance records with routinely collected survey data is becoming increasingly popular in health services research. However, if consent is not universal, the requirement of written informed consent may introduce a number of research biases. The participants of a national health survey in Taiwan were asked to have their questionnaire results linked to their national health insurance records. This study compares those who consented with those who refused. METHODS: A national representative sample (n = 14,611 adults) of the general adult population aged 20 years or older who participated in the Taiwan National Health Interview Survey (NHIS) and who provided complete survey information were used in this study. At the end of the survey, the respondents were asked if they would give permission to access their National Health Insurance records. Information given by the interviewees in the survey was used to analyze who was more likely to consent to linkage and who wasn't. RESULTS: Of the 14,611 NHIS participants, 12,911 (88%) gave consent, and 1,700 (12%) denied consent. The elderly, the illiterate, those with a lower income, and the suburban area residents were significantly more likely to deny consent. The aborigines were significantly less likely to refuse. No discrepancy in gender and self-reported health was found between individuals who consented and those who refused. CONCLUSION: This study is the first population-based study in assessing the consent pattern in a general Asian population. Consistent with people in Western societies, in Taiwan, a typical Asian society, a high percentage of adults gave consent for their health insurance records and questionnaire results to be linked. Consenters differed significantly from non-consenters in important aspects such as age, ethnicity, and educational background. Consequently, having a high consent rate (88%) may not fully eliminate the possibility of selection bias. Researchers should take this source of bias into consideration in their study design and investigate any potential impact of this source of bias on their results. [Accessed December 9, 2009]. Available at: http://www.biomedcentral.com/1472-6963/7/18.

Randomization Process in Question: Efficacy Trials Evaluating Psychotherapy vs Medications May Not Be Valid (Irving Kuo). Description: In a study comparing various combinations of medication and/or cognitive behavioral therapy for treating depression, only 1% of all patients surveyed found all seven arms of the study acceptable. This leads to serious problems with volunteer bias. This website was last verified on 2007-12-03. URL: www.medscape.com/viewarticle/564001

G S May, D L DeMets, L M Friedman, C Furberg, E Passamani. The randomized clinical trial: bias in analysis. Circulation. 1981;64(4):669-673. Abstract: "The realization that bias in patient selection may influence the results of clinical studies has helped to establish the randomized controlled clinical trial in medical research. However, bias can be equally important at other stages of a trial, especially at the time of analysis. Withdrawing patients from consideration in the analysis because of ineligibility on account of study entry criteria, lack of compliance to the protocol, or data of poor quality may be a source of systematic error. Examples to illustrate the possible consequences are taken from trials in the cardiovascular field. We recommended that reported study results should include outcome data from all subjects randomized in the group to which they were originally assigned." [Accessed December 9, 2009]. Available at: http://circ.ahajournals.org/cgi/content/abstract/64/4/669.

Record linkage research and informed consent: who consents? Nicole Huang, Shu-Fang Shih, Hsing-Yi Chang and Yiing-Jenq Chou. BMC Health Services Research 2007, 7:18 doi:10.1186/1472-6963-7-18. [Abstract] [PDF]. Description: Asking patients for permission before linking their data in a survey with health insurance records may be required from an ethical perspective, but it is well known to cause problems with selection bias. Those who agree to the linkage are different than those who refuse. In this study, researchers showed that age, income, literacy level, and other factors were different between patients who provided consent and those who did not provide consent.

Fred Andersen, Torgeir Engstad, Bjorn Straume, et al. Recruitment methods in Alzheimer's disease research: general practice versus population based screening by mail. BMC Medical Research Methodology. 2010;10(1):35. Abstract: "BACKGROUND: In Alzheimer's disease (AD) research patients are usually recruited from clinical practice, memory clinics or nursing homes. Lack of standardised inclusion and diagnostic criteria is a major concern in current AD studies. The aim of the study was to explore whether patient characteristics differ between study samples recruited from general practice and from a population based screening by mail within the same geographic areas in rural Northern Norway. METHODS: An interventional study in nine municipalities with 70000 inhabitants was designed. Patients were recruited from general practice or by population based screening of cognitive function by mail. We sent a questionnaire to 11807 individuals [greater than or equal to] 65 years of age of whom 3767 responded. Among these, 438 individuals whose answers raised a suspicion of cognitive impairment were invited to extended cognitive testing and a clinical examination. Descriptive statistics, chi-square, independent sample t-test and analyses of covariance adjusted for possible confounders were used. RESULTS: The final study samples included 100 patients recruited by screening and 87 from general practice. Screening through mail recruited younger and more self-reliant male patients with a higher MMSE sum score, whereas older women with more severe cognitive impairment were recruited from general practice. Adjustment for age did not alter the statistically significant differences of cognitive function, self-reliance and gender distribution between patients recruited by screening and from general practice. CONCLUSIONS: Different recruitment procedures of individuals with cognitive impairment provided study samples with different demographic characteristics. Initial cognitive screening by mail, preceding extended cognitive testing and clinical examination may be a suitable recruitment strategy in studies of early stage AD. Registration: ClinicalTrial.gov Identifier: NCT00443014" [Accessed May 6, 2010]. Available at: http://www.biomedcentral.com/1471-2288/10/35.

Journal article: Elke Vervölgyi, Mandy Kromp, Guido Skipka, Ralf Bender, Thomas Kaiser. Reporting of loss to follow-up information in randomised controlled trials with time-to-event outcomes: a literature survey BMC Med Res Methodol. 2011;11:130. Abstract: "BACKGROUND: To assess the reporting of loss to follow-up (LTFU) information in articles on randomised controlled trials (RCTs) with time-to-event outcomes, and to assess whether discrepancies affect the validity of study results. METHODS: Literature survey of all issues of the BMJ, Lancet, JAMA, and New England Journal of Medicine published between 2003 and 2005. Eligible articles were reports of RCTs including at least one Kaplan-Meier plot. Articles were classified as "assessable" if sufficient information was available to assess LTFU. In these articles, LTFU information was derived from Kaplan-Meier plots, extracted from the text, and compared. Articles were then classified as "consistent" or "not consistent". Sensitivity analyses were performed to assess the validity of study results. RESULTS: 319 eligible articles were identified. 187 (59%) were classified as "assessable", as they included sufficient information for evaluation; 140 of 319 (44%) presented consistent LTFU information between the Kaplan-Meier plot and text. 47 of 319 (15%) were classified as "not consistent". These 47 articles were included in sensitivity analyses. When various imputation methods were used, the results of a chi2-test applied to the corresponding 2 × 2 table changed and hence were not robust in about half of the studies. CONCLUSIONS: Less than half of the articles on RCTs using Kaplan-Meier plots provide assessable and consistent LTFU information, thus questioning the validity of the results and conclusions of many studies presenting survival analyses. Authors should improve the presentation of both Kaplan-Meier plots and LTFU information, and reviewers of study publications and journal editors should critically appraise the validity of the information provided." [Accessed on December 26, 2011]. http://www.biomedcentral.com/1471-2288/11/130.

Bridget Kelly, Taressa Fraze, Robert Hornik. Response rates to a mailed survey of a representative sample of cancer patients randomly drawn from the Pennsylvania Cancer Registry: a randomized trial of incentive and length effects. BMC Medical Research Methodology. 2010;10(1):65. Abstract: "BACKGROUND: In recent years, response rates to telephone surveys have declined. Online surveys may miss many older and poorer adults. Mailed surveys may have promise in securing higher response rates. METHODS: In a pilot study, 1200 breast, prostate and colon patients, randomly selected from the Pennsylvania Cancer Registry, were sent surveys in the mail. Incentive amount ($3 vs. $5) and length of the survey (10 pages vs. 16 pages) were randomly assigned. RESULTS: Overall, there was a high response rate (AAPOR RR4 = 64%). Neither the amount of the incentive, nor the length of the survey affected the response rate significantly. Colon cancer surveys were returned at a significantly lower rate (RR4 = 54%), than breast or prostate surveys (RR4 = 71%, and RR4 = 67%, respectively; p < .001 for both comparisons). There were no significant interactions among cancer type, length of survey and incentive amount in their effects on response likelihood. CONCLUSION: Mailed surveys may provide a suitable alternative option for survey-based research with cancer patients." [Accessed October 25, 2010]. Available at: http://www.biomedcentral.com/1471-2288/10/65.

White Paper on the Shortcomings of How Clinical Trials are Designed, Carried Out and Funded in the U.S. [PDF]. EDICT

(Eliminating Disparities in Clinical Trials). Excerpt: Clinical trials are a critical resource for the discovery of new, life-saving drugs and for developing better prevention and diagnostic screening methods. Today's most effective prevention and treatment modalities are based on previous clinical trial results. But while the need for clinical research is undisputed, how clinical trials are now conducted remains problematic. Increasing research finds major deficiencies in the way clinical trials are designed, carried out and funded in the U.S. with serious implications for the outcomes of medical research studies. Of key significance for the future of scientific innovation is the exclusion or underrepresentation of women, older people, minorities, disabled persons, and rural populations in the vast majority of the research studies conducted in the U.S. Without adequate representation of all patient populations, researchers cannot learn about potential differences among groups and cannot ensure the generalization of results. This website was last verified on 2008-04-11. URL: www.bcm.edu/edict/PDF/EDICT_Project_White_Paper.pdf

Creative Commons License All of the material above this paragraph is licensed under a Creative Commons Attribution 3.0 United States License. This page was written by Steve Simon and was last modified on 2011-01-03. The material below this paragraph links to my old website, StATS. Although I wrote all of the material listed below, my ex-employer, Children's Mercy Hospital, has claimed copyright ownership of this material. The brief excerpts shown here are included under the fair use provisions of U.S. Copyright laws.

2006

4. Stats: Characterizing reasons for refusal (January 17, 2006). A participant in the IRB Forum raised a question about a research study where people were asked to participate, and when some of them said "no" they were then asked why they did not want to participate. The  reasons were tallied across all of the refusals, and percentages were published as part of the full research study. Since the people had already said "no" to participating in the research study, did that also imply that they did not want to share information about their reasons for not participating?

2004

3. Stats: Excluding placebo responders (June 25, 2004). I've always been fascinated by the placebo effect and the ethical issues associated with use of placebos in research. A correspondent in the IRBForum email discussion group asked about the recent efforts of drug companies to identify patients who are likely to show a placebo effect and then exclude them from randomized trials.

2. Stats: Selection bias (August 24, 2004). The CarTalk radio show has an interesting puzzle every week and often these puzzles involve mathematics. These puzzles can sometimes help you understand complex mathematical concepts that are important in Statistics. In the summer, they re-use puzzlers from earlier years, and just last week, they re-used one of my favorites. A "nameless mathematician" during World War II was asked to help with a military problem. A lot of bombers were not returning from their missions, so the Royal Air Force wanted to put armor on the bombers. But where to put it?

1. Stats: So you want to volunteer for a research study? (August 4, 2004). Here's a draft of a speech that I am planning to give on August 5, 2004 for the Bluejacket Toastmasters humorous speech competition. So you want to volunteer for a research study? Good for you! Mister Contestmaster, fellow Toastmasters, and Guests. I work as a statistician at Children's Mercy Hospital. So when you volunteer for a research study, you provide the data that gives me job security.

What now?

Browse other categories at this site

Browse through the most recent entries

Get help

Creative Commons License This work is licensed under a Creative Commons Attribution 3.0 United States License. This page was written by Steve Simon and was last modified on 2011-01-03.