P.Mean: What does the FDA think about Bayesian statistics (created 2008-07-08).
The FDA is, in general, a cautious agency (as it should be), but they are allowing newer approaches for establishing efficacy and safety of new drugs. Many of these new approaches involve Bayesian methods. A draft guidance "Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials - Draft Guidance for Industry and FDA Staff" is available in HTML format or PDF format.
The FDA cites several reasons why Bayesian statistics might be preferred.
When good prior information on clinical use of a device exists, the Bayesian approach may enable FDA to reach the same decision on a device with a smaller-sized or shorter-duration pivotal trial. The Bayesian approach may also be useful in the absence of informative prior information. First, the approach can provide flexible methods for handling interim analyses and other modifications to trials in midcourse (e.g., changes to the sample size or changes in the randomization scheme). Second, the Bayesian approach can be useful in complex modeling situations where a frequentist analysis is difficult to implement or does not exist.
The FDA points out that the Bayesian approach requires extensive pre-planning.
Planning the design, conduct, and analysis of any trial are always important from a regulatory perspective, but they are crucial for the Bayesian approach. This is because decisions are based on:
- prior information
- information obtained from the trial
- the mathematical model used to combine the two.
Different choices of prior information or different choices of model can produce different decisions. As a result, in the regulatory setting, the design of a Bayesian clinical trial involves pre-specification of (and agreement on) both the prior information and the model. This includes clinical agreement on the appropriateness of the prior information and statistical agreement on the mathematical model to be used. Since reaching this agreement is often an iterative process, we recommend you meet with FDA early on to discuss and agree upon the basic aspects of the trial design. A change in the prior information or the model at a later stage of the trial may imperil the scientific validity of the trial results. For this reason, formal agreement meetings may be appropriate when using a Bayesian approach. Specifically, the identification of the prior information may be an appropriate topic of an agreement meeting.
The document offers a nice overview of the likelihood principle:
The likelihood principle is important in all of statistics, but it is especially central to the Bayesian approach. The principle states that all information about the endpoint of interest, x, obtained from a clinical trial, is contained in the likelihood function. In the Bayesian approach, the prior distribution for x is updated using the information provided by the trial through the likelihood function, and nothing else. Bayesian analysts base all inferences about x solely on the posterior distribution produced in this manner.
A trial can be altered in many ways without changing the likelihood function. As long as the modification schemes are pre-specified in the trial design, adherence to the likelihood principle allows for flexibility in conducting Bayesian clinical trials, in particular with respect to:
- modifications to sample size
- interim looks for the purpose of possibly stopping the trial early
- adaptive designs in general.
The FDA will allow an informative prior, but you need to proceed carefully.
Prior knowledge is described by an informative prior distribution. Because using prior information may decrease the sample size in a trial, we recommend you identify as many sources of good prior information as possible when planning a trial. FDA should agree with your choice of prior distributions. Possible sources of prior information include:
- clinical trials conducted overseas
- patient registries
- clinical data on very similar products
- pilot studies.
We recommend the proposed prior information be submitted as part of the IDE (when an IDE is required). In some cases, existing valid prior information may be unavailable for legal or other reasons (e.g., the data may belong to someone else who is unwilling to allow access). We recommend you hold a pre-IDE meeting with FDA to come to agreement on what prior information is scientifically valid and how it will be used in the analysis. Quantitative priors (i.e., those based on data from other studies) are the easiest to evaluate. We recommend the prior studies be similar to the current study in as many as possible of the following aspects:
- target population
- time frame.
Priors based on expert opinion rather than data are problematic. Approval of a device could be delayed or jeopardized if FDA advisory panel members or other clinical evaluators or do not agree with the opinions used to generate the prior.
The document talks about hierarchical models
Bayesian hierarchical modeling is a specific methodology you may use to combine prior results with a current study to obtain estimates of safety and effectiveness parameters. The name hierarchical model derives from the hierarchy in which observations and parameters are structured. The Bayesian analyst refers to this approach as “borrowing strength.” For device trials, strength can be translated into sample size, and the extent of borrowing depends on how closely results from the new study reflect the prior experience.
If results are very similar, the current study can borrow great strength. As current results vary from the previous information, the current study borrows less and less. Very different results borrow no strength at all, or even potentially “borrow negatively”. In a regulatory setting, hierarchical models can be very appealing: They reward having good prior information on device performance by lessening the burden in demonstrating safety and effectiveness. At the same time, the approach protects against over-reliance on previous studies that turn out to be overly optimistic for the pivotal study parameter.
You do need to be cautious about a hierarchical model.
The key clinical question in using hierarchical modeling to borrow strength from previous studies is whether the previous studies are sufficiently similar to the current study in covariates such as:
- devices used
- patient population
- physician training
- patient management
- time frame.
Statistical adjustments for certain differences in covariates such as demographic and prognostic variables may be appropriate, using patient-level data. Generally, proper calibration of your study depends on using the same covariate information at the patient level as in previous studies.
Calibration based only on covariate summaries (such as from the literature) may be inadequate because the relationship of the covariate level to the outcome can be determined in the current study but not in the previous studies. This forces the untestable assumption that covariate effects in your study and previous studies are the same; that is, that study and covariate effects do not interact.
When you use more than one study as prior information in a hierarchical model, the prior distribution can be very informative. As discussed previously, if the prior probability of a successful trial is too high, we recommend the study design and analysis plan be modified.
Finally, the document encourages the Bayesian approach for post-marketing surveillance.
FDA believes the Bayesian approach is well suited for surveillance purposes. The key concept: “Today’s posterior is tomorrow’s prior” allows you to use the posterior distribution from a pre-market study to serve as a prior distribution for surveillance purposes, to the extent that data from the clinical study reflect how the device is used after approval. In other words, you may readily update information provided by a pre-market clinical trial with post-market data via Bayes’ theorem if you can justify exchangeability between pre- and post-market data. You may continue to update post-market information via Bayes’ theorem as more data are gathered. You may also use Bayesian models to mine large databases of post-market medical reports.
This is a draft guidance, and it is unclear from this document when a final guidance document will be available.
This work is licensed under a Creative Commons Attribution 3.0 United States License. This page was written by Steve Simon and was last modified on 2010-04-01. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Bayesian statistics.