FDA, an agency within the Department of Health and Human Services (HHS), is responsible for overseeing the safety and effectiveness of drugs marketed within the United States. These responsibilities begin before a product is brought to the market, and include reviewing drug sponsors’ proposals for conducting clinical trials, providing advice and publishing guidance regarding these trials, as well as reviewing applications for new drugs. Once a drug sponsor identifies a promising chemical compound it believes to be capable of curing or treating diseases, the sponsor may decide to conduct clinical trials on humans to gather the evidence necessary to demonstrate to FDA that the drug is safe and effective for its intended use. Before beginning clinical trials in the United States, a sponsor generally must submit an investigational new drug application to FDA for review. This application provides FDA with extensive information about the drug, including safety and manufacturing information, and outlines the sponsor’s plans for clinical trials, which gradually introduce new drugs to increasingly larger numbers of patients. FDA assesses the information in the application—which is later included as part of the NDA—to ensure that the drug is reasonably safe to begin studying in humans. Sponsors may use these clinical trials to gather evidence of a drug’s safety and effectiveness. In general, FDA requires sponsors to submit the results of more than one clinical trial demonstrating effectiveness in order to provide substantial evidence that a drug is effective for the intended indication and population. FDA has issued regulations and guidance that provide industry with information to properly design, conduct, and interpret these trials. For example, in 1985 FDA substantially revised its regulations including the provision addressing the characteristics of adequate and well-controlled trials and the types of controlled trials that can be used to gather evidence of a new drug’s effectiveness. Sponsors may use trials of varying designs to obtain evidence of a drug’s effectiveness. One type of clinical trial is a non-inferiority trial. The objective of a non-inferiority trial is to show that any difference in the effectiveness of two drugs is small enough to allow a conclusion that the new drug is also effective, but not substantially less effective than the active control. To conduct a non-inferiority trial, sponsors must make many decisions regarding how the trial will measure the new drug’s effectiveness. For example, they must select the trial’s primary endpoint, the principal measure used to determine a drug’s effectiveness. The primary endpoint may be a clinical endpoint—a direct measure of how a patient feels, functions, or survives—or, in some cases, a surrogate endpoint—a laboratory measure or physical sign used as a substitute for a clinical endpoint that reasonably predicts a clinical benefit. Sponsors must also determine when to measure the trial’s endpoint—for example, are patients cured within 7, 14, or 30 days after starting treatment—in addition to determining the number and type of patients to be enrolled in each trial. Sponsors conducting non-inferiority trials must also make decisions to account for the new drug’s comparison to the active control. Sponsors must identify an available treatment for use as an active control in the non- inferiority trial. They must then use evidence of the active control’s effectiveness as shown in prior clinical trials to estimate the effect that the active control will have in the planned non-inferiority trial, adjusting for any differences between the prior and planned trials. Using this estimate, sponsors determine the trial’s non-inferiority margin—the maximum clinically acceptable extent to which the new drug can be less effective than the active control and still show evidence of an effect. FDA considers the selection of a margin to be the single greatest challenge in designing, conducting, and interpreting non-inferiority trials. Its calculation is not only dependent on a string of other decisions related to the trial—for example, the data collected on the active control’s effectiveness in other trials—but also includes the application of clinical judgment to determine the maximum amount of effectiveness that could be lost without having a substantial impact on the drug’s effectiveness. If a non-inferiority margin is incorrectly calculated and is set too large, a drug that is not effective may appear to be effective; if the margin is too small, an effective drug may appear to be ineffective. In a non-inferiority trial, patients are randomly assigned to receive either the new drug or an active control. After the trial, the sponsor identifies the observed effect of each drug in the trial, and calculates the observed difference in the drugs’ effectiveness. The actual difference in the drugs’ effectiveness in the entire population could be greater or less than what is observed in the trial. For that reason, sponsors calculate a confidence interval around the observed difference in effectiveness between the new drug and active control drug. The confidence interval provides a range of values for the difference in effectiveness within which the true difference is likely to be found. The confidence interval around the observed difference in effectiveness is used to determine if the new drug is non-inferior to the active control. It is compared to the non-inferiority margin—the maximum clinically acceptable extent to which the new drug can be less effective than the active control. If the confidence interval is within the non-inferiority margin, and the sponsor provides adequate evidence that the active control demonstrated its expected effect in the trial, the new drug may be deemed non-inferior to the active control. A new drug can be non-inferior to an active control even if the estimated difference in effectiveness and its confidence interval lies entirely below zero, meaning that the active control drug is more effective than the new drug, but by an irrelevant amount. However, if the confidence interval shows that the effect of the drug could be below the margin—even if the observed effect of the drug was within the margin—the drug would not have shown an effect, and is therefore considered inferior. In addition, if the confidence interval lies entirely above zero—demonstrating that the new drug is more effective than the active control—the drug can be considered superior. (See fig. 1.) Since issuing regulations that address the elements of adequate and well- controlled trials, FDA has also periodically issued guidance documents to provide updates on the agency’s current thinking on a range of topics. These guidance documents encompass broad issues such as statistical principles for use in clinical trials and how to select an appropriate control, whereas others are more focused and serve to consolidate relevant recommendations on the development of drugs treating a particular indication. In addition to disseminating guidance on non-inferiority trials, FDA provides specific advice regarding the design of clinical trials at the request of sponsors. For example, sponsors may ask FDA to review and provide advice on a trial’s proposed active control, non-inferiority margin, or endpoint before the given trial has begun. After the conclusion of their clinical trials, sponsors may consult with FDA regarding the interpretation of trial results or to discuss the information the agency would expect to see submitted in an NDA. FDA’s advice and recommendations to sponsors are considered advisory; sponsors are not required to implement any of the agency’s suggestions. If sponsors believe they have successfully demonstrated a new drug’s safety and effectiveness, they may submit an NDA to FDA for review. The NDA contains information about the safety and effectiveness of the drug as demonstrated in clinical trials and other research, such as studies in animals. Once the agency receives an NDA, the application is reviewed by one of FDA’s medical review divisions, depending on the indication the drug has been proposed to treat. If FDA determines that the drug is safe and effective for its intended use—that its clinical benefits outweigh its potential health risks—and that other requirements are met, it will approve the application. After approving a new drug, FDA’s responsibilities continue as it is charged with monitoring the safety, effectiveness, and promotion of approved drugs. FDA executes these responsibilities in the same manner regardless of whether drugs were approved on the basis of evidence from non-inferiority trials. Non-inferiority trials present unique issues in measuring the effectiveness of new drugs. For example, the use of these trials can raise uncertainties about the true effectiveness of new drugs because non-inferiority trials cannot measure this directly. Instead, these trials measure the effectiveness of the new drug relative to the active control, and sponsors must assess whether the active control can be considered to be as effective in the non-inferiority trial as was expected based on past experience. Using data from the non-inferiority trial and from prior trials measuring the effectiveness of the active control, the effectiveness of the new drug is estimated—but not ever fully known. In addition, non- inferiority trials are more prone to certain biases than superiority trials. For example, if patients in a superiority trial do not take the new drug as directed, this poor compliance will dilute the measured effectiveness of the new drug, making it less likely that the trial will successfully demonstrate superiority. In a non-inferiority trial, however, poor compliance by patients taking the active control drug can have a different effect. It can reduce the difference in the measured effectiveness between the new drug and the active control, making the treatments appear more similar than they might otherwise be. As such, poor compliance in a non- inferiority trial can increase the likelihood that an ineffective drug is concluded to be effective. The use of non-inferiority trials over time also raises concerns about the potential for “biocreep” to occur. This term is used to describe the concern that successive generations of drugs approved based on non-inferiority trials, with the active control changing in each new generation, could lead to the adoption of decreasingly effective drugs and ultimately to the approval of drugs that are no more effective than a placebo. Non- inferiority trials that are poorly designed are especially prone to biocreep. The selection of inappropriate active controls—that is, drugs that are not known to be consistently effective, or drugs that were themselves approved on the basis of non-inferiority trials—could lead to biocreep. Even if successive generations of non-inferiority trials are conducted and each trial is itself well-designed, biocreep may still occur because placebo controls are not included in these trials. Non-inferiority trials are only able to measure the effectiveness of the new drug relative to the active control, not a placebo. As a result, the true effectiveness of any of the new drugs, compared to a placebo, is not measured. Without this metric, it is impossible to determine the extent to which the effectiveness of the new drug is similar to that of a placebo and whether biocreep has occurred. FDA has acknowledged some concerns over the uncertainties inherent in non-inferiority trials and the potential these trials create for biocreep. For example, FDA stated in a 1992 guidance document that, in order to avoid biocreep, sponsors should consult with the agency regarding the active controls they were considering for their trials. In other guidance documents, FDA has also encouraged sponsors to consult with the agency regarding their planned non-inferiority trials. One-quarter of NDAs submitted to FDA for review from fiscal years 2002 through 2009 included evidence from non-inferiority trials, and many of these applications were for antimicrobial drugs. FDA approved a majority of the applications that included evidence from these trials. Forty-three, or one-quarter, of the 175 NDAs we reviewed that were submitted to FDA from fiscal years 2002 through 2009 included evidence from at least one non-inferiority trial. The number of NDAs with evidence from non-inferiority trials varied from year to year and generally declined from fiscal years 2002 through 2009. On average, FDA received five NDAs each year that included evidence from non-inferiority trials. (See fig. 2.) About half of the 43 NDAs submitted with evidence from at least one non- inferiority trial—or 22—were for antimicrobial drugs, such as those that treat bacterial, viral, or fungal infections. The remaining portion of NDAs submitted with evidence from these trials represented a variety of drug types. (See table 1.) FDA approved 29 of the 43 NDAs submitted for review from fiscal years 2002 through 2009 that included evidence from at least one non-inferiority trial. Most NDAs—18 of the 29—were approved based on evidence from pivotal non-inferiority trials. FDA approved the remaining 11 applications based on other evidence, such as the superiority of the new drug compared to a placebo or an active control. As of December 31, 2009, FDA had decided not to approve 14 applications that included evidence from dence from non-inferiority trials. (See fig. 3.) non-inferiority trials. (See fig. 3.) Many NDAs including evidence from non-inferiority trials were for antimicrobial drugs, and the majority of approvals based on this evidence were also for these types of drugs. Two-thirds, or 12 of the 18, NDAs approved on the basis of non-inferiority trials were for antimicrobial drugs. The remaining one-third of NDAs approved on the basis of non- inferiority trials were for various other types of drugs, including those treating diabetes and chemotherapy-induced nausea and vomiting. See appendix I for a list of all 18 NDAs approved based on evidence from non- inferiority trials, including fiscal year of approval, drug type, and approved indication. Characteristics varied among the non-inferiority trials providing primary evidence to support FDA’s approval of 18 NDAs. Some other applications also included non-inferiority trials that FDA identified as being poorly designed; these trials did not provide primary evidence for approval. Characteristics varied among the non-inferiority trials that provided primary evidence for the approval of the 18 NDAs. FDA relied on primary evidence from multiple pivotal non-inferiority trials to support the approval of most of these applications. The number of pivotal non- inferiority trials used as primary evidence for these 18 NDAs ranged from one to four, with an average of two pivotal non-inferiority trials supporting the approval of each application. In addition to including evidence from pivotal non-inferiority trials, five applications included evidence from other types of pivotal trials; for example, trials demonstrating superiority to a placebo or active control drug. Thirteen of the 18 applications included only pivotal non-inferiority trials in their applications. Of these applications, FDA approved four based on evidence from a single pivotal non-inferiority trial. Two-thirds, or 12, of the 18 NDAs included trials that measured drug effectiveness using a surrogate, rather than a clinical, primary endpoint in at least one of their pivotal trials. Although FDA generally prefers that drug sponsors demonstrate the effectiveness of a new drug by showing its impact on a clinical endpoint, in certain cases, it will consider a surrogate endpoint if it determines it is a reasonable substitute. However, all experts we interviewed who commented on this topic noted that the approval of drugs on the basis of both non-inferiority trials and surrogate endpoints increases uncertainty in the drugs’ true effectiveness. Half of the 18 NDAs FDA approved on the basis of non-inferiority trials tested the effectiveness of the new drug against more than one active control. A majority of the active controls used in non-inferiority trials were FDA-approved for the indication. However, three applications included evidence from non-inferiority trials that used one active control that was not FDA-approved for the indication. For example, in fiscal year 2003, FDA approved Cubicin for the treatment of complicated skin and skin structure infections on the basis of evidence from two pivotal non- inferiority trials that used a total of five different active control drugs. While three of these active control drugs were FDA-approved to treat this indication, two were not. In addition, some of the active controls used in non-inferiority trials were themselves approved on the basis of evidence from other trials that compared the drug to another active control. However, FDA reviewed the selection of nearly all of the active controls used in the pivotal non-inferiority trials that supported the approval of the 18 NDAs, and found the active controls appropriate for use in these trials. FDA officials also told us that if a new drug was approved on the basis of evidence from non-inferiority trials, the active control used in these trials would most likely also be used in subsequent trials, except in cases where the newer drug proved to be superior to the active control. The margins used for most of the 18 NDAs approved on the basis of evidence from non-inferiority trials ranged from 5 to 20 percent, with the most commonly used margin being 10 percent. That is, for trials using a 10 percent non-inferiority margin, the new drug could be estimated to be up to 10 percent less effective than the active control. However, the observed difference in the effectiveness of the new drug and active control, as measured in the clinical trials, would be less than 10 percent. At the time of its review of the NDAs, FDA agreed with the non-inferiority margins set for all of the pivotal trials submitted for the majority of drugs approved on the basis of evidence from non-inferiority trials. All of the pivotal trials submitted for these drugs—that is, those where FDA agreed with the margin—demonstrated that the new drug was non-inferior to the active control drug as measured on the primary endpoint, with one exception. These trials showed that the confidence interval for the difference in the drugs’ effectiveness was within the non-inferiority margin. FDA did not agree with the non-inferiority margins set for pivotal trials submitted with three applications, though the agency approved these drugs based on evidence from these trials. For two drugs, Exjade and Reyataz, FDA stated that the proposed margins could not be used to measure the drugs’ effectiveness. FDA conducted additional analyses of data from pivotal trials submitted in these drugs’ applications which showed that the drugs were superior to a placebo. For the third drug, Noxafil, FDA did not agree with the sponsor’s proposed justification of the margin for one trial, although this trial showed the difference in the drugs’ effectiveness to be less than the disputed margin. FDA approved Exjade in fiscal year 2006 to treat chronic iron overload in certain patients receiving blood transfusions. Exjade’s NDA included evidence from one pivotal non-inferiority trial that had an objective of showing that Exjade lowered iron levels to a similar extent as the active control. Upon reviewing the application, FDA disagreed with the non- inferiority margin proposed for this trial. FDA analyzed data from the trial which showed that Exjade was effective in lowering patients’ iron levels despite ongoing blood transfusions (which typically result in increased iron levels), particularly among those patients who began the trial with very high iron levels. FDA approved Exjade on the basis of this evidence, which showed that the drug would have been more effective than a placebo. In addition, FDA officials noted that Exjade presented a valuable alternative in the treatment of this indication. FDA approved Reyataz in fiscal year 2003 for the treatment of human immunodeficiency virus (HIV) infection. Reyataz’s NDA included evidence from two pivotal non-inferiority trials, including one in patients that were naïve to HIV treatment and one in patients that had experience receiving HIV treatment. FDA agreed with the margin proposed for the trial conducted in the treatment-naïve population, which was successful in demonstrating that Reyataz was non-inferior to its active control. However, FDA disagreed with the margin proposed for the trial conducted in the treatment-experienced population. Agency officials analyzed data from this trial which showed that Reyataz was effective in treatment- experienced patients, and this effect was greater than what would have been expected with a placebo. FDA approved Reyataz to treat HIV infection on the basis of this evidence, as well as other pivotal evidence of effectiveness in the treatment-naïve population. In addition, FDA officials noted that Reyataz presented an alternative to HIV-infected patients that were not responding to available HIV treatments. FDA approved Noxafil in fiscal year 2006 for the prevention of invasive Aspergillus and Candida infections in certain patients on the basis of evidence from two pivotal trials. In its review of this NDA, FDA noted that the sponsor had not adequately explained the relevance of the proposed 15 percent non-inferiority margin. One of these trials demonstrated that Noxafil was superior to its active control, and the other trial demonstrated that the drug was at most three percent less effective than the active control. FDA approved this drug on the basis of this evidence of effectiveness. Table 2 provides a summary of the characteristics of non-inferiority trials for the 18 NDAs we identified as approved on the basis of evidence from non-inferiority trials. We found that FDA reviewed the characteristics of the non-inferiority trials supporting the approval of the 18 NDAs to ensure that the drugs it approved were more effective than a placebo. FDA’s review therefore minimized the potential for biocreep. Similarly, our examination of the trials’ characteristics also revealed no evidence of biocreep. While non-inferiority trials provided primary evidence of effectiveness to support the approval of 18 NDAs, other non-inferiority trials were poorly designed and did not provide such evidence. Of the other 25 NDAs that included evidence from non-inferiority trials, FDA identified 9 applications that included poorly designed non-inferiority trials. These trials were unable to accurately measure the new drugs’ effectiveness and did not provide primary evidence for the approval of these drugs. Some of the concerns FDA identified with sponsors’ non-inferiority trials were inappropriate use of non-inferiority trials for the indication being treated, inappropriate selection of an active control, including cases where the drug was not FDA-approved or the sponsor did not provide an adequate justification, and improper calculation or justification of the non-inferiority margin. FDA informed sponsors of its concerns with all of these applications’ non- inferiority trials prior to the sponsors’ submission of the NDAs. Specifically, FDA notified the sponsors between 1 month and 94 months before submission, with an average of about 30 months prior to submission. With the exception of one application, FDA notified all sponsors at least 6 months prior to submission. For example, FDA advised one sponsor before the sponsor began its non- inferiority trials—24 months prior to submitting its NDA—that the agency did not consider it appropriate to use non-inferiority trials to support the approval of the drug for the indication being sought—treatment of schizophrenia. FDA reiterated this position on another occasion prior to the NDA submission. FDA did not consider the results of this trial to provide primary evidence to support its approval decision. The agency ultimately approved the drug based on evidence that the drug was superior to placebo as demonstrated in several other trials. In another case, a sponsor conducted the non-inferiority trial outside of the U.S. and had not requested FDA’s input while planning or conducting the trial. The sponsor requested a meeting with FDA to discuss its planned NDA. During this meeting, which occurred 1 month before FDA received the NDA, the agency learned of the sponsor’s non-inferiority trial and communicated its concerns regarding the design of the trial. FDA did not consider the results of this non-inferiority trial in its approval decision, but ultimately approved the drug based on evidence of superiority to placebo as demonstrated in another trial. In March 2010, FDA issued draft guidance on non-inferiority trials that provides detailed recommendations on using these trials to provide evidence of a new drug’s effectiveness. This March 2010 draft guidance offers broader and more comprehensive information to supplement other indication-specific guidance documents the agency previously issued. In March 2010, FDA issued new draft guidance on non-inferiority trials that provides detailed recommendations on how these trials may be used to establish the effectiveness of new drugs. Although FDA had previously issued guidance documents that included information regarding the use of non-inferiority trials for certain indications, this March 2010 guidance is the first focused solely on the use of non-inferiority trials. It explains the key principles involved in using a non-inferiority trial to demonstrate the effectiveness of a drug and provides detailed recommendations for such trials, including how to select an active control and how to set the non- inferiority margin (that is, determining the maximum clinically acceptable extent to which the new drug can be less effective than the active control), among other things. The March 2010 guidance also explains why the agency considers its recommendations appropriate, offers answers to frequently asked questions, and lists detailed examples to illustrate some common challenges in designing and interpreting non-inferiority trials. FDA officials told us that they developed the March 2010 guidance on non- inferiority trials because it was clear to them that these trials were not well understood. The concepts elaborated on in the March 2010 guidance are not new, however. They have been part of FDA’s considerations since at least 1985 when the agency substantially revised NDA regulations to include a provision describing the characteristics of adequate and well- controlled trials. These concepts have also been addressed, in part, in other agency guidance documents. However, FDA officials saw the need for more detailed guidance as they noticed many errors, especially related to the selection of a non-inferiority margin, in sponsors’ execution of these trials. FDA officials also expect that the use of non-inferiority trials will rise as more drugs become available to prevent death or serious illness and the use of placebos may become unethical. FDA’s March 2010 guidance explains when non-inferiority trials may be used to establish a drug’s effectiveness. The guidance states that these trials are generally used when an available treatment is known to provide an important benefit—for example, the prevention of death or irreversible harm. In these cases, it would be considered unethical to use a placebo in a clinical trial. The guidance also states that non-inferiority trials may only be used when they are capable of measuring the effect of the new drug in the study—that is, when the active control is able to consistently demonstrate its expected effect in the non-inferiority trial. FDA’s March 2010 guidance explains that non-inferiority trials may not be able to demonstrate the effectiveness of drugs treating certain indications because not all drugs have a consistent effect in treating these indications. The guidance also offers suggestions for other types of trials that may be useful in demonstrating a drug’s effectiveness in cases where a non-inferiority trial is unable to provide evidence of effectiveness. The March 2010 guidance provides detailed recommendations on how to select an active control. For example, when more than one potential active control exists, the guidance recommends that the most effective drug be chosen as the active control. In addition, the frequently asked questions section also clarifies that the active control does not need to be FDA- approved for the indication. However, FDA officials we interviewed stated that active controls used in non-inferiority trials are usually FDA- approved. If the active control is not FDA-approved, FDA asks sponsors to provide evidence of the active control’s effectiveness. FDA’s March 2010 guidance also offers detailed advice on a range of other topics related to the use and interpretation of non-inferiority trials. For example, it suggests two methodologies that can be used to set the margin, offers step-by-step instructions on how to use each of these approaches, and addresses the role of clinical judgment in determining the margin. It also explains how to adjust the margin to account for some of the uncertainties related to non-inferiority trials, such as differences between the planned non-inferiority trial and prior trials that measured the effectiveness of the active control. The guidance offers advice on how to determine the proper number and type of patients to enroll in the trial, and how to select an endpoint. For example, the guidance states that the endpoint should be “one for which there is a good basis for knowing the effect of the active control.” Most of the experts we interviewed who reviewed FDA’s March 2010 guidance told us that they thought the recommendations it included were clear and detailed, and addressed the key principles involved in conducting non-inferiority trials. Some experts noted that the guidance’s frequently asked questions and examples were useful in illustrating the key principles described in the document, and said that FDA’s recommendations would help sponsors appropriately use these trials to prove a drug’s effectiveness. While experts we interviewed who reviewed FDA’s March 2010 guidance noted that it addressed key principles, most identified additional technical issues that they would have liked this guidance to have addressed. For example, the March 2010 guidance does not address how the use of a surrogate endpoint impacts the design and interpretation of a non- inferiority trial. FDA officials told us that the guidance applies to non- inferiority trials that use surrogate endpoints. However, some experts we interviewed noted that such trials are difficult to design and interpret; therefore, additional guidance on this topic may be helpful. Since the non- inferiority margin represents the maximum clinically acceptable extent to which the new drug can be less effective than the active control, experts told us that sponsors would need to translate the drug’s effect on a surrogate endpoint into its expected effect on a clinical endpoint in order to calculate the non-inferiority margin and interpret the trials’ results. Some experts also noted that the guidance does not include enough detailed instructions on how to estimate the effect of the active control in the non-inferiority trial. Finally, some experts who reviewed FDA’s March 2010 guidance told us that they wished the guidance more emphatically stated that non-inferiority trials should only be used as a last resort when seeking drug approval. FDA’s March 2010 draft guidance provides broader and more comprehensive information about the use of non-inferiority trials, supplementing other indication-specific guidance documents the agency had already issued. The objective and content of these two types of guidance documents differ. The March 2010 guidance offers comprehensive information on one topic, non-inferiority trials, that may be generally applied for all drugs using these trials. In contrast, FDA’s indication-specific guidance documents present recommendations on many topics—including trial design—for consideration in developing drugs to treat a particular indication or set of indications. Some of these indication-specific documents provide recommendations on how to use non-inferiority trials for that particular indication; for example, by suggesting a specific margin or a specific endpoint. However, unlike FDA’s March 2010 guidance, not all indication-specific guidance documents include information on all of the key principles involved in using a non- inferiority trial to establish a drug’s effectiveness. In addition, these indication-specific guidance documents do not include the same level of detail on the key principles that is in the March 2010 guidance. For example, several of FDA’s indication-specific guidance documents state that sponsors should justify their selection of non-inferiority margins in their NDAs. However, in these documents FDA does not elaborate on the methods sponsors could use to select or justify the margins. In contrast, the March 2010 non-inferiority guidance provides detailed instructions on how to calculate the margin. FDA’s indication-specific guidance documents provide sponsors with additional clarity on when non-inferiority trials may be used to establish the effectiveness of drugs treating a particular indication. From January 2002 through June 2010, FDA issued 17 guidance documents that state the agency’s position regarding the use of non-inferiority trials in demonstrating the effectiveness of drugs treating certain indications. In these indication-specific guidance documents, FDA stated that non- inferiority trials may be able to demonstrate the effectiveness of drugs treating eight indications, including those for HIV, cancer, diabetes mellitus, and certain severe infections. During the same period, FDA also issued nine indication-specific guidance documents which state that non- inferiority trials may not be able to demonstrate the effectiveness of drugs treating other indications—including some less severe infections such as sinusitis and acute bacterial otitis media (ear infections)—because the agency has been unable to identify available drugs that have a consistent effect and could serve as active controls in non-inferiority trials. (See table 3.) Appendix II identifies the guidance documents FDA has issued with information on the use of non-inferiority trials from January 2002 through June 2010, including indication-specific documents as well as the March 2010 draft guidance on non-inferiority trials. Our review of FDA’s indication-specific guidance showed that the agency has become more conservative in allowing evidence from non-inferiority trials to demonstrate the effectiveness of new drugs. First, FDA has revised its view regarding when non-inferiority trials may be used. Prior to 2007, for example, FDA had approved drugs treating several less severe infections—including acute bacterial sinusitis, acute bacterial otitis media, and acute bacterial exacerbations of chronic bronchitis—on the basis of evidence from non-inferiority trials. Experts we interviewed noted that these infections can often be resolved without treatment—and thus it is difficult to estimate the effect that an active control drug would have in a non-inferiority trial. In 2007 and 2008, FDA issued several guidance documents stating that non-inferiority trials may not be able to demonstrate the effectiveness of drugs treating these indications. Second, FDA has become more rigorous in its review of evidence from non- inferiority trials. For example, prior to 2001, FDA’s guidance on the development of anti-infective drugs had not advised sponsors to scientifically calculate or justify their selected non-inferiority margins—a step that FDA’s March 2010 guidance recommends. We provided a draft of this report to HHS for review. We received technical comments from HHS, which we incorporated as appropriate. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Commissioner of FDA and appropriate congressional committees. The report also will be available at no charge on the GAO Web site at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or crossem@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. The Food and Drug Administration (FDA) approved 18 new drug applications (NDA) that were submitted from fiscal year 2002 through fiscal year 2009 on the basis of evidence from non-inferiority trials. The majority of these were antimicrobial drugs, such as those that treat bacterial, viral, and fungal infections. (See table 4.) From January 2002 through June 2010, FDA issued 17 indication-specific guidance documents that included information about non-inferiority trials, and one guidance document that included broad recommendations regarding the use of non-inferiority trials. In addition to the contact named above, Geri Redican-Bigott, Assistant Director; Kathleen Diamond; Carolyn Garvey; Cathy Hamann; Julian Klazkin; Kaitlin McConnell; and Patricia Roy made key contributions to this report.

Summary: Before approving a new drug, the Food and Drug Administration (FDA)--an agency of the Department of Health and Human Services (HHS)--assesses a drug's effectiveness. To do so, it examines information contained in a new drug application (NDA), including data from clinical trials in humans. Several types of trials may be used to gather this evidence. For example, superiority trials may show that a new drug is more effective than an active control--a drug known to be effective. Non-inferiority trials aim to demonstrate that the difference between the effectiveness of a new drug and an active control is small--small enough to show that the new drug is also effective. Drugs approved on this basis may provide important benefits, such as improved safety. Because non-inferiority trials are difficult to design and interpret, they have received attention within the research community and FDA. FDA has issued guidance on these trials. GAO was asked to examine FDA's use of non-inferiority trial evidence. This report (1) identifies NDAs for new molecular entities--potentially innovative new drugs not FDA-approved in any form--that included evidence from non-inferiority trials, (2) examines the characteristics of these trials, and (3) describes FDA's guidance on these trials. GAO reviewed NDAs submitted to FDA between fiscal year 2002 (the first full year that FDA documentation was available electronically) and fiscal year 2009 (the last full year of submissions), examined FDA's guidance, and interviewed agency officials. Evidence from non-inferiority trials was included in about one-quarter, or 43, of the 175 NDAs for new molecular entities that were submitted to FDA for review from fiscal years 2002 through 2009. Many of these applications were for antimicrobial drugs, such as those treating bacterial, viral, and fungal infections. As of December 31, 2009, FDA approved 18 of the 43 NDAs on the basis of evidence from non-inferiority trials. Of the remaining 25 NDAs, FDA approved 11 based on other evidence, such as proof that the new drug was more effective than a placebo (no treatment), and decided not to approve 14. The non-inferiority trials included in these NDAs varied with respect to their characteristics. FDA generally requires sponsors to provide evidence of a drug's effectiveness as shown in more than one trial. For the 18 NDAs that were approved based on evidence from non-inferiority trials, the number of non-inferiority trials used to provide primary support for approval ranged from one to four, with an average of 2 such trials per NDA. Half of these applications included non-inferiority trials that tested the effectiveness of the new drug against more than one active control. The non-inferiority margins--the maximum clinically acceptable extent to which the new drug can be less effective than the active control and still show evidence of an effect--ranged from 5 to 20 percent among trials that supported approval. Among the other 25, FDA identified nine NDAs that included poorly designed non-inferiority trials which did not provide primary evidence for approval. Some of these problems included an inappropriate selection of an active control and an improper calculation of a non-inferiority margin. FDA notified sponsors of its concerns with the poorly designed trials prior to the sponsors' submissions of all NDAs that included such trials. In March 2010 FDA issued draft guidance which focused solely on the use of non-inferiority trials. This guidance presents detailed and comprehensive recommendations on how non-inferiority trials may be used to provide evidence of a drug's effectiveness. For example, it provides advice on how to select an active control and how to set the non-inferiority margin, as well as how to interpret the trials. This guidance offers broad, generally applicable recommendations to supplement indication-specific guidance documents that FDA had previously issued. These indication-specific guidance documents include FDA's advice on many issues related to the development of drugs for particular indications, some of which are related to the use of non-inferiority trials. GAO's review of FDA's guidance showed that the agency has become more conservative in allowing evidence from non-inferiority trials to demonstrate a drug's effectiveness. First, FDA has limited the indications for which these trials may be used. Second, the agency has also become more rigorous in its review of evidence from non-inferiority trials. We sent a draft of this report to HHS for review. HHS provided us with technical comments, which we incorporated as appropriate.