Women's Health: Women Sufficiently Represented in New Drug Testing, but FDA Oversight Needs Improvement
Published: 2001-07-06T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: This report reviews the Food and Drug Administration's (FDA) inclusion of women in clinical drug trials.
section_title: What GAO Found
paragraphs: GAO found that women were a majority of the clinical trial participants in the new drug applications (NDA) it examined and that every NDA included enough women in the pivotal studies to be able to statistically demonstrate that the drug is effective in women. Although these findings are welcome, GAO also found three areas of concern. The first is the relatively small proportion of women in early small-scale safety studies. These early studies provide important information on drugs' toxicity and safe dosing levels for later stages of clinical development, and many of the NDAs GAO examined found significant sex differences in a drug's pharmacokinetics, or how it is absorbed, distributed, metabolized, excreted, and concentrated in the bloodstream. Second, GAO is not confident that either NDA sponsors or FDA's reviewers took full advantage of the available data to learn more about the effects of the drug in women and to explore potential sex differences in dosing. This is because NDA summary documents are not required to include analyses of sex differences, and many of them do not. Third, FDA lacks appropriate management systems to monitor how many women are in clinical trials, to be certain that NDAs and investigational new drug applications (IND) annual reports comply with regulations for presenting outcome data by sex and tabulating the number of women included in ongoing trials, and to confirm that its medical officers have adequately addressed sex-related issues in their reviews. Although FDA has taken some promising initial steps to address these deficiencies, it is important that the agency finalize the pilot programs it has underway and give sustained attention to these management issues.

section_title: Letter
paragraphs: Men and women sometimes respond differently to the same drug. For example, we recently reported that four of the ten prescription drugs withdrawn from the U.S. market in recent years induced potentially fatal cardiac arrhythmias in women more often than in men. Because of potential sex differences in the safety and efficacy of new drugs, it is important to include women and men in all stages of drug development and to analyze the resulting data for sex differences. In 1992, we reported that the Food and Drug Administration (FDA) was not adequately ensuring the representation of women or the study of sex differences in clinical drug trials conducted by the pharmaceutical industry. Although FDA subsequently has taken some steps to increase the participation of women in clinical drug trials, concerns remain that women continue to be underrepresented and that sex differences in responses to drugs continue to go unexamined during drug development.
You asked us to investigate FDA’s progress in addressing the inclusion of women in clinical drug trials since our 1992 report. In response to your request, our work addressed: (1) what FDA guidance documents and regulations govern the inclusion of women in clinical drug trials; (2) are the regulations being followed; (3) are appropriate numbers of women included in the clinical drug trials to ensure the safety and efficacy of drugs for women; and (4) how does FDA oversee the collection, presentation, and analysis of data related to sex differences?
To address these questions, we reviewed new drug applications (NDA) for new molecular entities (novel drugs subject to FDA review for the first time), submitted to FDA from August 10, 1998 through December 31, 2000.Out of 82 original NDAs for new molecular entities (NME) submitted during that period, we reviewed all 36 of the NDAs that met our selection criteria, namely that FDA had approved them or categorized them as approvable by December 31, 2000, and had labeled them for use in both men and women. We excluded NDAs for biologic products, such as vaccines, diagnostic drugs used in medical imaging, drugs for sex-specific conditions, and pediatric drugs. For each NDA, we analyzed three critical summary documents submitted by the drug’s sponsor—the Integrated Summary of Safety, the Integrated Summary of Efficacy, and the Pharmacokinetics and Bioavailability Summary—and the FDA Medical Officer Review. These documents were chosen because they summarize clinical trial data and because pertinent regulations direct NDA sponsors to include relevant information in the safety and efficacy summary documents submitted to FDA. We also randomly sampled 100 annual reports for investigational new drugs. These are drugs in development for which drug manufacturers typically have not yet sought FDA’s approval for marketing. In addition, we interviewed FDA officials, pharmacology and drug safety experts, and representatives of the pharmaceutical industry. We also reviewed relevant literature. (For additional information on our methodology, see appendix I.) We conducted our work from July 2000 through May 2001 in accordance with generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: FDA is responsible for helping to ensure the safety and efficacy of drugs marketed in the United States. It does this by overseeing the drug development process, reviewing applications for the marketing of new drugs, and monitoring the safety and efficacy of drugs once they are marketed. A growing body of literature has demonstrated that in responses to some drugs there are medically important sex differences that require the participation of women in clinical trials for new drugs. In the 1970s, FDA recommended the exclusion of women of childbearing potential from early clinical drug trials because of concerns for the health of the women and of their potential offspring.
subsections: section_title: The Role of FDA
paragraphs: FDA, an agency in the Department of Health and Human Services, is charged with helping to ensure that safe and effective food, drugs, medical devices, and cosmetics reach the United States market. FDA assists drug manufacturers in designing clinical drug trials, reviews proposals for conducting clinical drug trials, and approves drugs for sale in the United States based on its determination that the clinical benefits of a drug outweigh its potential health risks. FDA also approves drug labeling, which indicates the medical conditions and patient populations for which the drug has been tested and approved as safe and effective. Once a drug reaches the market, FDA continues to monitor its safety and efficacy.
section_title: The Drug Development and Approval Process
paragraphs: Before any new drug can be tested on humans, a drug’s sponsor must submit an investigational new drug (IND) application to FDA that summarizes the investigations conducted prior to trials in humans, lays out a plan for how the drug will be tested in humans, and provides assurances that appropriate measures will be taken to protect study participants. Specifically, the IND application demonstrates that the drug is reasonably safe for subsequent testing in humans based on laboratory and animal testing and exhibits enough potential effectiveness to justify its commercial development. Unless FDA determines that a proposed study is unsafe, clinical testing may begin 31 days after the IND application is submitted to FDA. The sponsor then proceeds with the three main stages of clinical drug testing:   Phase 1 small-scale safety trials generally study small numbers of healthy volunteers to determine toxicity and safe dosing levels. These trials also study a drug’s pharmacokinetics, or how it is absorbed, distributed, metabolized, and excreted, and its concentration in the bloodstream;   Phase 2 small-scale efficacy trials generally study patient volunteers with the disease or condition against a comparison group to assess drug efficacy and side effects; and   Phase 3 full-scale safety and efficacy trials study thousands of patient volunteers against a comparison group to further evaluate efficacy and monitor adverse responses to the drug. Drugs for life-threatening diseases for which there is no other effective course of treatment sometimes cannot be compared against another treatment and will sometimes use historical information about patient outcomes as a point of comparison.
Drug sponsors are required to submit IND annual reports to FDA during the typically 2- to 10-year span of the clinical drug trials. When the sponsor wants to market a new drug, it submits a new drug application (NDA). FDA regulations on NDA content and format require that the NDA include integrated summaries of the evidence demonstrating the drug’s safety, including adverse events suffered by those in the clinical drug trials, and effectiveness. Evidence is also required to support the dosing section of the labeling, including the recommended dose and modifications in dose for specific population subgroups. Each NDA must include at least one pivotal clinical trial, generally an “adequate and well-controlled” Phase 3 study that demonstrates the drug’s efficacy, or effectiveness.
section_title: The Importance of Ensuring Women’s Representation in Trials
paragraphs: There are many examples in the medical literature of sex differences in the way men and women absorb, distribute, and metabolize drugs. The effects of women’s hormones and the variations in body size between men and women are the likely causes of many sex differences in responses to drugs. Women metabolize some drugs differently if they are pregnant, lactating, pre- or postmenopausal, menstruating, or using oral contraceptives or hormone replacements. Women’s generally smaller body weight compared to men can result in higher levels of drug concentration in the bloodstream.
These and other established physiological and anatomical differences may make women differentially more susceptible to some drug-related health risks and demonstrate the importance of including women in all stages of drug development. For example, phenylpropanolamine (PPA), a common ingredient in over-the-counter (OTC) and prescription cough and cold medications and OTC weight-loss products, was found to increase the risk of bleeding into the brain or tissue around the brain in women, but not in men. Certain classes of drugs can in some circumstances prolong the interval between the heart muscle’s contractions and induce a potentially fatal cardiac arrhythmia. Women have a higher incremental risk of suffering such an arrhythmia after taking these drugs than do men, probably because (1) the interval between heart muscle contractions is naturally longer for women than for men and (2) male sex hormones moderate the heart muscle’s sensitivity to these drugs. We recently reported that four of the ten prescription drugs withdrawn from the U.S. market in the last 3 years posed a greater health risk to women than to men because they induced arrhythmia. Similarly, there is evidence that not all drugs are effective in both sexes. For example, one class of painkillers, kappa opioids, has been found to be twice as effective in women as in men.
section_title: Women Were Historically Excluded From Some Clinical Drug Trials
paragraphs: Discoveries of birth defects and other problems resulting from fetal exposure to certain drugs between the 1940s and early 1970s prompted societal interest in protecting women and their fetuses from the potentially devastating effects of clinical drug research. For example, diethylstilbestrol (DES) was taken by women in the 1940s and 1950s to protect against miscarriages. About 20 years later, many daughters of women who had taken the drug developed reproductive abnormalities and had an increased risk of developing vaginal cancer. Similarly, in the 1960s many women outside of the United States took thalidomide to prevent early miscarriages, and the drug caused over 10,000 birth defects worldwide. In 1977, partially in response to the thalidomide scare, FDA recommended that women of childbearing potential be excluded from participating in small-scale safety and efficacy trials unless the drug was intended to treat a life-threatening disease. As a result, women were typically excluded from these clinical drug trials. Through the next decade there were growing concerns that the 1977 guideline may have restricted the early accumulation of information about women’s responses to drugs that could be used in designing later clinical drug trials and that it stifled the production and analysis of data on the effects of drugs in women.
In 1994, the Institute of Medicine (IOM) reported that the FDA guidance that discouraged the participation of women of childbearing potential in initial small-scale trials led to the widespread exclusion of women in later large scale trials. In addition, analyses of published clinical drug trials for life-threatening conditions have concluded that many past clinical trials included few or no women, making it uncertain whether the studies’ results applied to women. These conditions include cardiovascular disease and HIV.
section_title: Previous Studies on the Participation of Women in Clinical Drug Trials
paragraphs: This report is our second to address FDA and women in clinical drug trials. In 1992, we investigated FDA’s policies and the pharmaceutical industry’s practices regarding research on women in clinical drug trials. We reported that women were generally underrepresented in clinical drug trials in comparison to the proportion of women among those persons with the disease for which the drug was intended and that sex-related analyses were not routinely conducted. Even so, there were enough women in most clinical drug trials to detect sex differences in men and women’s response to drugs.
FDA has conducted its own studies on the inclusion of women in clinical drug trials. Surveys of NDAs in 1983 and 1988 found that, in general, both sexes were represented in clinical drug trials in proportions that usually reflected the prevalence of the disease in the total population but were not necessarily statistically sufficient to prove the safety or efficacy of the drug for either sex. Despite the participation of women, few analyses of the data were being conducted to detect possible sex differences in drug safety or efficacy. FDA has also looked at the tabulation of demographic data in IND annual reports. FDA recently reported that in IND annual reports filed with the agency women made up 44 percent of participants in clinical drug trials in which sex was identified. However, the FDA researchers found that sex could not be determined for more than one half of the participants in the IND annual reports.
section_title: FDA’s Regulation Not As Specific As Earlier Guidance
paragraphs: FDA has addressed women in clinical drug trials through the publication of guidance in 1993 and regulations in 1998 and 2000. The 1993 guidance for the pharmaceutical industry recommends that clinical studies include men and women “in numbers adequate to allow the detection of clinically significant gender differences in drug response” and that analyses of sex differences be included in NDAs. The 1998 regulation is less specific. It does not include references to how the number of women to be included in clinical drug trials should be determined. It requires only that safety and efficacy data already collected be presented separately for men and women in NDAs, but it does not require any discussion or analysis of these data. The 1998 regulation also requires the tabulation of study participants by sex in IND annual reports. The regulations issued in 2000 allow FDA to temporarily halt research programs for drugs for life-threatening conditions if men and women with reproductive potential are excluded from participation in ongoing studies.
subsections: section_title: FDA’s Guidance and Regulations
paragraphs: In response to our 1992 report, FDA issued policy guidance in 1993 regarding women in clinical drug trials, explicitly reversing its 1977 recommendation to restrict some women’s participation in drug development. Its 1993 Guideline for the Study and Evaluation of Gender Differences in the Clinical Evaluation of Drugs recommended that clinical drug trials should, in general, reflect the population that will receive the drug when it is marketed. This guidance also advised that enough men and women be included in clinical drug trials to allow for the detection of clinically significant sex differences in drug response, including those differences attributable to hormones and body weight variations.
On August 10, 1998, FDA implemented regulations amending requirements for INDs and NDAs to include demographic data. The regulation requires sponsors to tabulate the sex, age, and race of study participants in IND annual reports and to present available safety and efficacy data by sex, age, and race in two NDA documents submitted to FDA: the Integrated Summary of Safety and the Integrated Summary of Efficacy. The regulation also requires that evidence be presented to support dose determinations. FDA has the authority under these regulations to refuse to accept, or “file,” any NDA for review that does not include this information. In addition, FDA promulgated regulations on June 1, 2000, allowing it to halt IND studies involving drugs that are intended to treat life-threatening diseases or conditions if men or women of reproductive potential are excluded from participation solely because of risks to their reproductive potential. This regulation does not, however, impose requirements to recruit or enroll a specific number of men or women with reproductive potential, and FDA has not halted any studies under this authority. We did not evaluate whether FDA should have invoked this rule.
section_title: FDA’s 1998 Regulation Lacks Important Provisions of 1993 Guidance
paragraphs: The language of the 1998 demographic regulation is less specific than the 1993 guidance. The 1998 regulation has the force and effect of law, while the 1993 guidance does not legally bind either FDA or drug sponsors. The 1993 guidance specifically discusses the need to analyze clinical data by sex, evaluate potential sex differences in pharmacokinetics, including those caused by body weight, and conduct specific additional studies in women, where clinically indicated. The 1998 regulation requires the presentation of safety and efficacy data already collected in the NDA by sex, but no analysis of such data is required. The regulation does not include a standard for the inclusion of women; it requires only “presentation of data” without clarifying the extent of data or the format to be used. The regulation does require the identification of any modifications in dose or dose interval because of sex, age, or race, but not weight.
section_title: NDA and IND Submissions Often Fail to Present Required Information
paragraphs: We found that the NDA summary documents and IND annual reports submitted to FDA by drug sponsors frequently did not present information already collected during drug development separately for men and women, as required by the 1998 regulation. We found that 33 percent of the NDAs in our sample did not include presentations of both safety and efficacy outcome data separately for men and women. Similarly, we found that 39 percent of the IND annual reports in our sample did not include the required information about the sex of study participants.
One-third of the NDAs we examined did not include presentations for men and women of both safety data in the Integrated Summary of Safety and of efficacy data in the Integrated Summary of Efficacy. We considered the presentation of outcome data by sex in an NDA for just one of the studies included in that NDA to meet our criteria for regulatory compliance. Safety outcome data by sex, either data about toxicity or adverse events or both, were not included in 17 percent of the NDAs we reviewed. Similarly, 22 percent of the NDAs did not present efficacy outcome data separately for men and women.
We found that 39 percent of the IND annual reports in our sample did not include the demographic information required by regulation: 15 percent of the annual reports were not submitted to FDA and 24 percent did not tabulate the number of men and women enrolled in clinical drug trial studies. Only 37 percent of the annual reports tabulated the enrolled study populations by sex, as required by the 1998 regulations; 24 percent of the annual reports stated that there were no ongoing studies.
section_title: NDAs Include Appropriate Numbers of Women, but Analyses Sometimes Missing
paragraphs: All of the NDAs we examined included enough women in the pivotal trials to demonstrate statistically that the drug was effective in women, even if the sponsors did not report such an analysis or did not include the required presentation of outcome data in the NDAs. Overall, more women than men participated in clinical trials for the drugs we examined, although women were a minority of the participants in the initial, small- scale safety studies used to set the dosing levels for subsequent trials. We found that most of the NDAs included analyses to detect differences between men and women, but fewer of the NDAs explicitly included descriptions of both safety and efficacy analyses that compared women taking the drug with a comparison group of women taking a placebo or an alternative treatment. Analyses often detected sex differences. The sex differences that were detected were sometimes attributed to differences in body weight between men and women; none of the sex differences that were detected were judged to be clinically relevant, even when statistically significant. The NDA sponsors did not recommend different dosage levels for men and women based on the sex differences they detected.
subsections: section_title: Sufficient Numbers of Women Included to Determine Efficacy and Safety
paragraphs: All of the NDAs in our sample included enough women in the pivotal trials to demonstrate statistically that the drug was effective in women; that is, the numbers of women in the treatment and comparison groups of the pivotal studies were sufficient to detect a statistically significant difference between the treatment and comparison groups, given the magnitude of symptom improvement experienced by the treatment group. However, one drug was approved for use in men even though the NDA reported that no men participated in the pivotal studies.
We did not attempt to demonstrate statistically that the drugs in our sample were safe for women, because there are no absolute standards for the number of required study participants for assessing drug safety. Generally, the more patients that are exposed to a drug during its development, the more likely that significant adverse events will be detected. Safety determinations are largely based on adverse events reported for all participants in all studies. Since more women than men were included in clinical trials for the NDAs we examined, the adverse event data gathered for women were at least as extensive as the adverse event data gathered for men.
section_title: Progress Made in Including Women Overall, but Relatively Few Women in Early Studies
paragraphs: A larger percentage of participants in clinical drug trials are women than we found in our 1992 analysis of trials performed between 1988 and 1992. Adjusting for differences in the classes of drugs included in the studies, we found that the percentage of women participants in small-scale efficacy and full-scale safety and efficacy trials increased from 44 percent in our 1992 study to 56 percent in the NDAs we examined. In the current study, summing across all the clinical trials for all of the NDAs we examined, 52 percent of the study participants were women, 39 percent were men, and 9 percent were not identified by sex. When participants’ sex was identified, women were the majority of participants for 58 percent of the NDAs.
Women made up more than one-half of all the participants in small-scale efficacy and full-scale safety and efficacy trials. However, women were 22 percent of the participants in the initial, small-scale safety studies. One of the NDAs included no women in the early safety trials. These early safety studies are important because they measure how participants absorb, metabolize, and excrete a drug, and their findings are used to help set the dosage amounts for subsequent trials.
section_title: Frequency of Sex-Related Analyses Differs by Type and Purpose of Trials
paragraphs: NDAs usually contained sex-related analyses of safety and efficacy, regardless of whether the outcome data were presented in the summary documents as required by regulation (see table 1). Evidence of these analyses ranged from one-line summaries stating that there were no sex differences, to more complete, multi-page tables and descriptions of statistical methods and results. Specifically, most NDAs included analyses of safety and efficacy outcome data to detect differences between men and women in their responses to drugs. NDAs were less likely to include discussions of analyses of the safety and efficacy of drugs in women specifically by comparing women who received the drug and a comparison group of women.
Fewer NDAs included analyses of pharmacokinetic data by sex, even though analysis of pharmacokinetic data is explicitly recommended in the 1993 guidance. We found that 42 percent of NDAs presented outcome data for these early studies for both men and women. Seventy-five percent of the NDAs we reviewed had some evidence of an analysis of pharmacokinetic data for sex differences.
section_title: When Reported, Analyses Sometimes Found Sex- Related Differences
paragraphs: Many of the NDAs we reviewed reported differences in men and women’s responses to drugs, but fewer reported these differences to be statistically significant (see table 2). For example, while one-half of the NDAs reported drug safety differences between men and women, less than one-fifth of the NDAs reported statistically significant sex differences in drug safety. We found no evidence that any of the sex differences reported in any NDA on any dimension—safety, efficacy, or pharmacokinetics—even when statistically significant, were judged to be clinically relevant by either the NDA sponsors or the FDA reviewers, and no dose adjustments based on sex were recommended.
Some NDA sponsors also reported differences in either safety or efficacy between women receiving the drug and women in a comparison group (see table 3). About one-fifth of the NDAs reported statistically significant differences in safety between women taking the drug and a comparison group, and about one-half found statistically significant differences in efficacy.
section_title: Sex Differences Often Attributed to Weight, but Sex-Related Dose Adjustments Not Recommended
paragraphs: Apparent sex differences in pharmacokinetics, and sometimes safety and efficacy, may be due to differences in weight between the sexes instead of other biological differences. At a constant dosage, individuals who weigh less have a higher exposure to the drug than heavier individuals, and, on average, women weigh less than men. The potential for higher drug concentration or exposure can lead to an increased risk of adverse events for women. In our sample of NDAs, 36 percent reported pharmacokinetic differences based on weight, whether or not sex differences were also reported. Twenty-five percent of NDAs reported apparent sex differences in drug response between men and women that were attributed to weight, not sex. In these cases, the sponsors reported sex differences in drug response but then noted that the differences disappeared when weight was taken into account. In all of these cases of weight-related differences in men and women’s responses to drugs, the sponsors asserted that no dose adjustments were necessary based on sex. For two intravenously administered drugs and one injectable drug the NDA did indicate dose adjustments based on weight for all patients.
section_title: FDA Oversight Needs Improvement
paragraphs: FDA has not effectively overseen the presentation and analysis of data related to sex differences in drug development. There is no management system in place to record and track the inclusion of women in clinical drug trials or to monitor compliance with relevant regulations, so FDA is unaware that many NDA submissions fail to meet requirements. The agency also does not routinely review the required tabulation of demographic data by sex in the IND annual reports for drugs in development. Finally, FDA’s medical officers have not been required to discuss sex differences in their reviews, and we found that their reviews frequently did not address the results of sex-related analyses conducted by NDA sponsors. Until recently, FDA has also lacked procedures to determine whether the reviews of its medical officers adequately discuss sex differences. We did not find, nor did we look for, any evidence that FDA’s reviews of the NDAs we examined had negative public health consequences. Such an examination was beyond the scope of this study. Recently, FDA has taken steps to pilot test several initiatives to address these management needs.
FDA does not know how many women are included in clinical trials for each NDA or if NDA summary documents comply with the data presentation requirements of the 1998 regulation. There has been no systematic attempt by FDA to routinely collect and organize data on the inclusion of women in clinical trials. Although FDA officials told us that they believe that regulatory requirements are being met, FDA has no system in place to provide information that would support that assertion. The agency has not routinely tracked the required presentation of safety and efficacy data from women participating in clinical trials for the drugs it reviews.
FDA does not routinely review the required presentation of data about the sex of study participants in the IND annual reports. As we noted earlier, 39 percent of the required IND annual reports did not include the tabulation of demographic information about study participants mandated by the 1998 regulation. We found no evidence that FDA follows up with sponsors that have not submitted annual reports—about 15 percent in our sample. A senior FDA official told us that the agency does not rely upon the information in these reports to monitor pre-NDA drug testing. According to this official, the agency instead uses other reports submitted by the sponsors for which there are no regulatory requirements to tabulate clinical trial participants by sex.
FDA’s Medical Officer Reviews are important documents that detail FDA’s evaluation of the safety and efficacy of new drugs. We found that FDA’s medical officers have not been required to address sex differences in their reviews, and many of the medical officers’ reviews we examined did not address the sex-related data and analyses included in the NDAs (see table 4). For example, FDA’s medical officers did not discuss in their written reviews why reported differences between men and women in their responses to drugs did not require dose adjustments. In some cases, apparent contradictions in the NDAs about the role of sex or weight within the text of a drug application were not addressed.
Since December 2000, FDA has pursued several initiatives that directly address areas of concern related to the review of sex differences. First, to help track the number of women in clinical trials and to monitor the compliance of NDAs with data reporting regulations, FDA began pilot testing a worksheet for reviewers to capture demographic information about the participants in large-scale efficacy trials. Instructions for the worksheet that will allow it to be used by all of FDA’s reviewers are being developed. Second, to help ensure that its medical officers address sex differences, FDA began pilot testing a standardized template for Medical Officer Reviews. The template instructs medical officers to discuss sex- related issues in a standard format in all of their reviews. Third, an electronic training package was recently implemented to provide information to FDA’s medical reviewers on the guidance and regulations applicable to the review of sex-related data and analyses included in NDAs. However, FDA does not require reviewers to use the training package.
section_title: Conclusions
paragraphs: We found that women were a majority of the clinical trial participants in the NDAs we examined and that every NDA included enough women in the pivotal studies to be able to demonstrate statistically that the drug is effective in women. While these findings are welcome, we also found three areas of concern. The first is the relatively small proportion of women in early small-scale safety studies. These early studies provide important information on a drug’s toxicity and safe dosing levels for later stages of clinical development, and many of the NDAs we examined found significant sex differences in a drug’s pharmacokinetics, or how it is absorbed, distributed, metabolized, excreted, and concentrated in the bloodstream. Second, we are not confident that either NDA sponsors or FDA’s reviewers took full advantage of the available data to learn more about the effects of the drug in women and to explore potential sex differences in dosing. This is because NDA summary documents are not required to include analyses of sex differences, and some of them do not. Similarly, FDA’s medical officers have not been required to discuss sex differences in their reviews, and many of the reviews we examined did not include complete discussions of potential sex differences. Third, FDA does not now have appropriate management systems to monitor how many women are in clinical trials, to be assured that NDAs and IND annual reports are in compliance with pertinent regulations for presenting outcome data by sex and tabulating the number of women included in ongoing trials, or to confirm that its medical officers have adequately addressed sex-related issues in their reviews. While FDA has taken some promising initial steps to address these deficiencies, it is important that the agency finalize the pilot programs it has underway and give sustained attention to these management issues.
section_title: Recommendations for Executive Action
paragraphs: We recommend that FDA adopt management tools that will ensure drug sponsors’ compliance with current regulations regarding the presentation of data by sex and that its reviewers’ consistently and systematically discuss sex differences in their written reviews of NDAs. Specifically, we recommend that the Acting Principal Deputy Commissioner of FDA:   Promptly implement management tools, such as the proposed demographic worksheet and the standardized template for Medical Officer Reviews, that will allow the agency to determine whether NDAs and IND annual reports are in compliance with regulations that mandate the presentation of available safety and efficacy outcome data for women in NDAs and the tabulation of study participants by sex in IND annual reports.
Fully implement the proposed template for Medical Officer Reviews or take other actions to ensure that FDA’s medical officers consistently and systematically consider and discuss sex differences in their written reviews of NDAs.
section_title: Agency Comments
paragraphs: We received written comments from FDA on a draft of this report (see appendix III). FDA generally agreed with our findings. FDA did not comment on our recommendations, but outlined additional steps it may take to monitor the inclusion of women in clinical trials. FDA questioned our description of comparisons between men and women, and comparisons between women taking the drug and a comparison group of women, as two distinct types of analyses. FDA pointed out that an analysis of sex differences implies that an analysis of the drug’s efficacy in women has been completed because an analysis of sex differences is a comparison of the drug’s efficacy in men and women. We have clarified the text, but we continue to present information about both analyses in order to accurately reflect the contents of the NDA summary documents we reviewed. Finally, FDA pointed out that its efforts to improve its management in this area have been underway for some time. In response, we modified our description of FDA’s activities. FDA also made additional technical comments that we have incorporated where appropriate.
As we arranged with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days after its issue date. At that time, we will send copies of this report to the Acting Principal Deputy Commissioner of FDA and to others who request them.
If you or your staff have any questions, please contact me at (202) 512- 7119. Another contact and major contributors to this report are listed in appendix IV.
section_title: Appendix I: Objectives, Scope and Methodology
paragraphs: Our work addressed four questions: (1) what FDA regulations govern the inclusion of women in clinical drug trials; (2) are the regulations being followed; (3) are appropriate numbers of women included in clinical drug trials to ensure the safety and efficacy of drugs for women; and (4) how does FDA oversee the collection, presentation, and analysis of data related to sex differences? Our work did not include an examination of post marketing adverse events or negative public health consequences.
To assess FDA’s oversight of the collection, presentation, and analysis of data related to sex, we reviewed the FDA Medical Officer Reviews for all sampled NDAs. We also interviewed officials in FDA’s Center for Drug Evaluation and Research, the Office of Special Health Issues, and the Office of Women’s Health. We also interviewed officials from drug companies and an industry trade association. To gain background knowledge on the issues related to our work, we spoke with women’s health advocates and consulted pharmacology experts. We conducted a literature review that included relevant FDA guidance and regulations, FDA and IOM reports, medical journal articles, prescription drug labels, and consumer advocacy publications.
subsections: section_title: NDA Sample
paragraphs: Because FDA maintains no central source of data on the inclusion of women in clinical drug trials, we sampled NDAs for new molecular entities (NME) submitted to FDA from August 10, 1998 through December 31, 2000. Of the 82 original NDAs for NMEs submitted to FDA during this period, we examined all 36 that were either approved for marketing or judged approvable by FDA by December 31, 2000, and that met our other selection criteria. We narrowed our focus to only approved and approvable NDAs because these drugs are the most likely to reach the public. We excluded diagnostic drugs used in medical imaging, drugs for sex-specific conditions, pediatric drugs, and drugs that were not approved for use in both men and women. We also did not examine biologic products, such as vaccines. As a result of our sampling criteria, the clinical drug trials for some drug classes that have been cited by experts as including insufficient numbers of women were not well represented. For example, our sample included only one cardiovascular drug.
We requested from FDA and reviewed critical summary documents for each NDA, including the Integrated Summary of Safety, the Integrated Summary of Efficacy, the Pharmacokinetics and Bioavailability Summary, and the FDA Medical Officer Review. We obtained and reviewed other NDA documents only when the summary documents referred to relevant information. We reviewed the NDA summary documents because the 1998 regulations specifically require NDA sponsors to present data about drug safety and efficacy in the Integrated Summary of Safety and the Integrated Summary of Efficacy and because we were unable to review all of the documents in each NDA (an entire NDA can contain as many as 250 volumes). Our findings speak only to what was included in the summary documents or in the supplemental documents we examined; we did not systematically review other relevant data, such as data in clinical pharmacology reviews, that may have been presented in NDA volumes other than the critical summary documents.
In our reviews of the critical summary documents we collected data on (1) the presentation of outcome data by sex, (2) the number of women participating in clinical drug trials by drug development stage, (3) the frequency and extent of sex-related analyses, (4) the detection of sex- related differences in drug response and their statistical significance, and (5) the relationship between body weight and sex-related differences. The decision rules we used to code the NDAs are presented in table 5. In general, we coded the information we sought as present if there was any mention of it in the summary documents.
section_title: IND Sample
paragraphs: To determine if IND annual reports filed with FDA met the regulatory requirement for tabulating the sex of enrolled study participants, we randomly selected a sample of 100 IND applications that met our inclusion criteria from FDA’s November 2000 listing of active commercial IND applications. That listing included a total of 3,636 IND applications. According to FDA’s management information system, 15 of the IND applications in our sample had been withdrawn and were not active, and sponsors for 9 of the IND applications were not required to submit annual reports because they had not been active for a long enough period. We also found that FDA could not find one of the annual reports (see table 6). Because we randomly selected the IND annual reports we examined, our findings are generalizable to the entire set of IND annual reports. However, because of the small size of our sample, our estimate of the proportion of annual reports not fulfilling regulatory requirements is not precise.
In our review of the remaining 75 IND annual reports, the reports were considered to have met regulatory requirements if the numbers of enrolled participants were reported by sex for at least one of the reported studies. The regulation requires “tabulation” of the data; for purposes of our review we considered any presentation of the demographic data to meet the IND regulatory requirements.
section_title: Comparison With Our 1992 Study
paragraphs: We weighted the percentage of women by drug class to compare the percentage of women in clinical drug trials from our sample to that of our 1992 study. In weighting the percentage of women in our study by the percentage of participants in trials for each drug class used in the 1992 study, we were able to control for differences in the types of drugs sampled and compare the two studies as if our sample included the same drugs. For example, participants in cancer drug trials made up 7 percent of all participants in the 1992 study of clinical drug trials but only 5 percent of the participants in the small-scale efficacy and full-scale safety and efficacy clinical trials we examined in this study. By weighting our sample so that 7 percent of the study participants we found were in trials for cancer drugs, for example, we can fairly compare the percentages of women participating in clinical drug trials from our 1992 study to those from this study.
section_title: Power Analysis for Women in Pivotal Trials
paragraphs: In reviewing the 36 NDAs, we also collected information to determine whether enough women were tested in the clinical drug trials to detect sex differences. Standards for participation of women in clinical drug trials have included nominal thresholds for women’s participation (e.g., in our 1992 report, we regarded NDAs that tested 250 or more women as having enough women) and the representation of the sexes in numbers that are proportional to those in the population for whom a drug is intended. For this study, we adopted the perspective that the clinical trials should include a large enough number of women to demonstrate the safety and efficacy of the drug for women. To determine if enough women were tested in clinical drug trials to demonstrate the drugs’ efficacy in women, we generally conducted a power analysis using the number of participants in, and outcome data from, pivotal trials. NDAs that reported a statistically significant improvement in women taking the drug compared to women in a control group clearly had enough women in the pivotal trials to meet this criterion. For NDAs that did not report this analysis, we took the largest effect size presented in the Integrated Summary of Efficacy (that is, the largest percentage improvement for those taking the drug), the total number of women participating in the treatment group for all of the pivotal trials, and the total number of women participating in the comparison group for all of the pivotal trials. We then calculated the critical ratio, and significance level, for that effect size and that number of cases. We found that all of the NDAs we examined in this way had enough women in the pivotal trials to demonstrate that the drug had a statistically significant effect. We followed the convention that statistical tests with a probability level less than or equal to .05 are regarded as statistically significant.
We conducted our work from July 2000 through May 2001 in accordance with generally accepted government auditing standards.
section_title: Appendix II: Estimates of the Number of Men and Women in Clinical Drug Trials
paragraphs: We were able to estimate the number of men and women who participated in the clinical drug trials for the 36 NDAs in our sample by reviewing the NDA summary documents and FDA Medical Officer Reviews. Table 7 represents the estimated percentage of men and women who participated in the clinical drug trials by drug development stage. Table 8 represents the estimated number of men and women who participated in the pivotal clinical drug trials overall, and, where available, in the treatment and comparison groups of the pivotal trials. The data in both tables are grouped according to drug class. For some NDAs, the sex of some or all of the participants was not specified by clinical drug development stage or treatment group.
section_title: Appendix III: Comments From the Food and Drug Administration
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: Lisanne Bradley, Emily J. Rowe, Robert M. Copeland, Lawrence S. Solomon, Anh Bui, and Jenny C. Chen also made major contributions to this report.
section_title: Related GAO Products
paragraphs: Drug Safety: Most Drugs Withdrawn in Recent Years Had Greater Health Risks for Women (GAO-01-286R, January 23, 2001).
Women’s Health: NIH Has Increased Its Efforts to Include Women in Research (GAO/HEHS-00-96, May 2, 2000).
Women’s Health: FDA Needs to Ensure More Study of Gender Differences in Prescription Drug Testing (GAO/HRD-93-17, October 29, 1993).
National Institutes of Health: Problems in Implementing Policy on Women in Study Populations (GAO/T-HRD-90-50, July 24, 1990).