NIH, which had total budgetary resources of $32 billion in fiscal year 2016, is comprised of the Office of the Director and 27 institutes and centers that focus on specific diseases, particular organs, or stages in life, such as childhood or old age. As the central office at NIH, the Office of the Director establishes agency policy and is responsible for overseeing the institutes and centers to ensure that they operate in accordance with NIH’s policies. The institutes and centers accomplish their missions primarily through extramural research programs. Most extramural research funding is provided for investigator-initiated research projects for which researchers, through their institutions, submit applications in response to NIH announcements. In addition to these announcements, the institutes and centers may issue more narrowly scoped solicitations, through request for proposals, for research targeting specific areas. All extramural research project applications are to follow NIH’s process of peer review, which includes two sequential levels of review. The first level involves non-governmental experts assessing the scientific merit of the proposed applications and assigning them a priority score. The second level involves advisory councils at the institute or center associated with the grant application, that, in addition to scientific merit, consider the institutes’ and centers’ missions and strategic plan goals and public health needs. Advisory councils review grant applications and their scores, and, based on this review, make recommendations about which grant applications should be awarded funding. The director of each institute or center makes the final extramural funding decisions. NIH investigators also conduct research through NIH’s intramural research program. These efforts accounted for approximately 10 percent of NIH’s total budgetary resources of $32 billion in fiscal year 2016. NIH employs about 3,600 investigators working in its own laboratories and clinics. In addition, this research relies on another 6,000 investigators at various stages of research training who come to NIH for a few years to work as non-employee trainees, including about 2,500 who are postdoctoral fellows. According to NIH officials, intramural investigators are generally not allowed to apply for extramural or private grants, because their salaries are funded with the agency’s appropriations. The career path to become an independent extramural investigator generally consists of students completing graduate level education (i.e., research doctorate or clinical doctorate), postdoctoral research, or medical residency. When postdoctoral research is completed, the researcher will generally seek opportunities to become an investigator at a medical research center or as a faculty member at a university and begin the process of obtaining academic tenure—that is, a full-time, permanent faculty position. Once the postdoctoral researcher becomes a faculty member, he or she can generally begin applying for large NIH research project grants. Some researchers may become affiliated with other types of research institutions and also apply for grants. Investigators in medical research centers and university faculty are generally dependent on external funding to cover the cost of their research. Although biomedical investigators may be funded by other federal agencies—such as the National Science Foundation—and nonfederal sources, studies have shown that NIH is the most likely source of government funding for biomedical research. NIH’s research support for extramural investigators includes research project grants, fellowships, training grants, and career development grants. Some of the main funding mechanisms provided to institutions by NIH that fund investigators beginning their research careers include the following extramural grants: Large grants. NIH awards large renewable research project grants: R01 and R01-equivalent (R01e) grants. According to NIH, in fiscal year 2016, the average size of large grants was typically in excess of $460,000 total. R01and R01e grants are NIH’s most common type of grant, according to NIH. They are generally the largest type of grant available to investigators beginning their careers and, for purposes of this report, are therefore referred to as “large” grants. Large grants provide 3 to 5 years of financial support for discrete, specified research projects. According to NIH, it is generally expected that within that period a project can be completed, results published, and sufficient time will remain for the investigator to prepare a subsequent application for a renewal or new award before funding ends. Smaller grants. While some non-R01 equivalent (non-R01e) grants may match or exceed the amount of some R01e grants, they are generally of a lesser amount and, for purposes of this report, are therefore referred to as “smaller” grants. According to NIH, in fiscal year 2016, smaller grants were, on average, amounts that ranged from about $61,000 to about $1.1 million total. These grants provide limited funding for a relatively short period of time to support a variety of exploratory or developmental projects, including pilot or feasibility studies, collection of preliminary data, and secondary analysis of existing data. Career development grants. Also known as K-series grants, these grants are intended to provide mentored research opportunities and career enhancement experiences to support investigators or postdoctoral fellows at various stages of their research careers. NIH’s data show that in fiscal year 2016, career development grants were, on average, about $178,000 total. NIH generally classifies the career status of an extramural investigator based on whether the investigator has received a large NIH research grant. NIH considers early career investigators to be those who meet the definition of early stage and intermediate stage investigators. NIH also recognizes established and “other” investigators among those who apply for research grants. Table 1 lists NIH extramural investigators’ career stages and descriptions of these stages. According to NIH, it generally takes an early stage investigator up to 2 years to develop a successful application for a large grant and receive funding. Typically, investigators devote between 6 months to 1 year to write their first large NIH grant application. Most of these grants, with a funding period of over 3 years, require significant preliminary data to support the proposed hypothesis contained in the application. In addition, the median average time elapsed for applicants to learn whether they have been awarded a grant is 270 days, or 9 months. According to NIH, because most investigators beginning their careers do not receive large NIH research grants on their first attempt, these investigators might apply for smaller grants. They may also apply for career development grants that are intended to provide mentored research or training opportunities. According to research by the National Academies of Sciences, Engineering, and Medicine, and others, the biomedical research workforce is growing older at a rate that is disproportionate to the general American labor force. Some stakeholders in the scientific community have voiced concerns that large NIH research grants that can launch early career investigators are often being awarded to established investigators rather than early stage and intermediate stage investigators. For example, a recent National Academies report pointed out that between 1998 and 2003, the NIH budget grew from $13 billion to $27 billion, but the percentage of grants awarded to investigators who were in the early stages of their careers steadily declined. Many in the field have reported on the need to support investigators who are researching varied biomedical issues in order to maximize the number of new discoveries. Further, stakeholders within the scientific research community have reported on the uncertain path that investigators may encounter early in their careers and the prospect that they will ultimately pursue other career options. Several reports have found that certain racial and ethnic groups are underrepresented in the biomedical research workforce and in science. These reports have also provided data on gender workforce disparities. For example, a 2011 publication by the National Academies of Sciences, Engineering, and Medicine showed that, in 2006, underrepresented minorities made up about 29 percent of the U.S. population, but, in 2007, were awarded about 5 percent of science and engineering doctorates. Other studies have shown significant research funding disparities for investigators from underrepresented groups that apply to NIH for large research grants, such as R01 grants. In 2011, NIH funded a study that examined the association between grant recipients and the applicants’ race and ethnicity. The study found that R01 applicants that self-identified as African American were 13 percentage points less likely than white applicants to receive these grants. After controlling for other variables— including educational background, training, previous research grants, and publication record—African American applicants were 10 percentage points less likely to be awarded such a grant than a white applicant. Further, while women comprise about half of the postdoctoral graduates for the biological sciences in the United States, studies have shown a disparity in the number of female investigators in senior science research positions at universities. This disparity may result in a smaller number of female investigators among NIH grant applicants and may further contribute to their underrepresentation in certain facets of science. However, we previously reported that once female investigators apply for NIH grants, their likelihood of receiving NIH grants is the same as their male counterparts. Over the last 10 years, NIH has introduced programs and policies to support extramural investigators competing for their first large NIH research grant that leads to research independence. NIH developed certain programs to fund extramural researchers with the goal of stabilizing the biomedical research workforce. These targeted programs were intended to promote support for extramural investigators that had not yet received a large NIH research grant. The various programs include both large and smaller research grants, career development grants, and student loan repayments. Of particular note are the NIH Director’s New Innovator Award, which is intended to support investigators beginning their research careers with reviewer-determined highly novel research; and the Director’s Early Independence Award, which is intended to support reviewer-determined exceptional investigators who wish to pursue independent research directly, forgoing the traditional postdoctoral training period. In addition, the Pathway to Independence Award provides investigators beginning their research careers with a mentored research experience, which may lead to independent research positions. Some institutes and centers have established their own programs to support investigators beginning their research careers. For example, a subset of the National Institute of General Medical Sciences’ “Maximizing Investigators’ Research Award program” targets funding for laboratories led by an early stage investigator. In addition, the National Institute of Arthritis and Musculoskeletal and Skin Diseases’ “Supplements to Advance Research from Projects to Programs,” supports intermediate stage investigators by providing supplemental funding to existing research projects to encourage broader innovation and exploration of high-risk ideas. In addition, NIH’s LRP is designed to help recruit and retain highly qualified individuals into biomedical research careers. This program provides student loan repayments in return for a commitment to engage in NIH mission-relevant and certain statutorily-defined approved research. We examined the funding rates of early stage and intermediate stage extramural and intramural investigators who applied for both initial and renewal LRP payments. LRP payments to extramural investigators: The LRP funding rate (awardees/applicants) for extramural investigators applying for total (both initial and renewal) payments between fiscal years 2013 through 2017 was about 50 percent. During this period, 8,186 extramural investigators applied for initial LRP payments and 3,206 received them; 5,131 extramural investigators applied for renewal payments and 3,426 received them. Therefore, the funding rates were 39 percent for initial applicants and 67 percent for renewal applicants. Early stage and intermediate stage investigators had similar funding rates in receiving LRP payments during the 5-year period, though there was some variation each year. Early stage and intermediate stage investigators seeking initial LRP payments had funding rates of about 40 percent and 35 percent, respectively. Both of these categories of investigators seeking renewal LRP payments had a funding rate of 67 percent. LRP payments to intramural investigators: The LRP funding rate (awardees/applicants) for intramural investigators applying for total (both initial and renewal) LRP payments from fiscal years 2013 through 2017 was about 87 percent; 397 intramural investigators applied for both initial and renewal LRP payments, and NIH funded 345 of the applicants. The funding rate for applicants seeking initial LRP payments during this 5-year period was about 83 percent, whereas the funding rate for those applying to renewal LRP payments was 90 percent. NIH also implemented policies to improve opportunities for early and intermediate stage extramural investigators. For example, to address the concerns about established investigators receiving a disproportionate share of research funds, NIH established its Early Stage Investigator Priority Policy in 2008. The policy specified that early stage investigator status would be considered a factor when applications were being selected for award. Studies have shown that under the Early Stage Investigator Priority Policy, grants being awarded to early stage investigators stopped declining and remained flat for several years. They also showed that the field of biomedical research continued to be very competitive for early stage investigators. However, some have expressed concern that these accomplishments are not sufficient. For example, according to a recent report by the National Academies of Sciences, Engineering, and Medicine, a variety of steps have been taken over the years to address the challenges facing early and intermediate stage investigators, but these efforts have not resolved the underlying problems that make it difficult for them to establish their careers. More recently, the Cures Act required that NIH implement the NGRI, which the agency established in August 2017. NIH’s Office of the Director, which oversees the initiative and its implementation, directed the NIH institutes and centers to reprioritize large NIH research grant support for early stage and intermediate stage investigators. The policy’s stated goal for fiscal year 2017 was to increase the number of large NIH research grants provided to both early stage investigators and intermediate stage investigators by 200 grants each compared to the number that were awarded in fiscal year 2016. These 400 grants would redirect approximately $210 million from NIH’s base budget to support additional early career investigators in the first year of NGRI’s implementation. However, with only one month to implement the policy, NIH did not meet this goal. From fiscal year 2016 to fiscal year 2017, the number of large NIH research grants awarded increased by 57 for early stage investigators and decreased by 2 for intermediate stage investigators. Similarly, the goal to increase funding for the additional 400 grants was not met; funding increased by about $107 million during this period. Given that this initiative is in the early stages and its goals were set late in fiscal year 2017, it is too early to fully assess the impact of this effort. According to NIH officials, the agency is in the process of reevaluating which investigators should be the focus of the NGRI initiative and may revise the program to include investigators whose careers are more advanced. NIH officials stated that the NGRI policy’s intention to direct more research funding to early stage investigators will remain in place. However, NIH’s NGRI Working Group no longer designates intermediate stage investigators—or what it calls “early established investigators”—as a distinct group. NIH’s current definition—that of being within 10 years of receiving a first large NIH research grant as an early stage investigator— includes investigators who could have completed their graduate level education (i.e., research doctorate or clinical doctorate), postdoctoral research, or medical residency between 15 and 20 years ago. According to NIH officials, NIH’s working group is considering broadening this definition even further. It is concerned that intermediate stage investigators, facing increasing pressure to secure additional sources of research funding to prevent the closure of their laboratories if their first large NIH research grant is not renewed, could lose all NIH support and become likely to leave the biomedical research workforce. Therefore, the working group is considering a different approach for all established investigators, with a focus on all meritorious investigators (regardless of career stage) who are doing high quality research, yet are still at risk for losing all NIH funding. Specifically, NIH officials said the working group plans to reevaluate ways that it can provide additional, prioritized support to these investigators in order to further their career trajectories. The working group may recommend to NIH that the NGRI be expanded to also target support for certain investigators whose careers are in more advanced stages, rather than just those in the early stages of their careers. In addition, NIH has not yet implemented the expansion of its LRP as directed by the Cures Act. The Cures Act amended the LRP by increasing the eligible annual loan repayment amount from a maximum of $35,000 to a maximum of $50,000. The act also gave the NIH Director the discretion to amend the research categories that are eligible for intramural or extramural loan repayment based on emerging scientific priorities or workforce needs. The agency has established a working group to provide recommendations to the NIH Director regarding any suggested structural changes and associated timelines for implementation. NIH officials told us that they are awaiting recommendations from this working group on how to use the agency’s new authorities. They said that they expect to implement program changes to the LRP, as permitted by the Cures Act, by fiscal year 2020. Our analysis shows that intermediate stage investigators are more successful at competing for grants than early stage investigators. Our examination of the trends of NIH grant data showed that the applicant funding rates (awardees/applicants) for investigators who had previously received an initial large NIH research grant was greater than the applicant funding rates for investigators who had never received such a grant. We analyzed 5 years of grant data to determine an overall perspective of funding rates from fiscal years 2013 through 2017. We found that intermediate stage and established investigators—groups comprised of investigators who had already received their first large grant award—had greater applicant funding rates for all three grant types compared to early stage and other investigators. For example, we found that in fiscal year 2017, the most recent year for which data were available, intermediate stage investigators had funding rates that were comparable to those of established investigators. Investigators that had not yet been awarded their first large NIH research grant—early stage investigators and other investigators—were not as successful when competing for large NIH research grants, small grants, or career development grants. (See table 2.) We also found that over time—from fiscal years 2013 through 2017— intermediate stage investigators and established investigators had greater applicant funding rates for all three grant types compared to early stage and other investigators. Of the investigators that had not yet been awarded their first large NIH research grant, early stage investigators were more successful in competing for NIH grants than the other investigators that were outside of the 10-year period of having completed their graduate level education (i.e., research doctorate or clinical doctorate), postdoctoral research, or medical residency. For instance, we found that early stage investigator funding rates ranged from about 5 to 11 percentage points lower than intermediate stage or established investigators for each of the five fiscal years examined. Similarly, other investigator funding rates ranged from about 12 to 14 percentage points lower than intermediate stage or established investigators for each of the five fiscal years examined. (See fig. 1.) Finally, we found that during this 5-year period, two of the four extramural investigator groups were more likely to receive large, small, and career development grants than the other two groups. Specifically, investigators beginning their research careers—the early stage and intermediate stage investigators—were more likely to receive these grants. Although early stage investigators were more likely than intermediate stage investigators to apply for smaller research grants (about 4,500 applicants compared to about 2,000 applicants, respectively) and career development grants (about 2,000 applicants compared to about 50 applicants, respectively), intermediate stage investigators were still more successful in competing for these grants, as well as the large NIH research grants. For more information on the trends in the number of grants awarded to early stage and intermediate stage investigators, by award type, for fiscal years 2013 through 2017, see appendix I. Over the last 7 years, NIH established advisory groups and other programs to determine how best to support extramural and intramural investigators from underrepresented groups. NIH’s Working Group on Diversity in the Biomedical Research Workforce was established in response to the 2011 NIH study that examined the association between R01 grant recipients and the applicants’ race and ethnicity. NIH directed the group to provide recommendations to improve retention of underrepresented minorities, the disabled, and scientists from disadvantaged backgrounds. In June 2012, the working group issued 13 recommendations, which, we found that NIH uses as the foundation of some NIH-wide efforts to diversify the extramural and intramural biomedical research workforce. Other advisory groups that have examined or are currently examining related topics include the following: NIH Working Group on Women in Biomedical Careers was established in 2007 in response to a report from the National Academies of Sciences, Engineering, and Medicine on barriers women in biomedical science experience in advancing their careers. It produced a workshop and report in 2008 on best practices for sustaining the careers of women in biomedical research; Addressing Gender Inequality in the NIH Intramural Research Program Action Task Force was established in 2016 in response to data showing women are underrepresented in top NIH research positions. It produced recommendations in 2017 aimed at ensuring that female and male investigators have equal opportunities in the intramural research program at NIH, among other things; and African-American/Black R01 Funding Disparities Working Group was established in response to the 2011 NIH study that found a funding disparity between blacks and whites applying for R01 grants. This group analyzed data on the funding rates of applicants that self- identify as African American or black compared to other racial groups. NIH has acted on some of the advisory groups’ recommendations. For example, in response to recommendations made by the Diversity in the Biomedical Research Workforce advisory group, the agency hired a Chief Officer of Scientific Workforce Diversity in 2014; implemented the three- tiered Diversity Program Consortium, which includes the Building Infrastructure Leading to Diversity program, the National Research Mentoring Network, and the Coordination and Evaluation Center; and established a permanent advisory group on diversity. NIH also developed a “toolkit” that includes training modules to educate intramural investigator search committee members on biases that can lead to a less diverse workforce, among other things. In fiscal year 2017, NIH created an Equity Committee to address recommendations made by the Addressing Gender Inequality in the NIH Intramural Research Program Action Task Force to further examine concerns about parity between male and female intramural investigators and other diversity issues. Other NIH-wide policies and programs may also help to attract, retain, and develop investigators from underrepresented groups. The 24 NIH institutes and centers that fund research and the Office of the Director provide funds for its investigators, called research supplements, to recruit graduate students, postdoctoral fellows, and others from underrepresented racial and ethnic groups, as well as those with disabilities and from economically disadvantaged backgrounds. These funds provide graduate students, postdoctoral fellows, and others an opportunity to conduct research and be mentored by an investigator supported by the specific NIH institute or center or office. Some stakeholders we interviewed said that the agency’s LRP also may help to retain investigators from underrepresented groups, noting that the student loan debt for African American or black graduate students is higher than that of white graduate students. Physicians from a professional organization we interviewed said that the LRP helps to attract physician scientists from underrepresented groups into research careers. Physicians we interviewed stressed the importance of the LRP to attract physician scientists into research careers, because these scientists often have significant medical school debt. Our analysis of extramural LRP data showed that, in 2017, African Americans or black, non-Hispanics had a funding rate of about 34 percent for receiving an LRP payment. White, non-Hispanic applicants had a funding rate for receiving an LRP payment of about 52 percent. More recently, the National Academies of Sciences, Engineering, and Medicine recommended that NIH make the LRP available to all individuals pursuing biomedical physician-scientist researcher careers, regardless of their research area or clinical specialty. They also suggested NIH increase the monetary value of loan repayment to reflect the debt burden of current medical trainees. Some stakeholders said that NIH’s family friendly policies, such as reimbursement for child care expenses and parental leave, may also help address work-life balance issues for female investigators that may otherwise forego some research duties to care for young children. Additionally, many—at least 17 of 27—of NIH’s institutes and centers have established their own policies and programs to attract, retain, and develop investigators from underrepresented groups. For example, the National Cancer Institute initiated the Continuing Umbrella of Research Experiences program to provide training and career development opportunities to enhance and increase diversity in the cancer research workforce. This program offers research opportunities and development to future and current scientists from underrepresented groups from middle school students to investigators who have yet to achieve research independence. Although NIH has implemented numerous diversity-related efforts, our analysis of NIH research grant funding and intramural workforce data from fiscal years 2013 through 2017 shows that some disparities persist for investigators from underrepresented racial and ethnic groups, and for female investigators. Our analysis of NIH data shows that investigators from underrepresented racial and ethnic groups comprise a small percentage of applicants. For example, in fiscal year 2017, applicants from underrepresented racial groups—that is, American Indian or Alaskan Native, African American or black, and Native Hawaiians and Pacific Islanders—were 0.2 percent, 1.8 percent, and 0.1 percent, respectively, of all applicants for large NIH research grants. Applicants from underrepresented ethnic groups— Hispanics or Latinos— comprised 4.3 percent of the applicants for large NIH research grants. (See table 3.) In contrast, white applicants were about 64 percent of all applicants for large NIH grants in fiscal year 2017. Investigators from underrepresented racial and ethnic groups also comprise a smaller number of applicants than other groups for smaller NIH grants and career development grants. Among grant applicants from underrepresented racial groups, African American or black applicants were consistently the largest group represented. For example, in 2017, among underrepresented racial groups, African American or black applicants were named as investigators on about 88 percent of applications for large NIH research grants, about 89 percent of applications for smaller NIH grants, and about 92 percent of career development grant applications. Hispanics and Latinos were about 5 percent of applicants for smaller NIH grants and about 6 percent of applicants for career development grants in 2017. According to data published by the National Science Foundation in 2017, women represent slightly more than half of all doctorates in biological sciences. However, from 2013 through 2017, women represented less than one-quarter of all tenured NIH intramural investigators. For example, in 2017, 191, (23 percent) of NIH’s 822 intramural tenured investigators were women. In addition, in 2017, 79, (37 percent), of NIH’s 211 tenure-track intramural investigators were women. Further, in fiscal years 2013 through 2017, nearly one-third of all extramural investigators that applied for large grants were women. (See table 4.) Nearly one-third of all applicants for smaller research grants, and close to half of all applicants for NIH career development grants, were women. (See app. II for information on the number of smaller and career development grant applicants by racial and ethnic groups and gender.) Stakeholders from 8 of the 12 entities we interviewed suggested potential reasons why the number of NIH research grant applicants among underrepresented racial and ethnic groups and for women may be limited. Attrition of biomedical science doctoral students and early career investigators from these groups is one explanation. Some stakeholders said that, while in graduate school, students from these groups may be discouraged from pursuing a biomedical research career as a result of implicit bias that they encountered with their mentors. Some stakeholders said lower numbers among women investigators is the result of decisions of some to start a family in the early stages of their careers, and further noted the difficulty in re-entering the biomedical research workforce. In addition, some stakeholders said that students from underrepresented groups may lack exposure to a sufficiently rigorous education in mathematics or the sciences prior to entering college, resulting in the low numbers of biomedical researchers from these groups. Others said the low numbers of investigators from these groups makes studying this issue difficult due to a small sample size. Additional administrative demands placed on individuals who pursue careers as investigators also affect the number of applicants. For example, some stakeholders said that once investigators from an underrepresented group attain faculty positions— particularly if there are few faculty members from such groups—they are frequently tasked with additional administrative duties. We were told that, often, they are selected because they may be one of a handful of members of underrepresented groups at some institutions. Their additional duties include participation on institutional committees as well as mentoring, particularly undergraduate or graduate students from underrepresented groups. In addition, representatives of one stakeholder group said that some research faculty from underrepresented groups feel additional pressure to participate in such activities, because their absence would be more apparent and they worry that this may adversely affect them. Stakeholders also told us that additional duties are time consuming and leave less time to devote to applying for grant funding. They said that some biomedical graduate students from underrepresented groups decide to pursue other fields, because of the competing demands associated with being an academic, such as grant writing and teaching responsibilities. Our analysis of NIH data from fiscal years 2013 through 2017 also shows that the funding rate for applicants from underrepresented racial groups applying for large and small NIH grants lags behind that of white applicants. For example, in fiscal year 2017, the applicant funding rate for large grants was about 17 percent for underrepresented racial groups and about 24 percent for Hispanics and Latinos. The funding rate for white applicants was about 27 percent. (See fig.2.) Among underrepresented racial groups, African American or black applicants consistently had a lower funding rate for large and smaller grants than well represented groups during this period (see table 5). The applicant funding rate for career development grants for underrepresented racial groups increased from about 22 percent to about 32 percent from fiscal years 2013 to 2017, and, for Hispanic and Latino applicants, from about 30 percent to about 36 percent during the same period. The applicant funding rate was about 34 percent for white applicants throughout this period. The large grant funding rate for female investigators was slightly lower than male investigators. (See fig. 3.) When looking exclusively at R01 grants, as opposed to all large grants, research has shown that women are less likely to have their initial R01 grant renewed. Our analysis of R01 grant renewal funding showed that, in fiscal year 2017, the R01 grant renewal funding rate for female applicants was about 31 percent compared to about 38 percent for male applicants. (See fig 4.) According to research by NIH, some applicants that are unsuccessful in obtaining an initial R01 grant may have greater success if they reapply; however, some stakeholders we interviewed said women, and some underrepresented racial groups, are less likely to reapply for an initial R01 grant if they are unsuccessful with their first attempt. (See app. III for information on the applicant funding rates for smaller grants and career development grants by gender.) Many stakeholders attributed the underrepresented groups’ lower funding rates to two factors. First, many stakeholders cited a perceived implicit bias within the peer review process, which they said may affect the funding rates for investigators from underrepresented racial and ethnic groups. They stressed that, many times, peer reviewers approve grants for investigators from top tier institutions that they are familiar with and are reluctant to provide high scores to grant applications from other institutions. Some stakeholders advocated for anonymizing grant applications to some extent to address this issue. NIH’s Center for Scientific Review—the center responsible for organizing peer reviews for grants—is conducting a study that anonymizes certain large grant applications, and a training module on implicit bias is currently being offered to NIH peer reviewers. In addition, NIH’s African American/Black R01 Funding Disparities Working Group has conducted an analysis on the R01 funding disparities for African American or black applicants from fiscal years 2010 through 2015, and is currently pursuing several efforts to address its findings. Lower grant application priority scores and application resubmission rates among African American or black applicants were among their findings. The working group is also pursuing a randomized control trial to assess the effect of mentoring and coaching on R01 resubmissions and award rates. Second, some stakeholders told us that only a very small percentage of biomedical science professors at top tier research schools are from underrepresented racial or ethnic groups. Some stakeholders suggested that many investigators from underrepresented groups seeking grants are affiliated with institutions outside of the top tier that may lack the infrastructure, grant writing support, and mentoring opportunities, which could help ensure their success. As a consequence, many investigators from underrepresented groups are at a disadvantage compared to their peers at top tier institutions, according to the stakeholders we interviewed. Although NIH has taken steps to address concerns about the diversity of the biomedical research workforce, its accomplishments have not been fully evaluated. Positive comments from some stakeholders we interviewed included praise for the steps NIH has taken to diversify the biomedical research workforce, the value of the National Research Mentoring Network, and the research supplements and other training grants offered by NIH’s centers and institutes, which provide opportunities for students and postdoctoral fellows from underrepresented groups to work with established investigators. NIH’s support of conferences and programs, such as the Annual Biomedical Research Conference for Minority Students and the Institutional Research and Academic Career Development Award, was also well regarded by stakeholders. They also noted NIH’s commitment to diversity and willingness to investigate diversity issues through advisory groups, and commended the agency on working to address recommendations from the Working Group on Diversity in the Biomedical Research Workforce, including hiring a Chief Officer of Scientific Workforce Diversity. Some stakeholders were actively engaged in working with NIH on diversity issues. For example, some physicians from an organization we interviewed said they are working with the National Institutes on Minority Health and Health Disparities on issues related to research workforce diversity. Stakeholders, though, also offered less favorable views and characterized NIH’s efforts as stagnant, ineffective, or in need of better coordination. For example, some stakeholders suggested that for NIH’s National Research Mentoring Network, the matching of mentees to mentors could be improved or mentioned uncertainty about the program; questioned how often research supplements are utilized, or noted that better mentoring and follow-up after the postdoctoral fellow’s work is completed is warranted; reported that while their organizations initially collaborated with the scientific workforce diversity office, that office is not very active or communication eventually dissipated; expressed concern about NIH’s outreach to minority serving institutions and organizations, such as historically black colleges and universities, when it began creating programs like the Building Infrastructure Leading to Diversity program and the National Research Mentoring Network and for other efforts; and stressed that NIH should collaborate more with organizations that represent underrepresented groups, which have already implemented programs shown to be effective in engaging these communities in biomedical research. According to NIH officials, evaluations of various NIH efforts are ongoing and have not been completed. Some examples include the following: Data collection and analysis by the Diversity Program Consortium’s Coordination and Evaluation Center began in 2017, and is ongoing. In 2017, NIH’s Center for Scientific Review began conducting a study to anonymize R01 grant applications from African American or black and white applicants to detect potential reviewer bias during peer review. The results of this study are expected in 2019. An evaluation of the National Cancer Institute’s Continuing Umbrella of Research Experiences program, which provides training and career development opportunities to enhance diversity in the cancer research workforce, was submitted for publication in a scientific journal and is currently pending review. Some NIH institutes and centers have conducted evaluations of their specific diversity efforts. For example, in 2015, the National Institute of General Medical Sciences analyzed the research supplements provided to graduate students and postdoctoral fellows from underrepresented racial and ethnic groups between 1989 and 2006. The study found that about 65 percent of graduate students and postdoctoral fellows supported by the program entered research careers in academia, industry, and government research. About 41 percent of doctoral graduates and 45 percent of postdoctoral fellows supported by this program entered careers in academic research or teaching compared to about 43 percent of the U.S. doctoral degree workforce. In 2011, the National Institute on Aging evaluated its research supplement program and found that the NIH research grant applicant success rate of former participants from 2002 to 2010 was about 21 percent. The average research grant success rate for National Institute on Aging grants was about 18 percent during this same period. In 2016, NIH’s Chief Officer of Scientific Workforce Diversity established a 5-year strategic plan that describes the agency’s five workforce diversity goals and supporting objectives. The strategic plan includes goals and objectives that apply to both extramural and intramural investigators. During the course of our audit work, NIH updated this plan to describe progress made on each of its diversity goals, which are to: expand scientific workforce diversity as a field of inquiry, build and implement evidence related to diversity outcomes, understand the role of sociocultural factors in biomedical recruitment sustain nationwide workforce diversity with seamless career transitions, and promote the value of scientific workforce diversity. NIH officials provided us with performance measures that its scientific workforce diversity office will use to gauge the agency’s progress in achieving each of its five strategic plan’s goals. However, these items outline the particular areas that NIH plans to evaluate, rather than provide quantitative metrics, evaluation details, or time frames associated with any of the areas by which to evaluate progress in fulfilling the goals of the strategic plan. For example, for the first scientific workforce diversity goal “expand scientific workforce diversity as a field of inquiry” one of the performance measures is “number of publications stored in the scientific workforce diversity office’s online database.” Neither the strategic plan nor the additional documentation that NIH provided specifies a quantitative metric for the number of publications to be stored in its database and the time frame for doing so. Similarly, for the second scientific workforce diversity goal, to “build and implement evidence related to diversity outcomes” one of the performance measures identified by NIH is to compare the large grants awarded to African American or black scientists to those received by scientists who are white or from other racial and ethnic groups. However, there is no description in either the strategic plan or the additional documentation provided by NIH that indicates how and when these comparisons will be made, how the results of these comparisons will be assessed, and what will be considered as fulfilling this goal. All of the other areas or “performance measures” associated with each of the five goals also do not include such details or time frames. According to documentation provided by NIH its strategic plan does not explicitly list “specific metrics” because they will be defined within “the implementation phase of the plan.” However we are at the midpoint of the implementation of NIH’s 5-year plan, which covers the period of 2016 through 2020. As of May 2018, these specific metrics were not yet available. Without quantitative metrics, evaluation details, or time frames for assessing the agency’s performance against the five goals in its strategic plan, NIH will be unable to hold itself accountable for fulfilling its goals. This is inconsistent with best practices for strategic workforce planning, which call for agencies to monitor and evaluate their progress toward their human capital goals. These best practices also call for performance metrics to be specified at the outset to avoid a biased determination of what counts as “success” after the results are known. Further, this is inconsistent with federal internal control standards for monitoring, which require that an agency evaluate and document the results of ongoing monitoring to determine whether its management strategies are effectively supporting its objectives, or need corrective action. NIH’s establishment of goals and associated areas of future evaluation are positive steps, but absent specific measures by which to hold itself accountable, the agency will not have a basis to judge its success. NIH’s ability to fulfill its mission of advancing scientific knowledge and innovation to enhance health, lengthen life, and reduce illness and disability is dependent on its success in sustaining a thriving and diverse workforce. For decades, concerns have been raised by the biomedical research community about NIH’s ability to support investigators beginning their research careers. Similar concerns have been expressed regarding support for investigators from groups underrepresented in the sciences, including those from racial and ethnic groups and women. While the agency has taken many steps during this time, disparities in its research grant funding persist. NIH has conducted some evaluations of individual programs and activities, but these have been relatively narrow in focus and the results of many efforts are not yet available. More recently, NIH has taken positive steps such as by establishing the position of Chief Officer of Scientific Workforce Diversity, who in turn, created a strategic workforce diversity plan and related goals and identified areas of future evaluation. However, NIH does not have quantitative metrics, evaluation details, and time frames to assess its progress in meeting its strategic workforce diversity goals. Without these elements, NIH’s ability to assess how its diversity strategic plan goals are being achieved is hindered. Thus, NIH is missing an opportunity to better position itself to support underrepresented groups and address longstanding disparities. The NIH Director should develop quantitative metrics, evaluation details, and specific time frames to assess its current efforts to support investigators from underrepresented groups against its scientific workforce diversity strategic goals, and use the results of its assessment to guide any further actions. (Recommendation 1) We provided a draft of this report to HHS for comment. In its written comments, which are reproduced in appendix IV, HHS concurred with our recommendation and outlined the steps NIH is taking to implement it. Notably, for example, HHS indicated that NIH is establishing time frames to assess its progress in meeting its workforce diversity goals. HHS also provided technical comments, which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees and the Secretary of Health and Human Services. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or crossem@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix V. Table 6 provides details on the number of grants awarded, number of awardees and award type for early stage and intermediate stage investigators from fiscal year 2013 through fiscal year 2017. Tables 7 through 10 provide details on the demographics of NIH grant applicants during fiscal years 2013 through 2017. Figures 5 and 6 provide details on the demographics of NIH grant applicants during fiscal years 2013 through 2017. In addition to the contact above, Geri Redican-Bigott (Assistant Director), Carolina Morgan (Analyst-in-Charge), Jackie Hamilton, Toni Harrison, and Drew Long made key contributions to this report. Muriel Brown, Giselle Hicks, and Hayden Huang also made contributions to this report.

Summary: NIH's success depends on its ability to attract, retain, develop, and otherwise support biomedical investigators—including those employed in its intramural research program as well as those working in its extramural program at universities, academic health centers, and other research institutions. For decades, the agency has faced challenges in supporting early career investigators and those from underrepresented groups, including ethnic and racial minorities and women. The 21st Century Cures Act included provisions that NIH coordinate policies and programs to promote early research independence and enhance the diversity of the scientific workforce. The act also contained a provision that GAO examine NIH's efforts. GAO reviewed the actions NIH has taken to support (1) investigators beginning their biomedical careers; and (2) investigators from underrepresented groups and women. GAO analyzed NIH data from fiscal years 2013 through 2017 on grant funding for investigators by career phase and demographic status. GAO also reviewed relevant laws and NIH policies, programs, and initiatives, and interviewed NIH officials and stakeholders from the scientific research community. The National Institutes of Health (NIH), within the Department of Health and Human Services (HHS), plays a prominent role in the nation's biomedical research. While it employs investigators in its intramural research program, over 80 percent of its budget supports its extramural program, primarily through grant funding to investigators at other research institutions. Given this, NIH has a vested interest in supporting a robust national biomedical workforce, but the agency has acknowledged that the environment is highly competitive and many investigators find that it takes years to obtain the type and amount of funding that typically spurs research independence. GAO's analysis found that extramural investigators who had received at least one large NIH research grant during fiscal years 2013 through 2017 were more likely to receive such grants in subsequent application cycles than investigators who had not yet received such grants. In response to the 21st Century Cures Act, enacted in December 2016, NIH introduced an initiative to prioritize these grants for (1) early stage investigators, who are beginning their careers and have never received a large research grant, and (2) intermediate stage investigators, who are within 10 years of receiving their first large grant as an early stage investigator. However, it is too early to assess this new initiative, which was introduced in August 2017. NIH is currently considering revising the program to include investigators whose careers are more advanced. NIH implemented recommendations made by internal advisory bodies to support investigators from racial and ethnic groups considered by NIH to be underrepresented in biomedical research. GAO's analysis shows disparities for underrepresented racial and ethnic groups, and for female investigators, from 2013 through 2017. For example, in 2017, about 17 percent of investigators from underrepresented racial groups—African Americans, American Indians/Alaska Natives, and Native Hawaiian/Pacific Islanders combined—who applied for large grants received them. In contrast, about 24 percent of Hispanic or Latino applicants, an underrepresented ethnic group, received such grants. Asians and whites—well represented groups—were successful in receiving large grants about 24 and 27 percent of the time, respectively. Though women represent about half of all doctorates in biological science, GAO found that women investigators employed by NIH in its intramural program comprised about one-quarter of tenured investigators. NIH has taken positive steps such as establishing the position of Chief Officer of Scientific Workforce Diversity, who in turn created a strategic workforce diversity plan, which applies to both extramural and intramural investigators. The plan includes five broad goals for expanding and supporting these investigators. However, NIH has not developed quantitative metrics, evaluation details, or specific time frames by which it could measure the agency's progress against these goals. The Director of NIH should develop quantitative metrics, evaluation details, and time frames to assess NIH's efforts to diversify its scientific workforce against its diversity strategic plan goals, and take action as needed. HHS agreed with GAO's recommendation.