U.S. Department of Health and Human Services 
  
Office of Inspector General 
 
 
 
Selected NIH Institutes Met 
Requirements for 
Documenting Peer Review But 
Could Do More To Track and 
Explain Funding Decisions
 
 
Suzanne Murrin 
Deputy Inspector General for Evaluation and Inspections 
June 2021, OEI-01-19-00140  U.S. Department of Health and Human Services 
O  ffice of Inspector General 
 
Report in Brief 
 
Ju ne 2021, OEI-01-19-00140 
 
Selected NIH Institutes Met Requirements 
Why OIG Did This Review 
Congress, the National Institutes of 
for Documenting Peer Review But Could Do 
Health (NIH), and others have raised 
concerns about the integrity of U.S.  More To Track and Explain Funding 
medical research.  In August 2018, 
Decisions 
the NIH Director stated that the risks 
to the integrity of peer review were 
Each year NIH uses a peer review 
increasing.  Subsequently, Congress 
Key Takeaway  process to evaluate thousands of 
provided the Office of Inspector 
applications for grants to support its 
General with $5 million for oversight  NIH met its requirements for 
research mission.  This process is  
of NIH.  Peer review is how NIH uses  documenting first-level peer 
intended to be fair, equitable, and 
scientific experts to evaluate grant  review for extramural grant 
timely.  ICs must document the peer 
applications for funding.  This study  applications in the six ICs we 
review process and associated 
assesses the extent to which select  reviewed.  It has limited 
decisions.  In funding grants, NIH 
NIH institutes and centers (ICs) met  insight, however, into the 
allows forand benefits fromICs 
NIHs requirements for documenting  extent and nature of the ICs 
discretion to fund less-favorably 
first-level peer review when  decisions to fund grants for 
ranked grants (a practice known as 
evaluating applications for grants for  less favorably ranked 
funding out of rank order) on a case-
extramural research, and the extent  applications, which can raise 
by-case basis.  The reasoning for 
to which ICs made funding decisions  questions about 
those funding decisions must also be 
that were not strictly limited to the  transparency, impartiality, 
documented. 
scores from the initial scientific  and fairness. 
review (i.e., the extent of funding 
 
grant applications out of rank order).  
What OIG Found 
NIHs peer review process is central 
to its upholding its values of 
For the six ICs we reviewed, NIH met its basic requirements for 
transparency, impartiality, and 
documenting first-level peer review.  It also followed up with applicants, 
fairness, among others.  Therfore, it 
as required, to resolve concerns about protections for human subjects 
is important for NIH to ensure that 
and animals.  However, ICs documentation to justify funding grant 
the process works as intended. 
applications out of rank order often appeared to fall short of the 
How OIG Did This Review  requirements in HHSs Grants Policy Administration Manual, and the 
We reviewed a representative  documentation of reasoning for those funding decisions was missing in 
sample of extramural grants funded  37 out of 109 grants in our sample.  As a result, NIH lacks insight into the 
by six ICs in fiscal year (FY) 2018.   reasoning for the ICs decisions that were not strictly limited to the scores 
For those grants, we assessed NIHs  from the initial scientific review.  A failure to document these 
compliance with selected aspects of  justifications reduces transparency and can raise questions about undue 
its peer review process.  We did so  influence. 
by reviewing summary statements, 
documentation of NIHs followup to  What OIG Recommends 
resolve peer reviewer concerns, and  Without better insight into where and why funding out of rank order is 
justifications for funding grants out  happening, the integrity of NIHs peer review process could come under 
of rank order.  We also reviewed the  question.  We recommend that NIH centrally track and monitor data on 
Department of Health and Human  funding out of rank order and update its policy and guidance to reflect 
Services (HHSs) grant policy, NIH  the latest HHS grants policy on justifying funding out of rank order.  NIH 
policies, and NIHs written responses  concurred with our recommendations.
to our questions. 
   
TABLE OF CONTENTS 
 
 
 
 
BACKGROUND  1 
      Methodology  7 
FINDINGS  10 
NIH met requirements that ensure transparency of first-level peer review of grants funded by  10 
select ICs in FY 2018 
NIH followed up, as required, on all but one application to resolvebefore funding grants 12 
concerns raised by first-level peer review about protections for human subjects and animals 
ICs varied widely in the extent to which they funded grant applications out of rank order for  12 
the grants in our sample 
ICs did not appear to justify funding decisions in accord with HHS policy for a third of the  13 
grants in our sample that were funded out of rank order 
NIH has limited insight into the extent and nature of funding out of rank order  16 
RECOMMENDATIONS  17 
NIH should centrally capture and monitor data on ICs funding of grant applications out of  17 
rank order 
NIH should update its policy and guidance to reflect the latest HHS grants policy on justifying  18 
funding out of rank order 
AGENCY COMMENTS AND OIG RESPONSE  19 
APPENDICES  20 
A.  Statistical Supplement  20 
B.  Agency Comments  22 
ACKNOWLEDGMENTS AND CONTACT  25 
ABOUT THE OFFICE OF INSPECTOR GENERAL  26 
ENDNOTES  27 
   
BACKGROUND 
 
 
Objectives 
1.  To determine the extent to which the National Institutes of Healths (NIH) 
Center for Scientific Review (CSR) and select Institutes and Centers (ICs) met 
NIHs requirements for documenting first-level peer review of applications 
for grants for extramural research funded in fiscal year (FY) 2018. 
 
2.  To determine the extent to which select ICs used their discretion by funding 
grant applications out of rank order for extramural research in FY 2018.  
 
For FY 2019, the U.S. Department of Health and Human Services (HHS) Office of 
Inspector General (OIG) received $5 million in congressional appropriations to 
conduct oversight of NIH grant programs and operations.1  As indicated by the 
conference report that accompanied the legislation, the conferees direct[ed] the OIG 
to examine NIHs oversight of its grantees compliance with NIH policies.2 
This review is part of a larger body of HHS OIG work focused on oversight of NIH 
grant programs and operations.  Our work will review (1) intellectual property and 
cybersecurity protections; (2) compliance with Federal requirements and NIH policies 
for grants and contracts; and (3) integrity of grant application and selection processes.  
As part of this oversight work, Congress directed that OIG examine (1) NIHs efforts to 
ensure the integrity of its grant evaluation and selection processes and (2) the 
effectiveness of NIHs and grantee institutions efforts to protect intellectual property 
derived from NIH-supported research. 
In his August 2018 statement on protecting the integrity of U.S. biomedical research, 
Dr. Francis Collins, the Director of NIH, stated that the risks to the security of 
intellectual property and the integrity of peer review were increasing.  Dr. Collins 
further stated that in response to the increasing risks, NIH would work to identify 
robust methods to protect the integrity of peer review, among other steps it would 
take.3 
This review examines NIHs peer review process, which is central to the integrity of 
evaluating and selecting grants.  NIH intends for the peer review process to be fair, 
equitable, timely, and conducted in a manner that strives to eliminate bias.4 
   
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Background | 1  NIH's Organization and Mission 
NIH consists of 27 ICs, most of which are focused on specialty areas (e.g., the National 
Cancer Institute and the National Heart, Lung, and Blood Institute).5  NIH's mission is 
"to seek fundamental knowledge about the nature and behavior of living systems and 
the application of that knowledge to enhance health, lengthen life, and reduce illness 
and disability."6  NIH's ICs accomplish this mission, in large part, by annually funding 
over $30 billion in extramural research through a grantmaking process.7  CSR does 
not fund its own grants.  Rather, it provides support to the other ICs through peer 
review and processing of incoming grant applications. 
NIH Peer Review Process 
In NIHs peer review process, experts evaluate grant applications for scientific and 
technical merit, among other requirements.  NIHs core values for peer review drive it 
to seek the highest level of scientific and ethical standards.  Those core values include 
expert assessment, transparency, impartiality, fairness, confidentiality, integrity, and 
efficiency.8  
Grant applications undergo two levels of review: an initial scientific review and 
a review by an advisory council.  All NIH research applications undergo this two-level 
peer review process to ensure that applications are reviewed to the greatest extent 
possible in a fair, unbiased fashion.9 
An IC may choose to manage the first level of the peer review process itself or use 
CSR to manage it.10  NIH annually receives about 80,000 grant applications for 
research funding across all ICs.11  CSR conducts the first level of peer review on about 
75 percent of these applications; the ICs conduct the first level of peer review on the 
remaining 25 percent.12   
NIH requires both CSR and ICs to follow NIH's peer review policies and procedures for 
evaluating applications for research grants.13  Peer reviewers are mostly non-Federal 
scientists who have the necessary expertise to evaluate the scientific merit of the 
grant applications.  NIH verifies peer reviewers scientific expertise using multiple 
sources, such as their publication and grant histories.  Reviewers may meet a few 
times over the course of several years or just one time on an ad-hoc panel to review 
grant applications on a highly specialized research topic.14  
First Level of Peer Review 
During the first level of peer review, or initial scientific review, peer reviewers at CSR 
or the funding IC evaluate a package of grant applications that NIH has received in 
response to a funding opportunity announcement.  To conduct this stage of the 
review, peer reviewers engaged by CSR or the funding IC evaluate each application on 
the basis of the criteria included in the funding opportunity announcement.  The 
funding opportunity announcement describes NIHs intent to fund discretionary 
grants or cooperative agreements that address a specified public health topic or 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Background | 2  scientific area.  The funding opportunity announcement specifies the scored criteria 
and additional criteria that peer reviewers will use to assess grant applications.15  A 
Scientific Review Officer, who is an NIH scientist, oversees the initial scientific review.16 
Scored Criteria.  Each peer reviewer assigns a score to each scored criterion.  Unless 
otherwise specified in the funding opportunity announcement, NIHs five scored 
criteria for research projects are significance, investigators, innovation, approach, and 
environment.  Reviewers then consider their criteria scores to assign an overall impact 
score to the grant application that reflects their assessment of the likelihood for the 
project to exert a sustained, powerful influence on the research field(s) involved.17  
(See Exhibit 1 below for a description of NIHs five scored criteria.) 
Exhibit 1: Peer reviewers generally assess five scored criteria to help 
determine the scientific merit of the research proposed in a grant 
application. 
 
Source: NIH notice number NOT-OD-09-025
 
Additional Review Criteria and Considerations.  Reviewers assess, but do not score, 
other factors as well, depending on the requirements in the funding opportunity 
announcement and the type of research being proposed in an application.18  
Additional review criteria factor into the overall score but do not receive individual 
criterion scores, and additional review considerations do not receive criterion scores 
and do not factor into the overall score. (See Exhibit 2 for the additional criteria and 
considerations.)   
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Background | 3  Exhibit 2.  Peer reviewers assess two types of additional criteria, called 
Additional Review Criteria and Additional Review Considerations, as 
appropriate for the type of research proposed in a grant application. 
Additional Review Criteria   
  Protection for Human Subjects    Biohazards 
  Inclusion of Women, Minorities,    Vertebrate Animals 
and Children  (Including Animal Welfare) 
  Resubmission Applications    Revision Applications 
  Renewal Applications    Study Timeline (for Clinical Trials) 
Additional Review Considerations 
  Budget and Period Support    Research Using Select Agents  
  Applications From Foreign    Resource-Sharing Plans 
Organizations 
  Authentication of Key Biological   
and/or Chemical Resources 
Sources: NIH, NIH Grants Policy Statement, Section 2.4, October 2017 revision. 
Bars to Funding.  Of the additional criteria, those for the protection of human 
subjects; inclusion of women, minorities, and children; and vertebrate animals can 
prevent a grant application from being funded if the application does not 
satisfactorily address them.19  NIH sometimes refers to these as being a bar to 
funding a grant.20  When peer reviewers identify concerns with these criteria, NIH 
follows up with grant applicants.  Grant applicants must then provide additional 
information or adjust their research plans before NIH will agree to fund their grants.21 
NIH may follow up on the other additional criteria when reviewers flag information 
addressing them as unacceptable.  NIH policy instructs reviewers to consider 
Additional Review Criteria when assigning an overall impact score to grant 
applications.22  For Additional Review Considerations, reviewers comments serve as 
administrative information that ICs consider when they make funding decisions. 
Summary Statements.  After first-level peer review is complete, the Scientific Review 
Officer produces a summary statement for each grant application in the package of 
applications from that review.  The summary statement contains the overall impact 
score for the grant application.  This score is derived from an average of the peer 
reviewers impact scores (and the applications percentile ranking among the grant 
applications in the package, if applicable) and individual critiques from the peer 
reviewers that address the scored criteria and additional criteria, as appropriate.  The 
summary statement also includes a resume, or summary, of the peer reviewers 
discussion (when applicable), a summary of reviewers comments by the Scientific 
Review Officer, and a roster of the reviewers who reviewed the application.23, 24  After 
the review, NIH makes the summary statement available to the grant applicant.25   
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Background | 4  IC Review and Funding Order 
On receiving a package of grant applications that have been through initial scientific 
review, officials at the funding IC review the package and make recommendations to 
the Director of the IC.  Officials rank the applications by impact score or percentile 
and compare them to their ICs programmatic needs and funding plan.  Generally, 
grant applications with rankings that fall below the funding ICs cutoff will not be 
funded but NIHs peer review process allows for discretion.  A designated official at 
the IC may recommend to the ICs director that an application that is less favorably 
ranked for funding consideration move up, or that a more favorably ranked 
application move down.  If the IC ultimately approves a less favorably ranked 
application for funding, this is referred to as "funding out of rank order."  When an IC 
decides to fund a grant application out of rank order, HHS and NIH policy requires 
the IC to write a justification for doing so.  HHS policy requires the justification to tie 
to factors documented in the funding opportunity announcement.26, 27  (See Exhibit 3 
for an illustration of funding out of rank order.)  After IC review, the entire package of 
grant applications moves on the second level of peer review. 
Exhibit 3: ICs have the discretion to fund grant applications that are less 
favorably ranked than competing applications. 
 
Source: OIG analysis of NIH policy. 
   
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Background | 5  Second Level of Peer Review 
The second level of peer review, or advisory council review, takes place at the funding 
IC.  Each IC has an advisory council that conducts the review.28  This council includes 
scientists from the extramural research community as well as public representatives.  
The advisory council examines applications and considers the overall impact scores 
that the applications received during the initial peer review process, including any 
recommendations to fund out of rank order, percentile rankings (if applicable), and 
the summary statements in light of the IC's priorities.  The advisory council advises the 
IC Director on funding decisions.  The IC Director makes the final funding decision on 
the basis of staff and council advice, and of the outcome of initial peer review.29 
Previous and Concurrent OIG Work  
In response to a congressional request, OIG examined the extent to which NIH's 
National Institute of Environmental Health Sciences (NIEHS) followed its peer review 
processes for funding research on bisphenol-A (BPA).  In the resulting 2017 report, 
OIG found that although the NIEHS met the peer review process requirements for all 
grants, NIEHS used its discretion to fund applications with less favorable scores than 
competing applications for 14 percent of BPA grants, versus for 4 percent of other, 
non-BPA grants.  OIG concluded that although NIEHS had discretion to fund grant 
applications with less favorable impact scores ahead of competing applications, 
applying its discretion frequently or disproportionately in one research area may 
create an appearance of impropriety.30  OIG made no recommendations for NIH in 
this report. 
In addition to this review, OIG has a body of work completed or underway in response 
to the Departments of Defense and Labor, Health and Human Services, and Education 
Appropriations Act, 2019 and Continuing Appropriations Act 2019.  Completed 
evaluations include reviews of NIH's vetting of prospective peer reviewers; NIHs 
monitoring of peer reviewers handling of confidential information; and NIHs 
monitoring of extramural researchers' financial conflicts of interests.31, 32, 33  
Completed audits include reviews of NIHs controls to prevent duplication among NIH 
research grants and NIH's implementation of regulations regarding financial conflicts 
of interest, among others.34, 35  This body of work resulted in recommendations calling 
for NIH to update its policies and training and to strengthen its approach to vetting 
and overseeing peer reviewers to identify foreign threats.  It also called for NIH to do 
more to address financial conflicts of interest, including conducting periodic quality 
assurance to ensure the adequacy of its oversight.  NIH concurred with all of these 
recommendations, and it has implemented three of them.  NIH has been taking steps 
to implement the remaining recommendations, including working closely with 
grantee institutions, Federal law enforcement, and the White Houses Office of 
Science and Technology Policy to help identify and mitigate foreign threats to 
research integrity.  According to NIH, these steps have led to the removal of scientists 
as NIH grantees and peer reviewers.  
   
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Background | 6  Methodology 
Scope 
We based this inspection on a review of documentation for a representative sample 
of grants funded by six ICs in FY 2018.  For those grants, we assessed NIHs 
compliance with selected aspects of its peer review process.  We did so by reviewing 
summary statements, documentation of NIHs followup to resolve bars to funding, 
and justifications for funding grants out of rank order.  We also based this inspection 
on a review of NIH policies and HHS grants policy in effect in FY 2018 as well as NIHs 
written responses to questions we sent during the inspection.  Our review of bars to 
funding did not include NIHs criterion on inclusion across the lifespan, as that took 
effect in January 2019. 
Sample Selection 
We used NIHs ExPORTER grants database to identify all new awards that NIH made in 
FY 2018 for extramural grants supporting basic and cooperative research.36  This 
resulted in a list of 9,714 grants that were funded by 27 ICs.  We divided the ICs into 
groups of small, medium, and large ICs based on the number of grants each funded.37  
To lessen the burden on NIH given numerous ongoing OIG reviews of ICs at the time, 
we purposively selected two ICs from each of these three groups.  (See Exhibit 4 
below for the ICs we selected.) 
Exhibit 4: The six ICs we selected for this review 
Large ICs   
National Institute of Neurological Disorders and Stroke (NINDS) 
National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) 
Medium ICs 
National Institute on Drug Abuse (NIDA) 
National Human Genomic Research Institute (NHGRI) 
Small ICs 
National Institute on Mental Health and Development (NIMHD) 
National Center for Advancing Translational Sciences (NCATS) 
 
We grouped grants into seven strata based on the entity that conducted the first level 
of peer review:  either CSR or one of the six selected ICs.  We selected a stratified 
random sample from the seven strata.  The first stratum consisted of all grants for 
which CSR performed the first level of peer review.  Strata 2 through 7 consisted of 
grants that were reviewed by each of the six selected ICs.  This sampling approach 
allowed us to generalize and compare our findings regarding the first level of peer 
review and the overall extent to which ICs funded grants out of rank order.  However, 
our sampling approach does not allow us to make generalizations about the subset of 
grants that had bars to funding or were funded out of rank order.  For these subsets 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Background | 7  of findings, we have presented sample counts.  (See Exhibit 5 below for details on the 
population and sample size for each of our strata.) 
Exhibit 5: Breakdown of population and sample size for seven strata 
Entity that   
Conducted First  Grant  Sample 
Stratum  Level of Review  Population  Size 
1  CSR  1184  150 
2  NCATS  46  36 
3  NIMHD  54  42 
4  NHGRI  56  42 
5  NIDA  176  82 
6  NIDDK  161  82 
7  NINDS  351  106 
Total    2028  540 
 
Document Review 
Summary Statements.  For each of the grants in our sample, we requested the related 
summary statement that documents the first level of peer review.38  We abstracted 
data from the summary statements into a database.  These data include elements 
about the peer reviewers assessment of scored criteria and additional criteria; 
a summary of reviewers comments by the Scientific Review Officer who oversaw the 
peer review; and whether the summary statement included a roster of peer reviewers.  
Followup Documentation on Bars to Funding.  We reviewed summary statements for 
instances documenting that peer reviewers raised concerns on criteria that were a bar 
to funding a grant.  Those criteria were protection for human subjects; inclusion of 
women, minorities, and children; and vertebrate animals (including animal welfare).  
When we identified those instances of concern, we asked NIH for documentation 
showing that it resolved the concerns.  For these grants, NIH sent us copies of email 
correspondence, worksheets, and other documents that we reviewed for evidence 
that NIH followed up with each grant applicant and approved the additional 
information or changes that the applicant made in response.  
Justifications.  For any of our sampled grants that ICs funded out of rank order, we 
also requested the written justifications for doing so.  We examined the text of these 
justifications for the reasoning for funding grants out of rank order.  We grouped the 
justifications into three categories: (1) those that included a reason and an argument 
for funding out of rank order that spoke to the merits of the proposed research or 
applicant; (2) those that provided a reason, such as IC priority area, but did not 
include an argument related to the specifics of the proposed research or applicant; 
and (3) those that provided no reason for funding the grant out of rank order.  
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Background | 8  NIH Policy and Written Responses.  We reviewed NIH policy on its peer review and 
funding processes that were in effect in FY 2018, as well as and written clarifications 
that NIH provided on its policy or specific grants in our sample.  We also reviewed the 
relevant HHS policy for documenting decisions to fund grants out of rank order. 
Limitations 
We did not independently verify the information on summary statements.  In our 
review of additional criteria, we did not determine whether peer reviewers assessed 
the appropriate criteria for each grant application.  In our review of justifications for 
funding grants out of rank order, we did not assess the validity of the scientific 
explanations that ICs cited; rather, we checked for language that offered reasoning for 
the funding decision. 
Standards 
We conducted this study in accordance with the Quality Standards for Inspection and 
Evaluation issued by the Council of the Inspectors General on Integrity and Efficiency.  
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Background | 9  FINDINGS 
 
 
NIH met requirements that ensure transparency of 
first-level peer review of grants funded by select ICs in 
FY 2018 
We found that NIH met requirements for documenting the first level of peer review, 
also known as initial scientific review.  These basic requirements are key to ensuring 
that the review reflects NIHs core values.  For each grant application, NIH requires the 
Scientific Review Officer who oversaw the peer review to create a summary statement 
that documents the review.  The summary statement must include complete sets of 
critiques from peer reviewers that address scored criteria and additional review 
criteria.  It can also include a roster of the peer reviewers who evaluated the grant 
application.  Overall, we found little variation in compliance with the requirements 
among the six ICs and CSR. 
NIH created the required summary statements for 100 percent of 
grants  
Summary statements are important because they serve as NIHs official record of the 
first level of peer review across all of the applications, ensuring an equivalent process.  
An equivalent process upholds NIHs core value of fairness.39 
All of the grant applications in our review had a summary statement.  The critiques, 
scores, rosters, and other information in summary statements document the review 
process that each grant application underwent.  Because NIH shares summary 
statements with grant applicants, summary statements support NIHs core value of 
transparency.   
Summary statements for 99 percent of grants had complete sets 
of scores from peer reviewers, as required 
Scored criteria reflect the peer reviewers expert assessments of an applications 
scientific and technical merit.  Expert assessments are another NIH value it strives to 
uphold.  Those scored criteria are a consideration in an applications overall impact 
score, which determines its ranking and likelihood of being funded by NIH.  NIHs five 
scored criteria for research projects are: significance, investigators, innovation, 
approach, and environment.  Summary statements for 99 percent of grant 
applications included critiques from each peer reviewer with complete sets of scores 
for the five criteria.40  In those few cases where the summary statement did not have 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Findings | 10  complete sets of scores, it was because a reviewers critique failed to address a single 
scored criterion or a reviewers critique omitted scores for all five criteria. 
Summary statements for 100 percent of grants had a roster of 
peer reviewers 
Rosters provide transparency with respect to the identities and institutional affiliations 
of the scientists who reviewed an application.  This enables applicants and others to 
verify that the membership of a review group was not overly weighted toward a single 
type of scientific discipline or research institution.  
Summary statements for 99 percent of grants addressed 
additional criteria and considerationsmost often the project 
budget, but sometimes potentially sensitive issues, such as the 
use of biological agents or involvement of foreign entities  
NIHs 14 additional criteria and considerations provide opportunity for peer reviewers 
to address factors relevant to the specific research proposed within a grant 
application.  This ability for peer reviewers to tailor their reviews is important given 
the breadth of research that NIH sponsors across its 27 ICs.  Indeed, our review of 
summary statements showed that peer reviewers assessed additional criteria and 
considerations for nearly all grant applications.  For 22 percent of grants, peer 
reviewers or the Scientific Review Officer found an additional criterion or 
consideration to be unacceptable or in need of revision; the two found most often 
were protections for human subjects and budget and period of support.41  See 
Appendix A for data on the percentage of grant applications for which reviewers 
assessed each additional criterion and consideration. 
In addressing the additional criteria and considerations, reviewers occasionally 
commented on potentially sensitive topics.  For example, they commented on the 
proposed use of certain biological agents or toxins (also known as select agents) in 
17 percent of grants.  Such agents require special handling and can pose a threat to 
the publics health; one example is the Ebola virus. 42  For 6 percent of grants, the 
reviewers commented on the involvement of foreign entities.  For applications 
submitted by foreign entities, NIH instructs reviewers to assess whether the use of 
foreign talent and other resources not readily available in the United States presents 
special opportunity to further research.43, 44  NIH has elevated its scrutiny of grantees 
and its own processes since identifying risks associated with foreign influences on the 
integrity of U.S. biomedical researchin fact, in 2018 NIH added security as one of its 
core values.45 
 
 
 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Findings | 11  NIH followed up, as required, on all but one application to 
resolvebefore funding grantsconcerns raised by first-level 
peer review about protections for human subjects and animals 
NIH prohibits funding applications that peer reviewers identify as not adequately 
addressing human subjects protections; inclusion of women, minorities, and children 
in research; and animal welfare.  When applicants did not adequately address these 
criteria in research, NIH flagged their applications for followup.  Peer reviewers 
identified concerns with these particular criteria in the applications for 51 of the 
grants in our sample.  NIH followed up on all but one application for additional 
information to address concerns raised during the initial scientific review.  In response, 
the applicants responded via email with information that NIH staff reviewed and 
approved before allowing the grants to be funded.  The one application on which NIH 
did not follow up was used by NIH as the basis to fund four of the grants in our 
Example of Followup on Concerns About Use of Vertebrate Animals 
Peer reviewers found that a grant applicant did not address all required 
criteria for the use of vertebrate animals.  The applicant submitted additional 
information to NIH detailing the experimental procedure to be used; why 
the study needed to use animals; how the study team would minimize pain 
and discomfort for the animals; and how the animals would be euthanized.  
sample.  According to NIH, after researching the application at our request, it 
determined that the flag was unnecessary.  In this case, the flag concerned excluding 
children without scientific justification, but NIH ultimately determined that children 
were appropriately excluded from only certain aspects of the study. 
ICs varied widely in the extent to which they funded grant 
applications out of rank order for the grants in our sample 
NIHs grantmaking process provides ICs with discretion to fund a grant application 
that was ranked less favorably by the first level of peer review over an application that 
was ranked more favorably.  Funding out of rank order allows the ICs to fund grants 
that align with their research priorities or that address emerging threats to public 
health, for example.  Overall, ICs funded 109 of the 540 grants in our sample out of 
rank order.  The second largest IC in our sample, NIDDK, funded 67 of its 130 sampled 
grants out of rank order.  By contrast, the largest IC in our sample, NINDS, funded 8 of 
its 184 sampled grants out of rank order.  (See Exhibit 6 for counts of sampled grants 
funded in and out of rank order by each IC.) 
   
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Findings | 12  Exhibit 6: ICs varied in the extent to which they funded sampled grants out of 
rank order. 
  200
180
 
160
140
  s
t
n
a120
r
G
  d 100
e
pl
m 80
  a
S 60
40
 
20
  0
NINDS NIDDK NIDA NIMHD NHGRI NCATS
Grants Funded in Rank Order Grants Funded Out of Rank Order
 
ICs did not appear to justify funding decisions in accord with 
HHS policy for a third of the grants in our sample that were 
funded out of rank order 
Documentation that ICs created to justify funding grants out of rank order often 
appeared to fall short of the requirements in HHSs Grants Policy Administration 
Manual for documenting funding out of rank order.  Those requirements call for 
approving officials to provide a statement of the specific reasons that influenced their 
judgment, including a justification for funding of the application that ties to factors 
documented in the funding opportunity announcement.46  However, in 
documentation for 37 of the 109 grants funded out of rank order, ICs offered little or 
no insight into the reasoning for their decisions that were not strictly limited to the 
scores from the initial scientific review.  For 22 of these grants, ICs offered a brief 
statement with little reasoning.  For 15 grants, ICs offered no reasoning. 
Examples of Justifications That Provide Little Reasoning 
High Program Priority.  Opioid dependence. 
Preclinical application that conducts stability tests and vaccine potency tests. 
This grant is part of the Diversity R21 program. 
 
Justifications That Provide No Reasoning 
Paid out of rank order by skipping other fundable applications. 
Applications are in response to an RFA [request for applications]; funding plan 
established in accordance with RFA. 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Findings | 13   
In contrast, documentation for 72 of the 109 grants funded out of rank order appears 
more in line with HHSs grants policy, referencing the strengths and promise of the 
research proposed in the grant applications.  ICs noted the track record of the 
research team, the scientific contribution the research would make, or other merits.47  
This provided insight into ICs reasoning behind their decision to fund grants out of 
rank order. 
Excerpts From Justifications That Provide Stronger Reasoning 
This R21 project is innovative, potentially high impact, investigating . . . techniques to 
detect and predict response to therapy of . . . disease.  Developing . . . techniques to track 
disease severity in these patients could be an essential adjunct to clinical practice . . . . 
Peer reviewers rated the proposal as innovative, feasible and highly consistent with the 
goals of the . . . initiative and noted that the research team is exceptionally strong, 
productive and well-chosen for the proposed research.  Furthermore, the project offers 
geographic and investigator diversity that will enhance the overall community of 
researchers funded under this FOA [funding opportunity announcement]. 
This proposal directly addresses the diagnosis and prognosis of concussion in . . .  
populations, which is a planning priority outlined [by NIH] and an area of recent 
Congressional inquiry.  This proposal is from an outstanding multidisciplinary team of 
investigators at . . . with strong complementary expertise in clinical . . . .  Review deemed 
this project highly rigorous and innovative . . . . 
 
We note that ICs varied in how robustly they justified their decisions to fund less-
favorably ranked grants in our sample.  Three of the six ICs in our review provided 
reasoning for all of their decisions, while one ICNIMHDprovided no justifications 
that explained its decisions for funding out of rank order.  NIDDK, which funded the 
most grants in our sample out of rank order, provided stronger reasoning in 
three-quarters of the justifications it wrote.  (See Exhibit 7 below for data on the 
breakdown of ICs justifications by strength of reasoning.) 
   
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Findings | 14  Exhibit 7: ICs varied in how well they documented their reasons for funding 
grants out of rank order. 
80
r
e
d
r
O
k 
n
a60
R
f 
o
t 
u
O
d 
e40
d
n
u
F
s 
t
n
ra20
G
d 
e
pl
m
Sa 0
NIDDK NIDA NIMHD NHGRI NCATS
Reasoning for Funding Out of Rank Order
Stronger Little None
 
Finally, the procedure for justifying funding grants out of rank order in NIHs Policy 
Manual does not incorporate current HHS policy on justifying decisions to fund grants 
out of rank order that requires agencies to explain the reasoning behind their 
decisions.   
   
NIHs policy manual does not cite the latest HHS grants policy, which created 
stronger requirements for documenting decisions to fund out of rank order 
NIHs Policy Manual 4204-204c cites HHS retired Grants Policy Directive, which stated, 
Should the official approve an application for funding out of rank order, the reason(s) 
for doing so must be documented in writing. 
In 2015, HHSs Grants Policy Administration Manual (GPAM) superseded the Grants 
Policy Directive.  When agencies fund out of rank order, the GPAM requires 
a statement of the specific reasons for the difference that influenced the judgment of 
the approving official.  This must include a justification for funding of the particular 
application and tie to factors documented in the FOA [funding opportunity 
announcement]. 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Findings | 15  NIH has limited insight into the extent and nature of funding 
out of rank order 
Monitoring how often ICs exercise this discretion and requiring justification for doing 
so can serve as compensating controls to help NIH ensure that ICs funding decisions 
uphold NIHs values and are sound and free from undue influence.  However, our 
review raises concerns about these tools. 
NIH lacks centralized, readily available data on how often ICs 
fund grants that are less favorably ranked 
NIH cannot easily determine the extent to which ICs fund grant applications out of 
rank order.  Data on funding out of rank order reside only with the ICs that made the 
funding decisions.  To respond to our data request for this review, NIH relied on the 
individual ICs to identify grants that they funded out of rank order and provide their 
justifications.  According to NIH, this task was a time-consuming, manual process that 
occurred at each of the six ICs we reviewed.  It is understandable that the funding ICs 
have the source data on their decisions to fund less-favorably ranked grants.  
However, NIHs lack of insight into the extent of the practice across ICs raises 
questions about its ability to monitor and oversee ICs funding decisions. 
Justifications that fail to provide reasoning for funding decisions 
limit NIHs ability to understand the nature of funding out of 
rank order 
ICs fund grants out of rank order for a number of reasons including, among others, to 
support their research priorities, respond to emerging threats to public health, and 
support NIH-wide initiatives.  However, funding out of rank order could also signal 
potential problems with NIHs peer review process, such as concerns with the initial 
scientific review or indications of undue influence.  We found that in 37 of 
109 instances of funding out of order, no reasoning was documented.  That means 
that NIH lacks insight into whether the ICs are funding out of rank order to further 
NIHs mission or because of problems that warrant NIHs attention. 
 
 
 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Findings | 16   
CONCLUSION AND RECOMMENDATIONS 
 
 
The integrity of NIHs peer review process is critical to achieving NIHs research 
mission.  In recent years, the integrity of the process has come under increased 
scrutiny.  Prior Office of Inspector General reviews highlighted the importance of 
vetting peer reviewer nominees and overseeing reviewers handling of confidential 
information.  This review adds to that body of work by focusing on the peer review 
process itself. 
Our review found that NIH followed processes that are key to ensuring that peer 
review is in accordance with NIHs core values.  For grants funded by the six ICs we 
reviewed, we found that NIH created summary statements that documented the 
requirements of first-level peer review.  We found that NIH followed up, as required, 
on serious concerns thatif left unresolvedwould bar it from funding grants.  Taken 
together, these practices help ensure that applications submitted for funding are 
evaluated by scientific experts in a manner free from inappropriate influences, which 
is the intent of peer review.  
However, our review also revealed a need for improved transparency.  In funding 
grants, NIH allows forand benefits fromthe discretion to make funding decisions 
that are not strictly limited to the scores provided during the first-level peer review.  
This, for example, enables ICs to respond to emerging threats to public health.  Such 
discretion calls for controls to ensure that ICs exercise it appropriately.  Yet we found  
NIH lacks ready access to information about which grants ICs funded out of rank 
order, and that ICs did not always provide clear justifications for doing so, as required 
by HHS grants policy.  More transparency around funding out of rank order 
represents an important control to help NIH identify and track outliers, and to ensure 
that its funding processes are in fact upholding its core values. 
Therefore, we recommend that NIH: 
Centrally capture and monitor data on ICs funding of grant 
applications out of rank order 
 
NIH needs better and more readily available information from the ICs to detect, 
analyze, and understand the reasons for funding out of rank order.  NIH already has a 
tool, the eRA Pay Plan Module, that allows ICs to flag grant applications that they 
fund out of rank order; however, NIH does not require ICs to use it.  NIH should 
require ICs to use the Pay Plan Module or another tool to centrally capture data on 
the extent and nature of funding out of rank order.  NIH should use these data to 
monitor funding out of rank order across ICs and, where it finds outliers, ask followup 
questions to ensure that its peer review process is working as intended.  Although ICs 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140                Conclusion and Recommendations | 17  have the discretion to fund less favorably ranked grant applications, doing so 
frequently could be a sign of systemic problems in first-level peer review or undue 
influence in funding decisions.   
 
Update its policy and guidance to reflect the latest HHS grants 
policy on justifying funding out of rank order 
 
Requiring justifications that explain the reasoning for funding less favorably ranked 
grants reinforces ICs accountability for their funding decisions.  Robust justifications 
for funding less-favorably ranked grant applications provide transparency into ICs 
funding decisions and build trust that NIHs grantmaking is fair, equitable, and free 
from bias.  However, NIHs policy does not provide guidance on what information to 
include in the written justification.  NIHs policy also has not been revised since the 
publication of HHSs latest grant policy and omits HHS requirements with respect to 
the content of ICs justifications.  In particular, HHSs grants policy requires a 
statement of the specific reasons that influenced the judgment of the approving 
official, including a justification for funding the application that ties to factors 
documented in the FOA. 
NIH should update its grants policy to reflect HHSs latest grants policy and thereby 
define minimum expectations in policy to ensure that ICs provide a clear and 
compelling rationale for funding out of rank order.  As such, NIH should also update 
its guidance to instruct ICs to identify the strengths of the grant application that 
warrant funding an application out of the rank order.  It should also provide examples 
that illustrate acceptable justifications. 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140                Conclusion and Recommendations | 18  AGENCY COMMENTS AND OIG RESPONSE
 
 
 
NIH concurred with our recommendations.  Regarding the first recommendation, NIH 
stated that it will develop structured data for identifying, tracking, and providing 
compliance oversight on grants awarded out of rank order, with implementation 
planned for fiscal year 2021.  NIH also stated it will conduct a comprehensive review 
in the same year, and implement further internal controls, guidance, and compliance 
testing as needed based on the results of the review. 
Regarding the second recommendation, NIH stated that it will update its internal 
policy and staff guidance to reflect the requirement of the HHS Grants Policy 
Administration Manual to tie funding justifications to factors documented in the 
Funding Opportunity Announcement.  NIH stated it will take this step in fiscal 
year 2021.  
We thank NIH for the actions it plans to take to address our recommendations and 
for its commitment to increasing its oversight of funding decisions made out of rank 
order.  For the full text of NIHs comments, see Appendix B. 
MMENTS AND OIG RESPONSE 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140           Agency Comments and OIG Response | 19   
APPENDIX A  Statistical Supplement 
 
Point Estimates and Confidence Intervals 
 
Grants including summary statements 
 
Sample  Point  95% confidence 
  size  estimate  interval 
NIH provided summary statement  540  100%  98.9-100% 
for grant 
Summary statements with complete sets of scores 
 
Sample  Point  95% confidence 
  size  estimate  interval 
Summary statements with complete   540  99%  99.4-99.9% 
sets of scores from peer reviewers 
Summary statements with roster of peer reviewers 
 
Sample  Point  95% confidence 
  size  estimate  interval 
Summary statements with rosters of  540  100%  98.9-100% 
peer reviewers 
Additional criterion with comment by at least one reviewer 
Sample  Point  95% confidence 
  size  estimate  interval 
Budget and Period of Support  540  99%  99.3-99.9% 
Resource Sharing Plans  540  92%  89.8-94.6% 
Authentication of Key Biological   540  74%  70.5-77.9% 
and/or Chemical Resources 
Vertebrate Animals  540  58%  53.9-62.7% 
Biohazards  540  47%  41.9-51.3% 
Protection of Human Subjects  540  44%  39.3-48.3% 
Inclusion of Women  540  40%  35.5-44.3% 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Appendix A | 20  Additional criterion with comment by at least one reviewer 
Sample  Point  95% confidence 
size  estimate  interval 
Inclusion of Minorities  540  40%  35.5-44.3% 
Inclusion of Children  540  39%  34.6-43.3% 
Resubmission Applications  540  38%  33.3-42.5% 
Select Agent Research  540  17%  13.0-20.1% 
Applications From Foreign Organizations  540  6%  3.8-8.0% 
Study Timeline  540  2%  1.0-2.7% 
Renewals  540  <1%  <0.1-0.3% 
Peer reviewers identified concerns that could result in bars to funding 
(i.e., protection of human subjects; inclusion of women, minorities, and 
children; and vertebrate animals) 
Sample  Point  95% confidence 
size  estimate  interval 
Peer reviewers identified concerns that  540  10%  7.1-13.5% 
could result in bars to funding 
Source: OIG analysis of NIH grants data from six ICs, 2020. 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Appendix A | 21  APPENDIX B: Agency Comments 
DATE:  May 18, 2021  
TO:    Suzanne Murrin 
Deputy Inspector General for Inspection and Evaluations, HHS 
FROM:  Principal Deputy Director, National Institutes of Health 
SUBJECT:  NIH Comments on Draft Report, Selected NIH Institutes Met 
Requirements for Documenting Peer Review But Could Do More To 
Track and Explain Funding Decisions (OEI-01-19-00140) 
Attached are the National Institutes of Healths (NIH) comments on the Office of 
Inspector General (OIG) draft report, "Selected NIH Institutes Met Requirements for 
Documenting Peer Review But Could Do More To Track and Explain Funding 
Decisions" (OEI-01-19-00140). 
NIH appreciates the review conducted by the OIG and the opportunity to provide 
clarifications on this draft report.  If you have questions or concerns, please contact 
Meredith Stein in the Office of Management Assessment at 301-402-8482. 
Lawrence A. Tabak, D.D.S., Ph.D. 
Attachment 
Selected NIH Institutes Met Requirements to Document Peer Review
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140         Appendix B | 22 Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Appendix B | 23  Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Appendix B | 24   
ACKNOWLEDGMENTS AND CONTACT 
 
Acknowledgments 
Ivan Troy served as the team leader for this study, and Matt Blackburn served as the 
lead analyst.  Others in the Office of Evaluation and Inspections who conducted the 
study include Chris Galvin.  Office of Evaluation and Inspections headquarters staff 
who provided support include Joe Chiarenzelli, Althea Hosein, and Christine Moritz.  
We would also like to acknowledge the contributions of other Office of Inspector 
General staff, including Jessica Swanstrom. 
This report was prepared under the direction of Joyce Greenleaf, Regional Inspector 
General for Evaluation and Inspections in the Boston regional office. 
Contact 
To obtain additional information concerning this report, contact the Office of Public 
Affairs at Public.Affairs@oig.hhs.gov.  OIG reports and other information can be found 
on the OIG website at oig.hhs.gov. 
Office of Inspector General 
U.S. Department of Health and Human Services 
330 Independence Avenue, SW 
Washington, DC 20201 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Acknowledgments and Contact | 25   
ABOUT THE OFFICE OF INSPECTOR GENERAL 
 
 
The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-
452, as amended, is to protect the integrity of the Department of Health and Human 
Services (HHS) programs, as well as the health and welfare of beneficiaries served by 
those programs.  This statutory mission is carried out through a nationwide network 
of audits, investigations, and inspections conducted by the following operating 
components: 
The Office of Audit Services (OAS) provides auditing services for HHS, 
either by conducting audits with its own audit resources or by overseeing audit work 
done by others.  Audits examine the performance of HHS programs and/or its 
grantees and contractors in carrying out their respective responsibilities and are 
intended to provide independent assessments of HHS programs and operations.  
These audits help reduce waste, abuse, and mismanagement and promote economy 
and efficiency throughout HHS. 
The Office of Evaluation and Inspections (OEI) conducts national 
evaluations to provide HHS, Congress, and the public with timely, useful, and reliable 
information on significant issues.  These evaluations focus on preventing fraud, waste, 
or abuse and promoting economy, efficiency, and effectiveness of departmental 
programs.  To promote impact, OEI reports also present practical recommendations 
for improving program operations. 
The Office of Investigations (OI) conducts criminal, civil, and administrative 
investigations of fraud and misconduct related to HHS programs, operations, and 
beneficiaries.  With investigators working in all 50 States and the District of Columbia, 
OI utilizes its resources by actively coordinating with the Department of Justice and 
other Federal, State, and local law enforcement authorities.  The investigative efforts 
of OI often lead to criminal convictions, administrative sanctions, and/or civil 
monetary penalties. 
The Office of Counsel to the Inspector General (OCIG) provides 
general legal services to OIG, rendering advice and opinions on HHS programs and 
operations and providing all legal support for OIGs internal operations.  OCIG 
represents OIG in all civil and administrative fraud and abuse cases involving HHS 
programs, including False Claims Act, program exclusion, and civil monetary penalty 
cases.  In connection with these cases, OCIG also negotiates and monitors corporate 
integrity agreements.  OCIG renders advisory opinions, issues compliance program 
guidance, publishes fraud alerts, and provides other guidance to the health care 
industry concerning the anti-kickback statute and other OIG enforcement authorities.
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  About OIG | 26  ENDNOTES 
 
 
 
1 Department of Defense and Labor, Health and Human Services, and Education Appropriations Act, 2019, and 
Continuing Appropriations Act, 2019, P.L. No. 115-245 (Sept. 28, 2018). 
2 Department of Defense and Labor, Health and Human Services, and Education Appropriations Act, 2019, and 
Continuing Appropriations Act, 2019, P.L. No. 115-245 (Sept. 28, 2018). 
3 NIH, Statement on Protecting the Integrity of U.S. Biomedical Research.  Accessed at https://www.nih.gov/about-
nih/who-we-are/nih-director/statements/statement-protecting-integrity-us-biomedical-research on February 22, 
2021. 
4 NIH, Peer Review.  Accessed at https://grants.nih.gov/grants/peer-review.htm on November 30, 2020. 
5 NIH, NIH Mission.  Accessed at https://www.nih.gov/about-nih/what-we-do/nih-almanac/nih-organization on 
November 30, 2020. 
6 NIH, About the NIH.  Accessed at https://www.nih.gov/about-nih/what-we-do/nih-almanac/about-nih on 
November 30, 2020. 
7 NIH, Budget.  Accessed at https://www.nih.gov/about-nih/what-we-do/budget on November 30, 2020. 
8 NIH Grant Policy Manual, ch. 4204-204B. pt. III. 
9 NIH Grant Policy Manual, ch. 4204-204B. pt. IV. 
10 NIH, NIH Grants Policy Statement, Section 2.4.1.1.  Accessed at https://archives.nih.gov/asites/grants/04-16-
2018/policy/nihgps/index.htm on November 30, 2020. 
11 NIH, NIH Peer Review: Grants and Cooperative Agreements, p. 2.  Accessed at 
https://grants.nih.gov/grants/peerreview22713webv2.pdf on November 30, 2020. 
12 NIH, About CSR.  Accessed at https://public.csr.nih.gov/AboutCSR on November 30, 2020. 
13 42 U.S.C.  284a and 42 CFR  52h.7. 
14 NIH, NOT-OD-10-089.  Accessed at https://grants.nih.gov/grants/guide/notice-files/NOT-OD-10-089.html on 
February 23, 2021. 
15 NIH, NIH Grants Policy Statement, Section 2.4.1.2.  Accessed at https://archives.nih.gov/asites/grants/04-16-
2018/policy/nihgps/index.htm on November 30, 2020. 
16 NIH, NIH Grants Policy Statement, Section 2.1.1.  Accessed at https://archives.nih.gov/asites/grants/04-16-
2018/policy/nihgps/index.htm on November 30, 2020. 
17 NIH, NIH Grants Policy Statement, Section 2.4.1.2.  Accessed at https://archives.nih.gov/asites/grants/04-16-
2018/policy/nihgps/index.htm on November 30, 2020. 
18 NIH, NIH Grants Policy Statement, Section 2.4.1.2.  Accessed at https://archives.nih.gov/asites/grants/04-16-
2018/policy/nihgps/index.htm on November 30, 2020. 
19 NIH, Bars to Grant AwardsResearch Animals SOP.  Accessed online at 
https://www.niaid.nih.gov/research/grants-bar-awards-research-animals on April 6, 2021.  NIH, Bars and 
Restrictions to Grant AwardsHuman Subjects SOP.  Accessed online at 
https://www.niaid.nih.gov/research/grants-bar-awards-human-subjects on April 6, 2021.  NIH, Inclusion of Women 
and Minorities as Participants in Research Involving Human Subjects.  Accessed online at  
https://grants.nih.gov/policy/inclusion/women-and-minorities.htm on April 6, 2021.  The three bars to funding 
listed were those in place during the time of our review. For grant applications due beginning in January 2019, 
NIH added inclusion across the lifespan to address the consideration of age as an inclusion variable in research 
involving human subjects. 
20 NIH, NIMH Summary StatementBars to Funding, retrieved at https://www.nimh.nih.gov/funding/grant-writing-
and-application-process/summary-statement.shtml on February 22, 2021. 
 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Endnotes | 27 
   
21 NIH, NIH Grants Policy Statement, Sections 4.1.1, 4.1.15, 4.1.15.8.  Accessed at 
https://archives.nih.gov/asites/grants/04-16-2018/policy/nihgps/index.htm on November 30, 2020. 
22 NIH, NIH Peer Review: Grants and Cooperative Achievements. 2019. Accessed online at 
https://grants.nih.gov/grants/PeerReview22713webv2.pdf on March 22, 2021.  
23 NIH Grant Policy Manual, ch. 4204-204B, pts. V, Section D(2) and IX, Section C(2).  
24 NIH also publishes all rosters of peer reviewers on its website. 
25 NIH, NIH Grants Policy Statement, Section 2.4.4.  Accessed at https://archives.nih.gov/asites/grants/04-16-
2018/policy/nihgps/index.htm on February 23, 2021. 
26 HHS Grant Policy Administration Manual, pt. G, ch. 1, Section b(56)(ii). 
27 NIH Grant Policy Manual, ch. 4204-204c. 
28 42 U.S.C.  284a. 
29 NIH, NIH Grants Policy Statement, Section 2.4.3.  Accessed at https://archives.nih.gov/asites/grants/04-16-
2018/policy/nihgps/index.htm on November 30, 2020. 
30 OIG, As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 
Discretion, OEI-01-15-00150, August 2017. 
31 OIG, Vetting Peer Reviewers at NIH's Center for Scientific Review: Strengths and Limitations, OEI-01-19-00160, 
September 2019. 
32 OIG, NIH Has Acted to Protect Confidential Information Handled by Peer Reviewers, But It Could Do More, OEI-05-
19-00240, March 2020. 
33 OIG, NIH Has Made Strides in Reviewing Financial Conflicts of Interest in Extramural Research, But Could Do More, 
OEI-03-19-00150, September 2019. 
34 OIG, The National Institutes of Health Has Controls to Mitigate the Risk That Grantees Receive Duplicate Grant 
Funding, A-02-19-02002, March 2020. 
35 OIG, The National Institutes of Health Has Limited Policies, Procedures, and Controls in Place for Helping To 
Ensure That Institutions Report All Sources of Research Support, Financial Interests, and Affiliations, A-03-19-03003, 
September 2019. 
36 The NIH activity codes we used to identify grants supporting basic and cooperating research were R01, R03, 
R21, R24, R33, R35, R56, R61, RF1, U01, U19, U24, U2C, U34, U41, U54, UA5, UF1, UG1, UG3, UH2, UH3, UL1, and 
UM1. 
37 We defined the groups based on the distribution of counts of grants that each IC funded in FY2018.  Large ICs 
had grant counts that were in the top third of ICs, medium ICs in the middle third, and small ICs in the bottom 
third. 
38 We received a total of 520 summary statements for the 540 grants in our sample.  This is because certain types 
of grant applications can result in multiple grant awards; for example, multiproject grants.  Twenty of the grants in 
our sample were from grant applications that NIH used as the basis for funding between two and five grants. 
39 NIH, NIH Peer Review: Grants and Cooperative Agreements, p. 2.  Accessed at 
https://grants.nih.gov/grants/peerreview22713webv2.pdf on November 30, 2020. 
40 By complete sets of scores, we generally mean a set of scores from at least three peer reviewers, with each set 
including a numerical score for each of NIHs five scored criteria.  However, for handful of grants, NIHs Funding 
Opportunity Announcements called for an alternative scoring approach, such as an overall single set of scores or 
no scores for the five scored criteria.  In these cases, we reviewed scores against the specified alternative 
approach. 
41 For these grants, at least three peer reviewers and/or the Scientific Review Officer deemed a criterion to be 
unacceptable. 
42 For details on select agents, see 
https://www.selectagents.gov/sat/list.htm?CDC_AA_refVal=https%3A%2F%2Fwww.selectagents.gov%2FSelectAge
ntsandToxinsList.html  
 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Endnotes | 28 
   
43 NIH, NIH Grants Policy Statement, Section 16.3.  Accessed at https://archives.nih.gov/asites/grants/04-16-
2018/policy/nihgps/index.htm on February 23, 2021. 
44 We note that beyond peer review, NIH has additional processes for vetting and approving the use of select 
agents and involvement of foreign entities in research, e.g., vetting applicants through the U.S. Department of 
State and the Centers for Disease Control and Prevention. 
45 NIH, Statement on Protecting the Integrity of U.S. Biomedical Research.  Accessed online at 
https://www.nih.gov/about-nih/who-we-are/nih-director/statements/statement-protecting-integrity-us-
biomedical-research on April 6, 2021; NIH, NIH Peer Review: Grants and Cooperative Achievements. 2019. Accessed 
online at https://grants.nih.gov/grants/PeerReview22713webv2.pdf on March 22, 2021.  
46 HHS Grants Policy Administration Manual, pt. G. ch. 1. (b)(56(ii). 
47 One grant was funded because the IC identified concerns with two more-favorably ranked grants ahead of it.  In 
this case, the IC wrote justifications for not funding the more-favorably ranked grants.  Those justifications 
acknowledged the strength of the proposed projects but cited administrative concerns with the principal 
investigators ability to do the research.  Examples include concerns that the investigator was overcommitted or 
uncertainty about whether the investigators employment status at the grantee institution would soon end. 
Selected NIH Institutes Met Requirements to Document Peer Review 
But Could Do More To Track and Explain Funding Decisions, OEI-01-19-00140  Endnotes | 29