ther general comments: 
 
U.S. Department of Health and Human Services 
Office of Inspector General 
 
Vetting Peer 
Reviewers at NIHs 
Center for 
Scientific Review: 
Strengths and 
Limitations 
 
Suzanne Murrin 
OEI-01-19-00160 
Deputy Inspector General 
September 2019 
for Evaluation and Inspections 
 
oig.hhs.gov  Report in Brief  U.S. Department of Health and Human Services 
September 2019 
Office  of Inspector General 
OEI-01-19-00160 
 
 
Vetting Peer Reviewers at NIHs Center for 
Why OIG Did This Review 
Scientific Review: Strengths and Limitations 
Congress, NIH, and Federal 
Each year the National Institutes of Healths  intelligence agencies have raised 
(NIH) Center for Scientific Review (CSR)  Key Takeaway  concerns about foreign threats to the 
relies on thousands of peer reviewers who  NIHs Center for Scientific  integrity of U.S. medical research and 
review tens of thousands of extramural  Review focuses its vetting of  intellectual property.  In August 
grant applications, helping NIH to  peer reviewer nominees on  2018, Dr. Francis Collins, Director of 
determine the most promising research to  scientific skills and  NIH, raised concerns that peer 
fund.  These peer reviewers are generally  preventing undue influence  reviewers were, in some cases, 
not Federal employees but are considered  generally, but does not  inappropriately sharing confidential 
professional service consultants.   focus its vetting specifically  information with foreign entities.  
on undue foreign influence.  Subsequently, in 2018 Congress 
What OIG Found  provided the Office of Inspector 
NIHs CSR has strengths in its approach to vetting nominees ability to be  General with $5 million for oversight 
effective peer reviewers.  CSR verifies nominees scientific expertise using  of NIH grant programs and 
multiple sources such as their publication and grant histories.  It also assesses  operations, including the 
nominees communication skills and their ability to leverage those skills in  effectiveness of NIHs efforts to 
a peer review setting.  Peer reviewer nominees are typically well known to  protect intellectual property derived 
CSR even before their nominations, having served as temporary reviewers or  from NIH-supported research.  This 
having been NIH grantees.  This allows CSR to evaluate the nominees on the  study assesses the strengths and 
basis of its past experiences with them.  NIH also has controls to ensure that  limitations of CSRs vetting of peer 
reviewer nominees before they begin 
it does not select nominees who have engaged in research misconduct or 
breaches of peer review.  However, CSRs vetting of peer reviewer nominees  reviewing applications for research 
gives little attention to foreign affiliation beyond requiring a justification for  grants.  These peer reviewers have 
reviewers who are not based in North America.  Although nearly all of CSRs  a unique opportunity to access 
peer reviewers work at institutions within the United States, they include both  confidential information in grant 
U.S. citizens and foreign nationals.  CSR vets all of these reviewers in the  applications.  Because of this access, 
same way.  CSR generally limits the sources it uses in vetting for other  it is important for NIH to ensure that 
concerns (e.g., legal and moral controversies) to publicly reported  peer reviewers do not 
information, which is unlikely to reveal such things as undisclosed foreign  inappropriately disclose or divert 
support.   confidential information, including 
intellectual property. 
What OIG Recommends  
How OIG Did This Review 
NIH is taking steps to address concerns about foreign threats to research 
We interviewed NIH staff at CSR, the 
integrity, but it could do more with respect to vetting peer reviewer 
Office of Extramural Research, the 
nominees.  We recommend that NIH update its guidance on vetting peer 
reviewer nominees to identify potential foreign threats to research integrity,  Office of Management Assessment, 
in consultation with national security experts, as needed, and that NIH work  and the Office of Federal Advisory 
with the Department of Health and Human Services Office of National  Committee Policy.  We discussed 
Security to develop a risk-based approach for identifying those peer reviewer  how CSR staff vet peer reviewer 
nominees who warrant additional vetting.  NIH concurred with both  nominees and how NIH policy 
recommendations. governs that vetting.  We also 
reviewed NIH policy, guidance, and 
training materials related to vetting 
peer reviewers. 
Full report can be found at oig.hhs.gov/oei/reports/oei-01-19-00160.asp   
TABLE OF CONTENTS 
BACKGROUND  1 
Methodology  5 
   
FINDINGS   
NIHs Center for Scientific Review has strengths in its approach to vetting nominees ability  7 
to be effective peer reviewers 
Beyond requiring a justification for reviewers who are not based in North America, NIHs  11 
Center for Scientific Review gives little attention to foreign affiliation in its peer reviewer 
vetting 
NIHs Center for Scientific Review generally limits the sources it uses in vetting for other  13 
concerns to publicly available information  
   
CONCLUSION AND RECOMMENDATIONS   
NIH should update its guidance on vetting peer reviewer nominees to identify potential  15 
foreign threats to research integrity, in consultation with national security experts, as 
needed  
NIH should work with the HHS Office of National Security to develop a risk-based approach  16 
for identifying those peer reviewer nominees who warrant extra scrutiny 
   
AGENCY COMMENTS AND OIG RESPONSE  17 
 
APPENDIX: AGENCY COMMENTS  18 
 
ACKNOWLEDGMENTS  20 
 
ENDNOTES  21 
 
 
   
BACKGROUND 
Objective 
To describe and assess the National Institutes of Healths 
(NIHs) process for vetting peer reviewer nominees at its 
Center for Scientific Review (CSR). 
 
Concerns About  For FY 2019, the U.S. Department of Health and Human Services (HHS) 
Office of Inspector General (OIG) received $5 million in congressional 
Foreign Threats to 
appropriations to conduct oversight of NIH grant programs and 
Research Integrity 
operations.1  As indicated by the conference report that accompanied the 
legislation, the conferees direct[ed] the OIG to examine NIHs oversight 
of its grantees compliance with NIH policies.2 
This review is part of a larger body of HHS OIG work focused on oversight 
of NIH grant programs and operations.  Our work will review 
(1) intellectual property and cybersecurity protections; (2) compliance with 
Federal requirements and NIH policies for grants and contracts; and 
(3) integrity of grant application and selection processes.  As part of this 
oversight work, Congress directed that OIG examine (1) NIHs efforts to 
ensure the integrity of its grant evaluation and selection processes, and 
(2) the effectiveness of NIHs and grantee institutions efforts to protect 
intellectual property derived from NIH-supported research.3 
In his August 2018 statement on protecting the integrity of U.S. 
biomedical research, NIH Director Dr. Francis Collins stated that the risks 
to the security of intellectual property and the integrity of peer review 
were increasing.  He identified specific areas of concern including, in some 
instances, the inappropriate sharing of confidential information by peer 
reviewers.  Dr. Collins further stated that in response to the increasing 
risks, NIH would work with other government agencies, institutions, and 
organizations to identify robust methods to protect the integrity of peer 
review.4  At the same time, Dr. Collins wrote to NIH grantee institutions 
alerting them to these foreign threats, and stated that NIH would be 
working with the institutions to address those threats.5 
Additionally, congressional committees have expressed concerns about 
potential threats to the integrity of taxpayer-funded research and 
intellectual property, including the theft of intellectual property and its 
diversion to foreign entities.6  Indeed, in a June 2019 Senate hearing, NIH 
Principal Deputy Director Dr. Lawrence A. Tabak testified that NIH was 
aware that a few foreign governments have initiated systematic 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  1 
OEI-01-19-00160   
programs to capitalize on the collaborative nature of biomedical research 
and unduly influence U.S.-based researchers.7 
 
Chinas Thousand Talents Plan 
Chinas central government announced the Thousand Talents plan 
in 2008.  One aspect of the plan provides financial incentives for 
Chinese scientists living abroad to return to China.8  According to 
NIH, access to foreign intellectual property is key to a scientists 
selection for the Thousand Talents program.  In a 2018 meeting of 
NIHs Advisory Committee to the Director, NIH identified several 
concerns related to the Thousand Talents plan: 
  undisclosed foreign financial conflicts; 
  undisclosed conflicts of commitment; and 
  peer review violations, including the inappropriate sharing of 
confidential information.9 
 
The peer review process is central to the integrity of evaluating and 
selecting grants.  According to NIH policy, the peer review process is 
intended to ensure that applications for funding submitted to NIH are 
evaluated on the basis of a process that is fair, equitable, timely, and 
conducted in a manner that strives to eliminate bias.10  However, as 
Dr. Collins statement suggested, peer reviewers also present a potential 
risk to intellectual property and other confidential information contained 
in grant applications if they inappropriately share that information.11 
Because peer reviewers conduct the initial review of research grants 
submitted to NIH, they have a unique opportunity to access confidential 
information in grant applications.  Accordingly, NIH policy seeks to ensure 
that peer reviewers do not inappropriately disclose or divert confidential 
information, including information related to intellectual property.12  
 
Steps NIH Has  In response to these concerns, NIH has taken steps to protect the integrity 
of the grant and peer review process.  To further raise awareness of 
Taken To Address 
confidentiality in peer review, NIH has launched ongoing communications 
Concerns About 
with its staff, the research community, and grantee institutions, some of 
Foreign Threats to  which have proactively raised concerns with NIH.13  NIH convened 
Research Integrity  a working group of the Advisory Committee to the NIH Director to 
explore additional steps to protect the integrity of peer review.14  NIH is 
also working with Federal agencies both inside HHS (e.g., the HHS Office 
of National Security and the Office of Inspector General) and outside 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  2 
OEI-01-19-00160   
(such as the Federal Bureau of Investigation) to identify and follow up on 
scientists of possible concern among NIH grantees.15  According to NIH, it 
has identified about 250 scientists of concern to date.  Furthermore, NIH 
has worked to improve the security of the electronic systems that 
researchers use to submit grant applications and that peer reviewers use 
to access these applications.16 
NIHs Peer Review  Each year, NIH receives about 80,000 extramural grant applications.  
(Extramural research is research that NIH supports through funds to 
Process 
outside researchers and organizations.)  It uses a two-level review process 
to evaluate the applications.17  For about 75 percent of those applications, 
CSR manages the first level of review.  For the remaining 25 percent of 
applications, NIHs other institutes and centers (ICs) manage the first level 
of review. 
At CSR, peer reviewers who serve on study sections (also known as 
scientific review groups) conduct the first level of review of grant 
applications.18  During this review, study sections evaluate whether 
proposed projects are likely to have a major scientific impact based on 
their scientific and technical merit.19  Study sections are organized by 
research area and include peer reviewers recognized as experts in that 
research area.  After the study section conducts its review, an Advisory 
Council or Board conducts the second level of review.20  The Director of 
each IC or a designee makes the final funding decision.21 
Over 27,000 peer reviewers review grant applications annually for NIH.22  
First-Level Peer 
NIH seeks to ensure that peer reviewers review grant applications in 
Reviewers 
a manner free from inappropriate influences.  It identifies the following 
core values for its peer reviewers: (1) expert assessment; (2) transparency; 
(3) impartiality; (4) fairness; (5) confidentiality; (6) security (added in 2018); 
(7) integrity, and (8) efficiency.23  NIHs peer reviewers are generally not 
Federal employees or Special Government Employees.24  NIH considers 
them to be professional service consultants.25  However, up to 25 percent 
of the members of a first-level peer review group can be Federal 
employees.26 
Nominating and Vetting Peer Reviewers at CSR 
Scientific Review Officers (SROs) at CSR are responsible for identifying 
potential peer reviewers and nominating them to CSR study sections for 
consideration.   
SROs recruit people to serve both as temporary and appointed peer 
reviewers.  Most people are first recruited to be temporary reviewers 
before being nominated to be appointed peer reviewers.   
   
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  3 
OEI-01-19-00160   
 
 
Temporary and Appointed Peer Reviewers 
Temporary peer reviewers provide specific expertise.  As temporary 
reviewers, they offer a fresh perspective for individual peer review 
meetings.  If temporary reviewers are successful, SROs may 
nominate them to become appointed peer reviewers. 
Appointed peer reviewers typically serve 4-year terms and are 
expected to attend all meetings of their respective standing study 
sections.27 
NIH also offers the public several ways to apply for consideration as 
a peer reviewer, including contacting an SRO from CSR, applying to NIHs 
Early Career Reviewer Program (a CSR program to help start research 
careers and enhance the pool of future reviewers), and e-mailing ones 
curriculum vitae to an NIH peer reviewer mailbox.28 
For appointed nominees, SROs prepare a nomination slate of peer 
reviewers based on the nominees expertise and the needs of each study 
section.  CSRs Nomination Slate Guidelines instruct SROs to nominate 
people with the necessary scientific expertise for that study section.29  CSR 
trains SROs on these guidelines. 
Other parts of NIH in addition to CSR also have roles overseeing and 
developing policies related to vetting peer reviewers.  The Office of 
Extramural Research both oversees the integrity of the peer review 
process and writes policies governing the external grantmaking process.30  
This office also maintains NIHs Electronic Research Administration 
program, which provides information technology support to NIHs 
grantmaking process.  Part of this program is the Query/View/Report 
module, a system that contains profiles of all extramural grantees and 
peer reviewers.31  The Office of Management Assessment conducts 
internal risk assessments and advises NIH on program integrity matters.32  
Finally, the Office of Federal Advisory Committee Policy develops policies 
related to the Federal Advisory Committee Act, which governs the 
establishment and operation of advisory committees across all Federal 
agencies.33 
 
Related Work  OIG conducted a previous evaluation related to NIHs peer review process.  
In response to a congressional request, we examined the extent to which 
NIHs National Institute of Environmental Health Sciences (NIEHS) 
followed its peer review processes for funding research related to 
bisphenol A (BPA).  In our 2017 report, OIG found that although NIEHS 
met the requirements of the peer review process for all its grants, it used 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  4 
OEI-01-19-00160   
its discretion to fund applications with less favorable scores than 
competing applications for 14 percent of BPA grants, versus for 4 percent 
of other, non-BPA grants.  OIG concluded that although NIEHS has 
discretion to fund grant applications with less favorable impact scores 
ahead of competing applications, NIEHSs applying its discretion 
frequently or disproportionately in one research area may invite scrutiny 
to its funding decisions.34 
OIG is currently evaluating how NIH oversees its peer reviewers once they 
begin reviewing grant applications.  That work is forthcoming. 
 
Methodology  Scope 
This report covers the vetting process that NIH used in 2019 for peer 
reviewers who serve on study sections organized by CSR.  These study 
sections review about 75 percent of all NIH grant applications.  This report 
defines vetting as the initial screening that NIH staff conduct for peer 
reviewer nominees before they begin reviewing grant applications.  Future 
OIG work will focus on NIHs oversight of peer reviewers once they begin 
reviewing grant applications.35 
This report does not include the first level of review organized by other 
ICs (e.g., the National Cancer Institute), special emphasis panels, or the 
second level of grant application review.  Scientific and disease experts 
who are appointed as Special Government Employees largely conduct the 
second level of review. 
Data Collection 
We requested from NIH its policies, procedures, and training materials 
governing peer reviewer vetting.  Some of the procedures were specific to 
CSR, and some policieslike those from the Office of Extramural 
Researchapplied to all of NIH.  We interviewed staff at CSR about how 
they vet peer review nominees.  We interviewed five SROs, four Integrated 
Review Group Chiefs, the CSR Research Integrity Officer, and staff from 
the Office of the Director of CSR.  SROs conduct the initial vetting of peer 
reviewer nominees.  They are followed by the Integrated Review Group 
Chiefs, who review the materials that the SROs have compiled.  We also 
interviewed NIH staff at the Office of Extramural Research, the Office of 
Management Assessment, and the Office of Federal Advisory Committee 
Policy about their roles in setting policy regarding the vetting of peer 
reviewers. 
   
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  5 
OEI-01-19-00160   
Data Analysis 
We analyzed the documents and interviews to identify strengths and 
limits of CSRs approach to vetting peer reviewer nominees, and the 
extent to which CSR considered undue foreign influence.   
Limitations 
Although some of the policies that we reviewed apply to all of NIH, our 
findings and recommendations are focused on CSR.  We did not verify 
staffs compliance with NIHs vetting policies. 
Standards  We conducted this study in accordance with the Quality Standards for 
Inspection and Evaluation issued by the Council of the Inspectors General 
on Integrity and Efficiency.   
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  6 
OEI-01-19-00160   
FINDINGS 
NIHs Center for  This report defines vetting as the initial screening that NIH staff conduct 
on peer reviewer nominees before they begin reviewing grant 
Scientific Review 
applications.   
Has Strengths in its 
CSRs Nomination Slate Guidelines state that the expertise of peer 
Approach to Vetting 
reviewers should match the core scientific topics and methods that their 
Nominees Ability 
study sections review.  SROs told us that their primary goal in vetting peer 
To Be Effective Peer  reviewer nominees was to verify that the nominees did in fact possess the 
Reviewers  required expertise, and that the nominees could effectively use that 
expertise within the context of peer review. 
CSR verifies nominees scientific expertise using multiple 
sources 
CSRs Nomination Slate Guidelines describe indicators of expertise that 
SROs should use to assess nominees scientific expertise.  These include 
obtaining their grants; presenting at conferences; being published in 
journals; holding patents; having professional appointments; and having 
editorial and review duties (e.g., at a journal). 
CSR staff told us that accordingly, they vet nominees scientific expertise 
and stature, and they described a thorough process for doing so.  CSR 
staff research nominees publications in scientific journals; the websites of 
their research institutions or companies; their prior research grants from 
NIH and other Federal agencies; as well as the nominees curricula vitae.  
For example, one SRO we spoke with checks online for recorded speeches 
that nominees have given at scientific conferences and symposia.  
Furthermore, Integrated Review Group Chiefs told us they also perform 
their own check of these sources when SROs send them the nomination 
slates for review.  
According to NIH, it values the scientific communitys assessment of its 
nominees for peer review.  CSRs guidelines call for every nominee to have 
two recommendations from references who endorse their credentials and 
expertise, and to reflect independent, broad vetting of the nominee by 
leaders of the scientific community served by the study section.  SROs 
told us that they sometimes talk in person with the reference to get 
a better sense of the nominees communication skills, objectivity, and 
understanding of the responsibilities of serving as a peer reviewer. 
With regard to stature and funding history, CSRs guidelines say that 
a nominee should have academic rank as a full or associate professor and 
should be an active and successful researcher.  The guidelines state that 
although NIH funding is preferred, other competitive sources can be used 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  7 
OEI-01-19-00160   
as indicators of a successful researcher.  (CSRs training materials for SROs 
offer as examples the Department of Defense and the Department of 
Veterans Affairs). 
NIH mentions that additional indicators of the nominees academic 
stature can be: 
  leadership roles in professional organizations; 
  faculty rank and any endowed chairs; and 
  editorial position. 
CSR also considers nominees ability to effectively leverage 
their scientific expertise in a peer review setting 
CSR values reviewers who can articulate their views succinctly, engage in 
productive exchanges, and demonstrate an ability to work collegially in 
a group setting.36  CSR guidelines state that nominees should serve as 
temporary reviewers before being nominated to serve 4-year terms as 
appointed peer reviewers.  This enables multiple CSR staffincluding 
SROs and Integrated Review Group Chiefsto observe the temporary 
reviewers during the meetings and gauge their performance.  CSR staff 
told us they look to see how well the temporary reviewers perform in their 
roles, including how well they interact with other reviewers.  If this 
experience raises questions about a nominees effectiveness, the SRO may 
decline to nominate that individual.  Finally, SROs told us they check 
nominees references to assess their communication skills, along with 
checking their scientific credentials. 
Peer reviewer nominees are typically well known to CSR, 
enabling CSR to vet them against its past experience  
In addition to having past experience with nominees as ad-hoc reviewers, 
NIH is likely to have had experience with them as grantees or appointed 
reviewers in other study sections before SROs nominate them.  In fact, 
CSR staff told us that they have often known nominees for years, and that 
if a person applied to be an appointed peer reviewer without having had 
any of those roles, it would raise a red flag with the SROs.  As a result, NIH 
typically has profiles of the nominees in its Query/View/Report database 
module, allowing SROs to vet the nominees against their past experiences 
with NIH.  SROs also check the module to see whether it includes a Do 
Not Use alert for any of the nominees, which would prevent those 
nominees from serving as peer reviewersin fact, the existence of such 
an alert prevents SROs from adding a person to a nomination slate or 
a meeting roster.  NIHs recent initiatives in working with grantee 
institutions and law enforcement partners have resulted in persons 
receiving Do Not Use alerts. 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  8 
OEI-01-19-00160   
Most nominees have been NIH grant recipients themselves.  This is one of 
the criteria SROs look for when nominating an appointed peer reviewer.  
In fact, SROs must justify nominees 
who have repeatedly applied for and 
Do Not Use Alerts 
failed to obtain NIH grants.  The 
NIH leadership can place a Do Not 
nominees having been grantees 
Use alert on an individuals  
gives NIH more information about 
Query/View/Report profile.  Persons 
their scientific expertise and record of 
with this flag cannot serve as peer 
accomplishment in their fields, which 
reviewers. 
are criteria in CSRs nomination 
guidelines.  
For peer reviewer nominees who have already served as appointed 
reviewers on other study sections, CSR staff can rely on their own 
observations.  CSR staff also share information with each other about 
these reviewers.  In fact, CSR developed a database to share internal 
recommendations from other NIH staff to consider regarding potential 
peer reviewers.  This enables SROs to nominate those with a record of 
effective review experience and to avoid those who did not perform well 
in the past.  As one Integrated Review Group Chief explained, Its not like 
were suddenly hearing about this person out of the blue.  We have plenty 
of time to really look at this person in quite some detail. 
One SRO told us that if someone applied to be a peer reviewer without 
having any of these prior experiences, the SRO would encourage that 
person to apply for NIHs Early Career Reviewer Program.  NIH designed 
this program to help less experienced researchers gain skills to make 
them better grant applicants and enrich NIHs pool of potential peer 
reviewers. 
NIH has controls to ensure that nominees with findings of 
research misconduct and peer review breaches are not selected 
to serve 
NIH relies not only on its own oversight but also that of others to prevent 
individuals who have engaged in research misconduct from being 
nominated.  NIH works with HHSs Office of Research Integrity regarding 
cases of research misconduct.  When that office confirms a finding of 
research misconduct, it notifies NIH, which then places a Do Not Use alert 
in that persons Query/View/Report profile, preventing SROs from 
nominating the person.  Likewise, if NIH learns of misconduct allegations 
first, it will assess the allegations and then forward the case to the Office 
of Research Integrity for followup. 
   
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  9 
OEI-01-19-00160   
 
 
 
The Office of Research Integrity 
  On behalf of the Secretary of Health and Human Services, HHSs 
Office of Research Integrity oversees and directs most research 
 
integrity activitiesincluding oversight of inquiries and 
 
investigations related to research misconductin the Public Health 
  Service divisions of HHS.37  Research misconduct is defined as 
"fabrication, falsification or plagiarism in proposing, performing or 
 
reviewing research, or in reporting research results and does not 
include honest error or differences of opinion.38 
 
Additionally, NIH prevents peer reviewers who have violated review policy 
from serving again.  NIH may also take action against persons who violate 
peer review policy in times other than during a review panelfor 
example, by attempting to communicate with or manipulate other 
reviewers prior to a panel.  NIH staff may become aware of such violations 
during its oversight of reviewers or learn about allegations from other 
reviewers.   
   
Being Vested in a Fair System 
NIH staff told us that they rely on peer reviewers to uphold a culture 
of fairness in the review process.  They explained that because peer 
reviewers are also grantees, they are vested in ensuring that the 
system is fair.  After all, for the majority of reviewers, their own grant 
applications must also go through the process.  NIH relies on peer 
reviewers to report to SROs any potential breach of peer review 
integrity of which they are aware.  SROs refer the allegations to 
Research Integrity Officers at CSR and the Office of Extramural 
Research, both of which evaluate such allegations.  If the Research 
Integrity Officers confirm the allegations, NIH leadership bars the 
individuals from participating as peer reviewers in the future.   
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  10 
OEI-01-19-00160   
Beyond Requiring a  According to NIH, nearly all of CSRs peer reviewers work at research 
institutions located within the United States.  These reviewers include 
Justification for 
both U.S. citizens and foreign nationals.  NIH does not require nominees 
Reviewers Who Are 
to disclose citizenship, but it typically knows the research institutions at 
Not Based in North  which they are based.  A very small number of peer reviewers
America, NIHs  0.6 percent of reviewers, according to CSRwork at foreign research 
institutions.  CSR staff told us that most of these foreign-based reviewers 
Center for Scientific 
are from Canadian institutions. 
Review Gives Little 
NIH focuses on preventing undue influence generally, but not 
Attention to Foreign 
specifically undue foreign influence 
Affiliation in its Peer 
One of the key ways in which NIH upholds the integrity of peer review is 
Reviewer Vetting  
by working to ensure that its peer reviewers cannot exert or be swayed by 
undue influence.  Both CSRs 
Nomination Slate Guidelines 
Characterizing Undue Influence 
and the Federal Advisory 
Committee Act39 specify  NIH intends its peer review process 
criteria for preventing undue  to be fair and conducted in a manner 
influence in peer review.   that strives to eliminate bias.  Bias 
These criteria assess a  can include undue influence by and 
nominee individually and  of peer reviewers.  
with respect to the peer 
Although NIH does not define undue 
reviewers with whom the 
influence, it considers several ways in 
nominee will serve.   
which it could occur.  For example, if 
a study section contained multiple 
The CSR staff we spoke with 
peer reviewers from a specific 
described taking steps 
research institution, or if it included 
accordingly to prevent any 
peer reviewers who had served on 
individual research 
that study section for a long time, 
institution from being able 
those reviewers could hold too much 
to overly influence the 
sway over the rest of the group, 
review process, and any 
adversely affecting the objectivity of 
reviewer from being 
peer review. 
co-opted by others.  For 
example, CSR staff said that  Additionally, Congress and NIH have 
to avoid undue influence  raised concerns about undue foreign 
from any single research  influencefor example, if foreign 
institution, study sections  entities systemically encouraged NIH 
rarely include more than one  peer reviewers to disclose 
reviewer from the same  confidential information or influence 
institution.  CSR staff also  grant funding decisions. 
said they try to ensure that 
study sections contain 
balanced and diverse representation in terms of demographics (e.g., race 
and gender) and region.  Furthermore, CSRs training materials for 
evaluating potential peer reviewers instruct SROs to consider diversity of 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  11 
OEI-01-19-00160   
scientific skills (e.g., methods and topics).  Finally, NIH policy and 
guidelines also include provisions to ensure that particular nominees do 
not have undue influence during grant reviews.  NIH policy prohibits any 
reviewer from serving on both the first and second levels of a given 
review, and CSRs Nomination Slate Guidelines states that reviewers 
should generally not serve multiple terms on the same study section.    
However, neither CSRs Nomination Slate Guidelines nor the Federal 
Advisory Committee Act address the circumstances in which foreign 
affiliation (e.g., persons employed by a foreign government or company, 
or who consult for a foreign government) could raise potential concerns 
of undue influence.  Although nearly all of CSRs peer reviewers work at 
research institutions within the United States, they include both U.S. 
citizens and foreign nationals.  CSR vets all of these reviewers in the same 
way.   
According to NIH, identifying undue foreign influence is not something 
that SROs do or could do.  NIH staff stated that many foreign affiliations 
are legitimate and are not, in and of themselves, suggestive of undue 
influence.  In fact, NIHs Review Integrity Training for SROs reminds them 
of the value that individuals with foreign affiliations and collaborations 
provide to NIH.  This training also states that, unless a Do Not Use alert 
exists in a persons Query/View/Report profile, the person is eligible to 
serve at the SROs discretion.  Furthermore, it would be difficult for CSR 
staff to detect undue foreign influence even if peer reviewer nominees 
had previously disclosed on their curricula vitaeeither as peer reviewer 
nominees or as NIH grant applicantsany foreign affiliations or foreign 
support.  It would be even more challenging to identify undue foreign 
influence if nominees had failed to disclose such support.  Because of this 
challenge, NIH risks bringing on board peer reviewers with conflicts of 
interest or commitment that could affect the outcome or integrity of peer 
review. 
NIH has efforts underway to address this challenge, led primarily by the 
Office of Extramural Research.  NIHs ongoing work with partners within 
HHS and elsewhere to identify and follow up on scientists of concern 
among NIH grantees shows that peer review is indeed at risk of undue 
foreign influence.  In fact, according to NIH, of the 250 scientists it has 
identified to date as individuals of possible concern, roughly 30 percent 
had served as a peer reviewer over the past 2 years, and NIHs follow up 
continues. 
 
CSR policy discouragesbecause of logistical considerations
the use of reviewers based outside of North America 
Because study section meetings generally take place in person in the 
United States, CSR policy favors the use of peer reviewers who work at 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  12 
OEI-01-19-00160   
North American research institutions.  In its 2009 guidance on reviewers 
who are based abroad, CSR stated that in general, SROs should not 
nominate reviewers who reside and work abroadregardless of their 
nationalityto serve as appointed peer reviewers.  CSR may make 
exceptions in cases in which, for example, a nominee has unique 
knowledge or expertise.  In such cases, SROs must first discuss the 
nominees candidacy with their Integrated Review Group Chiefs and 
submit a justification for approval to their Division Director.  However, 
CSR instructs SROs to keep this practice to a minimum.  This may explain 
in part why the overwhelming majority of peer reviewers work at research 
institutions based in the United States. 
The guidance focuses on logistical issuessuch as the travel required, 
cost, and administrative burdento explain why SROs should not 
nominate reviewers based abroad.  The guidance exempts reviewers from 
Canada and Mexico from requiring special approval because travel costs 
for such reviewers tend to be comparable to those for reviewers based in 
the United States.  In addition, because CSR has invited reviewers from 
Canada and Mexico to serve as temporary or appointed peer reviewers in 
the past, and because such individuals are often NIH grantees, they are 
likely to be familiar with the NIH peer review process.   
Furthermore, NIH views its close working relationships with grantee 
institutions to be a key part of its response to concerns over the security 
of intellectual property and the integrity of peer review.  Using peer 
reviewers who are based at grantee institutions is important because the 
institutionsas the peer reviewers employershave the authority to take 
actions against employees who violate the integrity of peer review.40 
 
NIHs Center for  In vetting peer reviewers, verifying nominees scientific ability is NIHs 
foremost priority.  CSRs Nomination Slate Guidelines address how to 
Scientific Review 
assess an individuals expertise, stature, and funding history as 
generally limits the 
a researcher.  The guidelines also state that SROs should conduct an 
sources it uses in  ethics screening to identify legal or moral controversies surrounding 
a nominee (e.g., findings of sexual harassment).  SROs have access to 
vetting for other 
candidates profiles in NIHs Query/View/Report, and they check for any 
concerns to publicly 
Do Not Use alerts that were placed on any candidates profiles as a result 
reported 
of research misconduct or breaches of peer review, or as a result of any 
information  nonpublic information that NIH becomes aware of. 
However, the guidelines do not advise vetting nominees against any type 
of law enforcement database.  Instead, they suggest generally reviewing 
the first page of results from a Google search.  NIH staff told us that in 
their screenings of nominees, they search to see whether any major news 
organizations have published anything regarding nominees and criminal 
activity, legal issues, or sexual misconduct.  Relying on the Internet or 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  13 
OEI-01-19-00160   
news organizations for this type of information means that NIH is unlikely 
to discover concerns that are not reported in the public domain.   
NIH staff described the burdens that would be associated with conducting 
traditional background checks on all peer reviewers.  They noted that 
SROs are scientists and not trained as investigators.  In fact, NIH estimates 
thatin its experience in vetting about 250 scientistsconducting 
a preliminary investigation would require roughly 10 hours for each 
potential reviewer.  NIH further estimates that vetting all 27,000 peer 
reviewers in this manner would require more than 100 additional full-time 
staff.   
   
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  14 
OEI-01-19-00160   
CONCLUSION AND RECOMMENDATIONS 
NIHs peer review process is a vast enterprise that is key to supporting its 
research mission.  Each year, this process encompasses thousands of 
reviewers who review tens of thousands of grant applications.  NIHs peer 
review enterprise relies on experts in their field being willing to serve.   
NIHs Center for Scientific Review places a high value on recruiting 
scientific experts to serve as peer reviewers.  However, the world is 
changing, and threats have emerged.  Peer reviewers who disclose 
confidential information to foreign entities could undermine confidence in 
the integrity of peer review, put intellectual property at risk, and diminish 
NIHs reputation.  NIH has been working with the HHS Office of National 
Security, Federal law enforcement (including the HHS Office of Inspector 
General and the Department of Justice), and research institutions to 
address concerns about foreign threats to research integrity in other 
contexts, such as grantees failing to disclose foreign financial interests as 
required.  NIH has also worked to better secure the systems that peer 
reviewers use when accessing grant applications.  
However, in its procedures for vetting peer reviewer nominees, NIH has 
not addressed the concerns about foreign threats to the integrity of the 
peer review process.  OIG recognizes that NIH has raised several concerns 
related to addressing foreign threats as part of the process of vetting peer 
reviewer nominees.  These include concerns about added burden for the 
nominees and NIH staff; concerns about perceptions of racial/ethnic 
profiling and stigmatization; and concerns that identifying undue foreign 
influence is challenging and largely outside of NIHs expertise.  In 
addition, no system of vetting is foolproof.  Nonetheless, NIH has 
an opportunity to enhance its vetting of peer reviewers and further 
protect its integrity and reputation.  Therefore, we recommend that NIH: 
Update its guidance on vetting peer reviewer nominees to 
identify potential foreign threats to research integrity, in 
consultation with national security experts, as needed 
Much of CSRs guidance to SROs about peer reviewer nominees with 
foreign affiliations is 10 years old and addresses only the logistical 
challenges of using reviewers who are based abroad.  NIH could 
strengthen its guidance by expanding it to include foreign threats to 
research integrity such as foreign support that nominees have omitted 
from their curricula vitae.  CSR is already including review integrity in its 
training for SROs.  If CSR updated its guidance to include what is already 
in this training, it could highlight these concerns and help ensure 
consistency across SROs and other staff in acting on them. 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  15 
OEI-01-19-00160   
NIH could further update its guidance by drawing on its experience and 
consultation with national security experts and include examples of 
situations that warrant followup.  The guidance could enhance awareness 
among SROs to report these situations to CSRs Research Integrity Officer, 
who could then assess them as appropriate.  We recognize that this could 
result in a higher volume of work for CSRs Research Integrity Officer (and 
ultimately the Research Integrity Officer at the Office of Extramural 
Research), but SROs are not investigators trained in assessing threats to 
research integrity.  Such guidance also provides NIH with the opportunity 
to elevate the prominence of its new core valuesecurityfor peer 
reviewers. 
Work with the HHS Office of National Security to develop 
a risk-based approach for identifying those peer reviewer 
nominees who warrant additional vetting 
As the HHS agency focused on national security, the Office of National 
Security is a natural partner for NIH to strengthen the process of vetting 
peer reviewer nominees.  In fact, the Office of National Security is already 
working with NIH to identify steps to mitigate threats to U.S. biomedical 
research from foreign entities.  For example, the Office of National 
Security currently vets some employees and foreign visitors to NIH by 
using databases of its national security partners. 
Using a risk-based strategy would enable NIH to expand its vetting for 
a subset of peer reviewer nominees, while also limiting the increase in 
burden on its staff.  For example, NIH could focus extra attention to 
vetting peer reviewer nominees who would be reviewing grant 
applications with particularly sensitive subject matter or that have 
lucrative commercial applications.  Additionally, by working with the 
Office of National Security, NIH can leverage the Office of National 
Securitys expertise and expand the sources that NIH uses to vet peer 
reviewer candidates.  
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  16 
OEI-01-19-00160   
AGENCY COMMENTS AND OIG RESPONSE 
NIH concurred with both of our recommendations. 
In response to the first recommendation, NIH noted that it is working 
closely with Federal partners both inside the Department (such as OIG and 
the HHS Office of National Security) and external agencies, including the 
Federal Bureau of Investigation, to update its guidelines for vetting peer 
reviewers. 
Regarding the second recommendation, NIH stated that is developing 
a systematic, risk-based, data-driven approach to identifying peer review 
nominees who warrant additional scrutiny.   
For the full text of NIHs comments, see the Appendix. 
   
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  17 
OEI-01-19-00160   
APPENDIX: AGENCY COMMENTS 
   
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  18 
OEI-01-19-00160   
 
   
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  19 
OEI-01-19-00160   
ACKNOWLEDGMENTS 
Ivan Troy served as the team leader for this study.  Others in the Office of 
Evaluation and Inspections who conducted the study include 
Matt Blackburn and Shakye Jones.  Office of Evaluation and Inspections 
staff who provided support include Clarence Arnold, Christine Moritz, and 
Melicia Seay.  
We would also like to acknowledge the contributions of other Office of 
Inspector General staff, including Lauren Anderson, Melissa Baker, 
Sarah Vogel, Kelly Waldhoff, and Abigail Wydra. 
This report was prepared under the direction of Joyce M. Greenleaf, 
Regional Inspector General for Evaluation and Inspections in the Boston 
regional office.  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
To obtain additional information concerning this report or to obtain 
copies, contact the Office of Public Affairs at Public.Affairs@oig.hhs.gov.  
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  20 
OEI-01-19-00160   
ENDNOTES 
1 Department of Defense and Labor, Health and Human Services, and Education 
Appropriations Act, 2019, and Continuing Appropriations Act, 2019, P.L. No. 115-245 
(Sept. 28, 2018). 
2 Department of Defense for the Fiscal Year Ending September 30, 2019, and for 
Other Purposes.  Conference Report to Accompany H.R. 6157, H.R. Rep. 
No. 115-952, 2018, pp. 531-532. 
3 Ibid. 
4 NIH, Statement on Protecting the Integrity of U.S. Biomedical Research.  August 23, 
2018.  Accessed at https://www.nih.gov/about-nih/who-we-are/nih-
director/statements/statement-protecting-integrity-us-biomedical-research on 
February 14, 2019. 
5 NIH Advisory Committee to the Director, ACD Working Group for Foreign 
Influences on Research Integrity: December 2018 Report.  Accessed at 
https://acd.od.nih.gov/documents/presentations/12132018ForeignInfluences_report.
pdf on August 9, 2019. 
6 Senator Charles Grassley, Chair, Senate Committee on Judiciary, letter to Dr. 
Francis Collins, October 23, 2018.  Accessed at 
https://www.grassley.senate.gov/sites/default/files/constituents/2018-10-
23%20CEG%20to%20NIH%20(Research%20Threats).pdf on February 14, 2019.  
Association of American Medical Colleges, NIH Describes for Senate Panel Efforts to 
Protect Research Integrity, August 24, 2018.  Accessed at 
https://www.aamc.org/advocacy/washhigh/highlights2018/490948/082418nihdescri
besforsenatepaneleffortstoprotectresearchintegrit.html on February 14, 2019.  
7 Testimony of Dr. Lawrence A. Tabak, Principal Deputy Director, NIH, before Senate 
Committee on Finance, June 5, 2019.  Accessed at 
https://www.finance.senate.gov/imo/media/doc/05JUN2019TabakSMNT.pdf on 
June 20, 2019. 
8 Jia, Hepeng, What is Chinas Thousand Talents Plan?  Accessed at 
https://www.nature.com/magazine-assets/d41586-018-00538-z/d41586-018-00538-
z.pdf on June 20, 2019. 
9 NIH Advisory Committee to the Director, ACD Working Group for Foreign 
Influences on Research Integrity: December 2018 Report.  (See Endnote 5 for URL 
information.) 
10 NIH, Grants Policy Statement, Part 1, October 2018, pp. 67-68.  Accessed at 
https://grants.nih.gov/grants/policy/nihgps/nihgps.pdf on July 9, 2019. 
11 NIH, Statement on Protecting the Integrity of U.S. Biomedical Research.  August 23, 
2018.  (See Endnote 4 for URL information.)  
12 NIH, Maintaining Confidentiality in NIH Peer Review.  Accessed at 
https://grants.nih.gov/grants/guide/notice-files/NOT-OD-14-073.html on 
January 17, 2019.  NIH, Conflicts of Interest Rules: Information for Reviewers of NIH 
Applications and R&D Contract Proposals, July 2015. 
13 Testimony of Dr. Lawrence A. Tabak, Principal Deputy Director, NIH, before 
Senate Committee on Finance, June 5, 2019.  (See Endnote 7 for URL information.) 
14 NIH Advisory Committee to the Director, ACD Working Group for Foreign 
Influences on Research Integrity: December 2018 Report.  (See Endnote 5 for URL 
information.) 
 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  21 
OEI-01-19-00160   
 
15 Testimony of Captain Michael Schmoyer, Assistant Deputy Secretary for National 
Security, Director, Office of National Security, before Senate Committee on Finance, 
June 5, 2019.  Accessed at 
https://www.finance.senate.gov/imo/media/doc/05JUN2019SchmoyerSMNT.pdf on 
June 24, 2019.  Testimony of Dr. Lawrence A. Tabak, Principal Deputy Director, NIH, 
before Senate Committee on Finance, June 5, 2019.  (See Endnote 7 for URL 
information.) 
16 Testimony of Dr. Lawrence A. Tabak, Principal Deputy Director, NIH, before 
Senate Committee on Finance, June 5, 2019.  (See Endnote 7 for URL information.) 
17 NIH, Grants Policy Statement (October 2018), Page I-68.  (See Endnote 10 for URL 
information.) 
18 Ibid, pp. I-36 and I-68. 
19 Ibid, p. I-69. 
20 Ibid, pp. I-68 and I-72. 
21 Ibid, p. I-73. 
22 NIH, Data Book.  Accessed at https://report.nih.gov/nihdatabook/category/12 on 
August 5, 2019. 
23 NIH, Peer Review: Grants and Cooperative Agreements.  Accessed at 
https://grants.nih.gov/grants/peerreview22713webv2.pdf on June 5, 2019. 
24 As defined in 18 U.S.C.  202, a Special Government Employee is an officer or 
employee who is retained, designated, appointed, or employed to perform 
temporary duties, with or without compensation, for not more than 130 days during 
any period of 365 consecutive days. 
25 NIH, New Process for Reimbursement of NIH Peer Reviewers.  August 5, 2005.  
Accessed at https://grants.nih.gov/grants/guide/notice-files/not-od-05-062.html on 
July 10, 2019. 
26 42 CFR  52h.4. 
27 NIH, How Scientists are Selected to be Members of a Chartered Review Group.   
Accessed at 
https://public.csr.nih.gov/ForReviewers/BecomeAReviewer/CharteredReviewers on 
January 25, 2019. 
28 NIH, How to Become A Reviewer.  Accessed at 
https://public.csr.nih.gov/ForReviewers/BecomeAReviewer/HowToApply on 
January 17, 2019. 
29 NIH, CSR Nomination Slate Guidelines for 2019 (document provided to OIG), and 
NIH Peer Review: Grants and Cooperative Agreements.  Accessed at 
https://grants.nih.gov/grants/peerreview22713webv2.pdf on June 11, 2019. 
30 NIH, About the Office of Extramural Research at NIH.  Accessed at 
https://grants.nih.gov/aboutoer/intro2oer.htm on June 11, 2019. 
31 NIH, Query, View, and Report.  Accessed at https://archives.nih.gov/asites/era/04-
14-2016/nih_and_grantor_agencies/other/query_view_and_report.cfm on June 11, 
2019. 
32 NIH, Office of Management AssessmentAbout OMA Services.  Accessed at 
https://oma.od.nih.gov/Pages/OMA-Services.aspx on June 11, 2019. 
33 NIH, Office of Advisory Committee PolicyAbout Us.  Accessed at 
https://ofacp.od.nih.gov/about_us/index.asp on June 11, 2019.  General Services 
Administration, FACA Management Overview.  Accessed at 
https://www.gsa.gov/policy-regulations/policy/federal-advisory-committee-act-
faca-management-overview on June 11, 2109. 
 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  22 
OEI-01-19-00160   
 
34 OIG, As Funding for BPA Increased, NIEHS Followed its Peer Review Process While 
Also Exercising Its Discretion, OEI-01-15-00150, July 2017. 
35 Ibid. 
36 NIH, How Scientists are Selected to be Members of a Chartered Review Group.  
(See Endnote 27 for URL information.) 
37 Office of Research Integrity, About ORI.  Accessed at https://ori.hhs.gov/about-ori 
on June 20, 2019. 
38 42 CFR pt. 93. 
39 See 5 U.S.C. app. 5(c). 
40 Testimony of Dr. Lawrence A. Tabak, Principal Deputy Director, NIH, before the 
Senate Committee on Finance, June 5, 2019.  (See Endnote 7 for URL information.) 
Vetting Peer Reviewers at NIHs Center for Scientific Review: Strengths and Limitations  23 
OEI-01-19-00160   
ABOUT THE OFFICE OF INSPECTOR GENERAL 
The mission of the Office of Inspector General (OIG), as mandated by 
Public Law 95-452, as amended, is to protect the integrity of the 
Department of Health and Human Services (HHS) programs, as well as the 
health and welfare of beneficiaries served by those programs.  This 
statutory mission is carried out through a nationwide network of audits, 
investigations, and inspections conducted by the following operating 
components: 
Office of Audit  The Office of Audit Services (OAS) provides auditing services for HHS, 
either by conducting audits with its own audit resources or by overseeing 
Services 
audit work done by others.  Audits examine the performance of HHS 
programs and/or its grantees and contractors in carrying out their 
respective responsibilities and are intended to provide independent 
assessments of HHS programs and operations.  These assessments help 
reduce waste, abuse, and mismanagement and promote economy and 
efficiency throughout HHS. 
Office of Evaluation  The Office of Evaluation and Inspections (OEI) conducts national 
evaluations to provide HHS, Congress, and the public with timely, useful, 
and Inspections 
and reliable information on significant issues.  These evaluations focus on 
preventing fraud, waste, or abuse and promoting economy, efficiency, and 
effectiveness of departmental programs.  To promote impact, OEI reports 
also present practical recommendations for improving program 
operations.   
Office of  The Office of Investigations (OI) conducts criminal, civil, and 
administrative investigations of fraud and misconduct related to HHS 
Investigations 
programs, operations, and beneficiaries.  With investigators working in all 
50 States and the District of Columbia, OI utilizes its resources by actively 
coordinating with the Department of Justice and other Federal, State, and 
local law enforcement authorities.  The investigative efforts of OI often 
lead to criminal convictions, administrative sanctions, and/or civil 
monetary penalties. 
Office of Counsel to  The Office of Counsel to the Inspector General (OCIG) provides general 
legal services to OIG, rendering advice and opinions on HHS programs 
the Inspector 
and operations and providing all legal support for OIGs internal 
General 
operations.  OCIG represents OIG in all civil and administrative fraud and 
abuse cases involving HHS programs, including False Claims Act, program 
exclusion, and civil monetary penalty cases.  In connection with these 
cases, OCIG also negotiates and monitors corporate integrity agreements.  
OCIG renders advisory opinions, issues compliance program guidance, 
publishes fraud alerts, and provides other guidance to the health care 
industry concerning the anti-kickback statute and other OIG enforcement 
authorities.