GAO 

United States Government Accountability Office

Report to the Ranking Member, 

Committee on Finance, U.S. Senate 

NATIONAL 

INSTITUTES OF 

HEALTH 

Completion of 

Comprehensive Risk 

Management Program 

Essential to Effective 

Oversight 

September 2009 

GAO-09-687 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

September 2009

NATIONAL INSTITUTES OF HEALTH

Completion of Comprehensive Risk Management 

Program Essential to Effective Oversight 

Highlights of GAO-09-687, a report to the 

Ranking Member, Committee on Finance, 

U.S. Senate 

The National Institutes of Health 

(NIH), an agency of the 

Department of Health and Human 

Services (HHS), is the primary 

federal agency for supporting 

medical research. The Office of the 

Director (OD) is the central NIH 

office responsible for setting policy 

and overseeing NIH’s 27 institutes 

and centers (IC). Allegations 

involving one institute raised 

questions about areas of oversight 

by the OD. In light of these 

questions, GAO examined (1) how 

NIH makes extramural research 

funding decisions and OD 

monitoring of this process, (2) the 

design of selected internal controls 

over NIH’s travel and personnel 

appointment processes, and (3) the 

design of NIH’s new risk 

management program and the 

program it is replacing. To address 

these objectives, GAO reviewed 

relevant NIH policies, procedures, 

and supporting documentation. 

GAO also selected 3 institutes that 

varied in size for in-depth reviews. 

What GAO Recommends

To help improve oversight, GAO 

made three recommendations to 

the Director of NIH: (1) monitor 

the extent to which IC directors 

use discretion in funding decisions, 

(2) add key components to the 

Enterprise Risk Management 

Program, and 

(3) ensure implementation of the 

program. HHS disagreed with the 

first recommendation, partially 

concurred with the second 

recommendation, and identified a 

final date for implementation of the 

program. 

NIH is required by law to make its extramural research funding decisions—

funding provided to scientists external to NIH such as those at universities—

using a dual peer review system. During the first level, initial peer review 

groups assess applications and assign a score to them based on their scientific 

merit. During the second level, advisory councils review the applications and 

their scores and, on the basis of this review, recommend to the ICs certain 

applications for funding consideration. IC directors can use their discretion 

and choose to fund applications based on factors in addition to scientific 

merit, “skipping” over applications with higher scores or making “exceptions” 

to fund applications with lower scores. GAO found that in fiscal year 2007, IC 

directors funded about 19 percent of NIH’s applications for a common type of 

grant based on factors in addition to scientific merit. However, the NIH OD 

does not monitor the extent to which IC directors use such discretion when 

making extramural funding decisions—an action that would be consistent 

with federal internal control standards. 

The NIH OD has established policies and procedures that incorporate key 

internal controls into the travel and personnel appointment processes. For 

example, the processes require multiple levels of review and approval. 

However, there is not an NIH-wide process for risk-based monitoring of the 

effectiveness of controls. Without monitoring actual implementation of 

controls based on assessed risk levels, NIH does not have adequate assurance 

that controls are operating as intended within those areas that have been 

identified as posing risks to the agency’s ability to achieve its mission. 

NIH’s Management Control Program, a risk management program updated in 

2004, did not comprehensively address risks to the agency’s overall operations 

and resulted in a lack of sufficient information for effective oversight and 

agencywide risk management. Recognizing this, in 2006, NIH began designing 

a new risk management program, the Enterprise Risk Management Program. 

Although an improvement over the earlier program, the design of the new 

program does not fully address the components identified in GAO’s 

framework for effective risk management. For example, the design does not 

incorporate strategic goals and objectives as a precondition for risk 

management, the evaluation of alternative responses to address identified 

risks, or documentation of the rationale for selecting a risk response. Further, 

NIH’s new program is not yet fully implemented, despite an over 3-year effort. 

According to NIH officials, NIH has experienced delays because of a change in 

contractors, balancing staff resources with competing demands, and 

underestimating time needed for implementation. 

View GAO-09-687 or key components.

For more information, contact Linda T. Kohn 

at (202) 512-7114 or kohnl@gao.gov or 

Susan Ragland at (202) 512-8486 or 

raglands@gao.gov. 
Page i GAO-09-687 NIH Oversight 

Contents 

Letter 1

Background 5 

NIH Is Required to Use a Peer Review System to Make Extramural 

Funding Decisions; NIH’s OD Does Not Monitor Key Decisions 

in which IC Directors Exercise Their Discretion Over Funding 9 

Design of NIH’s Travel and Personnel Appointment Processes 

Includes Key Control Activities and Some Monitoring Activities 

but Lacks Systemic Risk-Based Monitoring 15 

NIH’s Management Control Program and Enterprise Risk 

Management Program Do Not Fully Address Key Components of 

Effective Risk Management 17 

Conclusions 25 

Recommendations for Executive Action 26 

Agency Comments and Our Evaluation 27 

Appendix I Scope and Methodology 34 

Appendix II NIH Organization and Mission 39 

Appendix III Comments from the National Institutes of Health 45 

Appendix IV GAO Contacts and Staff Acknowledgments 62 

Tables 

Table 1: Extramural Research R01 Grant Applications Funded in 

Fiscal Years 2003 through 2007 13 

Table 2: GAO’s Risk Management Framework 19 

Table 3: Overview of ICs Including Establishment Date, Mission, 

and Fiscal Year 2008 Appropriation 42 
Figures 

Figure 1: Relationship of the Risk Management Framework 

Components 8 

Figure 2: National Cancer Institute’s Fiscal Year 2007 Payline for 

R01 Grant Applications 12 

Figure 3: Organizational Structure of NIH 40 

Abbreviations 

CSR Center for Scientific Review 

COSO Committee of Sponsoring Organizations 

GPRA Government Performance and Results Act 

HHS Department of Health and Human Services 

IC institutes and centers 

NCI National Cancer Institute 

NIAAA National Institute on Alcohol Abuse and Alcoholism 

NIDDK National Institute of Diabetes and Digestive and Kidney 

 Diseases 

NIEHS National Institute of Environmental Health Sciences 

NIH National Institutes of Health 

OD Office of the Director 

OMB Office of Management and Budget 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately. 

Page ii GAO-09-687 NIH Oversight 
Page 1 GAO-09-687 NIH Oversight 

 

United States Government Accountability Office

Washington, DC 20548 

 

September 11, 2009 

The Honorable Charles E. Grassley 

Ranking Member 

Committee on Finance 

United States Senate 

Dear Senator Grassley: 

The National Institutes of Health (NIH) is the primary federal agency for 

supporting medical research in the United States. In fiscal year 2008, NIH 

provided $24.4 billion—83 percent of its $29.5 billion budget—in 

extramural research funding, which supports scientists and research 

personnel working at universities, medical schools, and other research 

institutions.1

 NIH’s extramural research funding efforts reflect its large, 

decentralized organization. NIH comprises 27 institutes and centers (IC) 

and an Office of the Director (OD). Each of the ICs has its own budget, 

mission, and staff and focuses on particular diseases or research areas, 

such as cancer or aging issues. Twenty-four of the 27 ICs fund extramural 

research, each with a separate appropriation,2

 and these ICs make final 

decisions on which extramural research projects to fund following a 

standard process defined by law and NIH policy. As the central office at 

NIH, the OD establishes NIH policy and is responsible for overseeing the 

ICs, including their research funding efforts and their various 

administrative functions, such as hiring personnel and approving 

personnel travel. The OD’s oversight responsibilities have grown over the 

years. Between 1985 and 2000, 7 of the 27 ICs were created—and these 

additions have helped to increase the overall complexity of overseeing the 

ICs. More recently, under the American Recovery and Reinvestment Act of 

2009, NIH received $10.4 billion that NIH plans to use in 2009 and 2010 to 

fund extramural and other research and support the construction, 

renovation, and repair of certain research facilities. 

1

NIH also supports intramural research, which is performed by NIH scientists in NIH 

laboratories. 

2

The three centers that do not fund extramural research and do not receive separate 

appropriations (Center for Scientific Review, Center for Information Technology, and the 

Clinical Center) are funded through the NIH Management Fund, which is funded using a 

portion of other NIH appropriations. See 42 U.S.C. § 290. 
We and others have raised questions about the OD’s ability to effectively 

oversee IC activities. For example, in April 2007, we reported that NIH had 

not established clear policies related to managing conflicts of interest 

among senior NIH employees who have decision-making responsibilities 

for NIH’s research efforts, 3

 which include NIH’s extramural research 

funding. We noted that such policies are part of NIH’s framework for 

ensuring the integrity of NIH-funded research and recommended that NIH 

clarify them. NIH agreed with our recommendation. In mid-2007 you raised 

questions over allegations of improper travel, personnel appointments, 

and extramural research funding decisions involving the director of one of 

NIH’s ICs, the National Institute of Environmental Health Sciences 

(NIEHS), which supports research on environmental influences on the 

development and progression of human disease. Similar questions 

prompted the House Committee on Appropriations to request that NIH 

conduct a management review of NIEHS, which found management and 

operational problems at the Institute.4

The above issues focus on how NIH makes extramural funding decisions 

and the quality of its internal control over administrative functions such as 

travel arrangements and personnel appointments. Internal control can 

include the establishment of safeguards, such as supervisory reviews, that 

are incorporated into agency work processes. According to federal 

standards, effectively designed and implemented internal control provides 

reasonable assurance that an agency’s operations are effective and 

efficient, its financial reporting reliable, and that the agency complies with 

applicable laws and regulations.5

 The issues at NIH also raise broader 

questions about NIH’s risk management, the process whereby an agency 

or organization systematically identifies risks associated with achieving its 

mission or objectives; assesses the magnitude of those risks; puts in place, 

when necessary, mitigating actions to address those risks; and then 

monitors the effectiveness of those actions. During our review, NIH was in 

the process of implementing its Enterprise Risk Management Program, a 

new risk management program that is replacing the NIH Management 

Control Program—the agency’s previous risk management program. 

 

3

GAO, NIH Conflict of Interest: Recusal Policies for Senior Employees Need Clarification, 

GAO-07-319 (Washington, D.C.: Apr. 30, 2007). 

4

H.R. Rep. No. 110-231, at 161-62 (2007); NIH Office of Management Assessment, 

Management Review: National Institute of Environmental Health Sciences (Apr. 9, 2008). 

5

GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1

(Washington, D.C.: November 1999). 

Page 2 GAO-09-687 NIH Oversight 
You asked us to examine NIH’s oversight of the ICs. Specifically, we 

agreed to provide information on NIH’s extramural research funding 

decisions, employee travel arrangements and hiring practices for certain 

employees, and NIH’s process for identifying and addressing potential 

risks to its operations. In this report we 

1. describe how NIH makes extramural research funding decisions and 

the extent to which the NIH’s OD monitors this process, 

2. review the design of selected internal controls over NIH’s travel and 

personnel appointment processes, and 

3. review the design of the NIH Management Control Program and the 

Enterprise Risk Management Program to determine if they contain key 

components of an effective risk management program. 

To address these objectives, we reviewed relevant NIH policies, 

procedures, and supporting documentation on (1) the process used across 

NIH for making extramural research funding decisions and efforts by the 

OD to monitor this process, (2) the design of key internal controls for 

employee travel and Title 42 personnel appointments6

—specifically, 

control and monitoring activities—and (3) the design of the NIH 

Management Control Program and the Enterprise Risk Management 

Program. We also selected 3 ICs—the National Cancer Institute (NCI), 

National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), 

and National Institute on Alcohol Abuse and Alcoholism (NIAAA)—for 

more in-depth reviews of the process used across NIH for making 

extramural research funding decisions and for more in-depth reviews of 

the design of the ICs’ control and monitoring activities for travel and Title 

42 personnel appointment processes. We selected these 3 ICs because they 

vary in size and focus on different disease-specific research missions. We 

interviewed officials from the NIH OD and the selected ICs to clarify our 

understanding of the process used for making extramural research funding 

decisions and the OD’s monitoring of this process. We also collected data 

 

6

Under two provisions of title 42 United States Code, NIH has additional hiring flexibilities 

not permitted under title 5 authorities related to the general schedule and senior executive 

service. These flexibilities are referred to as “title 42” personnel appointments. Specifically, 

title 42 authorities allow NIH to hire scientists at salary levels comparable to those outside 

of the federal government. In 2008, under these authorities, NIH could hire scientists with 

salary levels up to $250,000. However, maximum pay for the general schedule was $149,000 

and for the senior executive service was $172,200 in 2008. See 42 U.S.C. § 209 (f), (g). 

Page 3 GAO-09-687 NIH Oversight 
on funding decisions for each of the 24 ICs that fund extramural research.7

We performed walkthroughs8

 at the 3 selected ICs and interviewed 

officials from the NIH OD and the selected ICs to clarify our understanding 

of the design of the ICs’ control and monitoring activities for travel and 

Title 42 personnel appointment processes. We also interviewed officials 

from the NIH OD to further our understanding of the NIH Management 

Control Program and the Enterprise Risk Management Program. 

As part of our review, we compared the OD’s monitoring of the process 

used for making extramural research funding decisions and the design of 

the control and monitoring activities at the three selected ICs to GAO’s 

Standards for Internal Control in the Federal Government.

9

 In reviewing 

the design of the NIH Management Control Program and the Enterprise 

Risk Management Program, we compared these designs to our framework 

for effective risk management.10 The scope of our audit did not include 

testing the implementation of internal control over travel and Title 42 

personnel appointments. Furthermore, we did not review the 

implementation of either the NIH Management Control Program or the 

Enterprise Risk Management Program because, at the time of our review, 

NIH did not plan to continue the Management Control program and the 

Enterprise Risk Management Program was not yet fully implemented. 

 

7

Specifically, we reviewed funding decisions made for R01 grants, the original grant 

mechanism used by NIH, which is a common type of grant awarded to organizations of all 

types (universities, colleges, small businesses, for-profit, foreign and domestic, etc.) to 

support a discrete, specified project to be performed by a specific investigator or group of 

investigators. 

8

A walkthrough is a method used to develop an understanding of key processes and 

controls in which an auditor traces a transaction through the organization’s procedures. 

9

See GAO/AIMD-00-21.3.1. 

10See table 2 for GAO’s framework for effective risk management. GAO developed the 

framework based on authoritative literature and standards, as well as previous GAO 

reports and testimonies. We consulted the Government Performance and Results Act 

(GPRA) of 1993; the Government Auditing Standards, 2003 Revision; GAO’s Standards for 

Internal Control in the Federal Government (November 1999); guidance from the Office of 

Management and Budget (OMB); the work of the President’s Commission on Risk 

Management; consulting papers; and the enterprise risk management approach of the 

Committee of Sponsoring Organizations (COSO) of the Treadway Commission. We 

reviewed numerous frameworks from industry, government, and academic sources. GAO, 

“Appendix I: A Risk Management Framework” of Risk Management: Further Refinements 

Needed to Assess Risks and Prioritize Protective Measures at Ports and Other Critical 

Infrastructure, GAO-06-91 (Washington, D.C.: Dec. 15, 2005). 

Page 4 GAO-09-687 NIH Oversight 
Appendix I includes additional details on our scope and methodology. We 

conducted this performance audit from March 2008 to September 2009, in 

accordance with generally accepted government auditing standards. Those 

standards require that we plan and perform the audit to obtain sufficient, 

appropriate evidence to provide a reasonable basis for our findings and 

conclusions based on our audit objectives. We believe that the evidence 

obtained provides a reasonable basis for our findings and conclusions 

based on our audit objectives. 

As the primary federal agency for supporting medical research in the 

United States, NIH’s mission is “science in pursuit of fundamental 

knowledge about the nature and behavior of living systems and the 

application of that knowledge to extend healthy life and reduce the 

burdens of illness and disability.” NIH is headed by a Director who is 

supported by staff and program offices within the OD and 27 ICs. Each of 

the ICs has its own director and staff. Each IC director reports to the OD.11

Appendix II provides more information about NIH’s organizational 

structure. 

Background 

NIH’s ICs were created over time, with each having an explicit mission 

focused on a particular disease, organ system, stage of development, or 

cross-cutting mission, such as providing scientists and researchers with 

the tools they need to understand, detect, treat, and prevent a wide range 

of diseases. The first institute, NCI, was created in 1937, and the newest 

institute, National Institute of Biomedical Imaging and Bioengineering, 

was created in 2000. 

Internal Control Internal control is an integral part of managing an agency.12 It comprises 

the plans, methods, and procedures used to meet missions, goals, and 

objectives. Effectively designed and implemented internal control 

provides management with reasonable assurance that the following 

objectives are being achieved: (1) effectiveness and efficiency of 

operations, (2) reliability of financial reporting, and (3) compliance with 

 

11The Director of NIH is appointed by the President, with Senate confirmation. The 

President also appoints the director of NCI, while the Secretary of the Department of 

Health and Human Services (HHS) appoints the other IC directors. 

12See GAO/AIMD-00-21.3.1. 

Page 5 GAO-09-687 NIH Oversight 
applicable laws and regulations.13 Internal control serves as the first line of 

defense in preventing and detecting errors and fraud. The following five 

elements of internal control provide the basis against which internal 

control is evaluated. 

• Control Environment—Sets the tone for an organization and is the 

foundation for all other standards. Management and employees should 

establish and maintain an environment throughout the organization that 

sets a positive and supportive attitude toward internal control and 

conscientious management. Among others, control environment includes 

management’s integrity and ethical values, commitment to competence, 

philosophy and operating style, and organizational structure. 

• Risk Assessment—The identification and analysis of relevant risks 

associated with achieving the objectives and forming a basis for 

determining how risks should be managed. This standard includes an 

assessment of the risks the agency faces from both external and internal 

sources. 

• Control Activities—The policies, procedures, techniques, and mechanisms 

that enforce management’s directives. Control activities occur at all levels 

and functions of the agency and include a wide range of diverse activities 

such as approvals, authorizations, verifications, and reconciliations. 

• Information and Communication—Information should be recorded and 

communicated to management and others within the entity who need it 

and in a form and within a time frame that enables them to carry out their 

internal control and other responsibilities. In addition to internal 

communications, management should ensure there are adequate means of 

communicating with, and obtaining information from, external 

stakeholders that may have a significant impact on the agency achieving 

its goals. 

• Monitoring—Includes ongoing monitoring in the course of normal 

operations (e.g., regular management and supervisory activities, 

comparisons, and reconciliations) and risk-based monitoring that includes 

separate evaluations of controls’ effectiveness whose scope and frequency 

depends primarily on the assessment of risks and effectiveness of ongoing 

monitoring procedures. 

 

13See 31 U.S.C. § 3512(c). 

Page 6 GAO-09-687 NIH Oversight 
One way to help ensure that internal control is continuously monitored 

and improved is through risk management. Risk management helps 

agencies to identify the most significant areas in which to place or 

enhance controls.14 Additionally, based on the assessment of risk that is 

performed as part of an overall risk management program, agencies can 

determine the scope and frequency of control evaluations. Risk 

management is a continuous process whereby an organization 

systematically identifies risks associated with achieving its objectives; 

assesses the magnitude of those risks; puts in place, when necessary, 

mitigating actions to address those risks; and then monitors the 

effectiveness of those actions taken. In addition, because governmental, 

economic, industry, regulatory, and operating conditions continually 

change, risk management provides a mechanism to identify and deal with 

any special risks prompted by such changes. While risk management 

programs do not provide absolute assurance regarding the achievement of 

an agency’s objectives, an effective risk management program can be 

particularly useful in a decentralized organization to help top management 

identify potential problems and allocate limited resources using a 

reasonable basis (such as risk). 

Risk Management 

In 2005, GAO identified risk management as an area of increasing concern, 

particularly with regard to the need for the completion of threat and risk 

assessments in a variety of areas.15 To help address the concern, GAO 

developed a framework for effective risk management activities in the 

federal government based on best practices and authoritative literature.16

This framework includes five components that define a risk management 

 

14Risk management does not replace, but rather incorporates and expands on internal 

control. Thus, risk management provides a more robust and extensive focus to effectively 

identify, assess, and manage risk. 

15GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.: January 2005). 

16See table 2. GAO developed the framework based on authoritative literature and 

standards, as well as previous GAO reports and testimonies. We consulted the Government 

Performance and Results Act (GPRA) of 1993; the Government Auditing Standards, 2003 

Revision; GAO’s Standards for Internal Control in the Federal Government (November 

1999); guidance from the Office of Management and Budget (OMB); the work of the 

President’s Commission on Risk Management; consulting papers; and the enterprise risk 

management approach of the Committee of Sponsoring Organizations (COSO) of the 

Treadway Commission. We reviewed numerous frameworks from industry, government, 

and academic sources. GAO, “Appendix I: A Risk Management Framework” of Risk 

Management: Further Refinements Needed to Assess Risks and Prioritize Protective 

Measures at Ports and Other Critical Infrastructure, GAO-06-91 (Washington, D.C.: 

Dec. 15, 2005). 

Page 7 GAO-09-687 NIH Oversight 
program for federal agencies: strategic goals, objectives, and constraints; 

risk assessment; alternatives evaluation; management selection; and 

implementation and monitoring. For the purposes of our analysis of NIH’s 

program, we also considered two additional components, internal 

environment and information and communications, based on guidance and 

standards on risk management and internal controls.17 Figure 1 illustrates 

the interrelationship of these seven components. The components of the 

framework should operate within an internal environment that supports 

the other components, and pertinent information should be communicated 

between and among internal and external stakeholders as well as 

personnel responsible for carrying out the duties associated with each of 

the components. 

with each of 

Figure 1: Relationship of the Risk Management Framework Components 

Source: GAO.

Strategic goals,

objectives,

and constraints

Risk

assessment

Alternatives

evaluation

Management

selection

Implementation

and

monitoring

INTERNAL ENVIRONMENT

INFORMATION &

COMMUNICATION

 

17The Committee of Sponsoring Organizations of the Treadway Commission, Enterprise 

Risk Management—Integrated Framework (Jersey City, N.J.: American Institute of 

Certified Public Accountants, September 2004) and GAO/AIMD-00-21.3.1. 

Page 8 GAO-09-687 NIH Oversight 
NIH is required by law to use a peer review system in its process for 

making extramural research funding decisions. NIH’s dual peer review 

system is designed to help ensure the objective evaluation of the scientific 

merit of applications for extramural funding. After NIH’s peer review 

process is concluded, IC directors have discretion when making final 

extramural funding decisions and are not required to fund applications 

based strictly on the scores resulting from the evaluation of their scientific 

merit. We found that in fiscal year 2007 IC directors decided to fund about 

19 percent of NIH’s applications for RO1 grants, a common type of grant, 

based on factors in addition to these scores. However, NIH’s OD does not 

monitor extramural funding decisions in which the IC Directors exercise 

their discretion. 

NIH Is Required to 

Use a Peer Review 

System to Make 

Extramural Funding 

Decisions; NIH’s OD 

Does Not Monitor Key 

Decisions in which IC 

Directors Exercise 

Their Discretion Over 

Funding 

By Law, NIH Must Use a 

Dual Peer Review System 

Designed to Evaluate 

Scientific Merit of 

Extramural Funding 

Applications 

NIH is required by law to use a peer review system in its process for 

making extramural research funding decisions. This system comprises two 

sequential levels of peer review by panels of experts in various fields of 

research who help NIH identify the most promising extramural grant 

applications to fund, as defined primarily by an assessment of the 

applications’ technical and scientific merit.18 According to NIH, compared 

to a single level of peer review, the dual peer review system allows for 

multiple reviews and therefore a more objective evaluation of the 

scientific merit of grant applications. 

Applications for NIH’s extramural funding are received by NIH’s Center for 

Scientific Review (CSR), which is responsible for assigning each 

application to the first level of peer review. The first level of peer review is 

conducted by what are known as initial peer review groups, to which CSR 

assigns applications for review, based on the applications’ proposed area 

of research and the initial peer review groups’ area of expertise. These 

initial peer review groups specialize in various research areas such as 

cancer or digestive disorders and are composed of scientists, who are 

 

18See 42 U.S.C. §§ 282(b)(9); 289a(a); 289a-1(a)(2). The Secretary of HHS promulgated 

regulations expanding on the use of peer review by groups appointed by the Director of 

NIH and the directors of the national research institutes. 

Page 9 GAO-09-687 NIH Oversight 
often recognized as experts in their field.19 Each group meets three times 

per fiscal year to review grant applications. 

The initial peer review groups are responsible for identifying the most 

promising applications for funding, based on an assessment of the 

applications’ scientific merit.20 The groups review the applications 

assigned to them and assess their scientific merit, using criteria that 

require reviewers to examine such components as a grant application’s 

design and methodology, innovation, and scientific significance.21 Using 

these criteria, the initial peer review groups assign a priority score to the 

applications they reviewed, which are used to rank the applications from 

among those in the cohort of applications. After the applications are 

scored and ranked, the information is forwarded to the appropriate IC—

based on the applications’ proposed area of research—for the second level 

of peer review. 

Each IC that funds extramural research has its own advisory council, 

which conducts the second level of peer review.22 Advisory councils 

consist of no more than 18 voting members, two-thirds of whom are 

scientists in the research areas of the IC and one-third of whom are 

leaders of non-science fields.23 Advisory councils meet at least three times 

 

19The composition of the initial peer review groups is specified in 42 C.F.R. § 52h.4 (2008). 

Based on these criteria, NIH staff select the initial peer reviewers, who generally agree to 

participate for 4 years. 

20See 42 C.F.R. § 52h.7 (2008). 

2142 C.F.R. § 52h.8 (2008) directs peer review groups to assess each proposed research 

project taking into account the following criteria, among other pertinent factors: (a) its 

significance, (b) the adequacy of its approach and methodology, (c) its innovativeness and 

originality, (d) the qualifications and experience of its principal investigator and staff, 

(e) the scientific environment and reasonable availability of resources for it, (f) the 

adequacy of its plans to include both genders, minorities, children, and special populations 

as appropriate for its scientific goals, (g) the reasonableness of its budget and duration, and 

(h) the adequacy of its protections for humans, animals, and the environment. 

2242 U.S.C. § 284a. Although the law setting forth the requirements for advisory councils is 

specific to institutes, each center that funds extramural research has an advisory council 

substantially similar to those of the institutes. See 42 U.S.C. §§ 287a (National Center for 

Research Resources), 287c-21(b) (National Center for Complementary and Alternative 

Medicine), 287c-31(j) (National Center on Minority Health and Health Disparities). 

23Advisory councils also include ex officio members, who are nonvoting. Voting members 

generally serve 4-year terms. At the NCI, the President appoints voting advisory council 

members, and the members serve 6-year terms. For all other advisory councils, the 

Secretary of HHS appoints voting members. 

Page 10 GAO-09-687 NIH Oversight 
per fiscal year.24 Under law and NIH policy, the advisory councils are 

responsible for reviewing the applications and their priority scores and, on 

the basis of this review, recommending to the ICs certain applications for 

funding consideration. The advisory council may ask for applications to be 

scored a second time, such as if they have questions about whether the 

scientific criteria were applied appropriately. NIH advisory council 

members we interviewed noted that councils only have time to discuss a 

few applications individually, so they consider many applications in large 

groups, particularly in cases when no concerns are apparent about the 

applications or their priority scores. Based on data we reviewed, we found 

that from fiscal year 2003 to 2007, in most cases, only a small number of 

applications were not recommended by advisory councils for funding 

consideration. The advisory councils’ recommendations conclude NIH’s 

peer review process. 

IC Directors Have 

Discretion to Make Final 

Extramural Funding 

Decisions, but NIH’s OD 

Does Not Monitor 

Decisions in Which IC 

Directors Exercise This 

Discretion 

After NIH’s peer review process has been concluded, the director of each 

IC is responsible for making final extramural funding decisions. In 

deciding which applications to fund, the IC directors choose applications 

from among those recommended for funding consideration by the 

advisory council.25 In general, IC directors select applications for funding 

based on their priority scores, which reflect the evaluation of the 

applications’ scientific merit by NIH’s peer review process. For each fiscal 

year, each IC establishes a funding line—known as the payline—which 

roughly corresponds with the number of extramural grant applications the 

IC will be able to fund that year. The payline for any given year is based on 

projections of the total funding available at the IC that year for grants, the 

average dollar amount expected to be awarded per application, and the 

number of applications received by the IC. For example, as shown in 

figure 2, based on the amount of funding available for extramural grants, 

NCI set its payline for R01 grants26—the most common grant category—at 

 

24By law, the advisory councils for NCI and the National Heart, Lung, and Blood Institute 

must meet at least four times per fiscal year. 42 U.S.C. § 284a(h)(2). 

25NIH may not approve or fund any application unless it has been recommended for 

approval by a majority of the members of the initial peer review group and a majority of the 

voting members of the advisory council. The initial peer review groups recommend 

applications for approval via the scoring system. 42 U.S.C. § 289a-1(a)(2). 

26The R01 grant is the original grant mechanism used by NIH. This type of grant is awarded 

to organizations of all types (universities, colleges, small businesses, for-profit, foreign and 

domestic, etc.) to support a discrete, specified project to be performed by a named 

investigator or investigators. 

Page 11 GAO-09-687 NIH Oversight 
the 15th percentile for fiscal year 2007. This means that NCI expected to 

have sufficient funding at a minimum for all of the applications with scores 

in the top 15 percent. In general, IC directors fund only those applications 

with priority scores above the fiscal year’s payline. 

Figure 2: National Cancer Institute’s Fiscal Year 2007 Payline for R01 Grant 

Applications 

Source: GAO.

Payline

Above the payline

Below the payline

National Cancer 

Institute’s

payline: 

15th percentile

0

100

15

▲

Application

order

based

on

percentile

ranking

▼

656 applications 

scored above the 

15th percentile.

3,108 applications 

scored below the 

15th percentile.

Note: Figure includes only applications that were recommended for funding consideration by an initial 

peer review group and the advisory council. The portion of applications that scored above the payline 

appears to be greater than 15 percent because applications that were considered for funding multiple 

times during the year and scored below the 15th percentile each time are only counted in this data 

once. 

Page 12 GAO-09-687 NIH Oversight 
While the IC directors only fund projects recommended by their advisory 

councils and typically work within the paylines, ultimately they have 

discretion to make final extramural funding decisions. In particular, 

directors are not required to fund applications based strictly on the 

applications’ priority scores or the payline. In some instances, IC directors 

decide not to fund applications that scored above the fiscal year’s payline, 

such as when the applications duplicate research that has already received 

IC funding. These applications are called “skips.” For example, of the 656 

applications that scored above NCI’s payline in fiscal year 2007, 3 

applications were skipped. Similarly, though the IC directors typically do 

not decide to fund applications with priority scores that fall below the 

fiscal year’s payline, in some cases they do. These applications are known 

as “exceptions.” For example, of the 3,108 applications that scored below 

NCI’s payline in 2007, 137 were funded as exceptions. In the case of 

exceptions, the IC directors may exercise their discretion and choose to 

fund these applications based on factors in addition to the applications’ 

priority scores. These factors can include efforts to support the IC or 

NIH’s research priorities. When skipping applications or funding 

applications as exceptions, IC directors are required under NIH policy to 

document the corresponding rationale used. 

In reviewing the IC data, we found that 18.5 percent of NIH’s funded R01 

grant applications were funded as exceptions in fiscal year 2007, as shown 

in table 1. These applications had scientific merit scores that were below 

the payline for their respective ICs and thus were funded based on factors 

in addition to their scientific merit scores. This represents a substantial 

increase from 9.7 percent of funded applications that were exceptions in 

fiscal year 2003. 

Table 1: Extramural Research R01 Grant Applications Funded in Fiscal Years 2003 through 2007 

Fiscal year 2003 2004 2005 2006 2007

Total number of applications funded 6,461 6,167 5,731 5,408 5,715

Number of applications funded above the payline 5,836 5,639 5,159 4,788 4,656

Number of applications funded below the payline – exceptions 625 528 572 620 1,059

Percentage of applications funded below the payline – exceptions 9.7 8.6 10.0 11.5 18.5

Source: GAO analysis of NIH grants data. 

Page 13 GAO-09-687 NIH Oversight 
Documentation that we reviewed from three of the ICs—NCI, NIDDK, and 

NIAAA—showed that IC directors funded applications as exceptions for 

various reasons. For example, IC officials cited the NIH-wide initiative to 

fund new investigators as one of the most frequent reasons for making the 

exceptions. In addition, IC officials told us that they funded applications as 

exceptions in order to maintain a diverse portfolio of research topics. 

NIH’s OD collects information on some aspects of the extramural research 

process. For example, the NIH OD collects information on the number of 

extramural grants funded by each IC; the percentage of applications that 

receive funding; and the priority rankings, by percentile, of funded 

applications. The NIH OD also targets some of these collection efforts 

towards specific types of extramural grants. For example, as part of its 

effort to support new investigators, the NIH OD has been collecting data 

on the number of extramural grants awarded to new investigators. 

Although NIH’s OD collects some information on the extramural research 

process, it does not monitor key funding decisions made by IC directors—

specifically, the instances in which IC directors exercise their discretion to 

make skips or exceptions to the funding payline. Skips and exceptions 

represent an area of potential risk because IC directors have latitude in 

making these decisions; monitoring these decisions would be consistent 

with federal internal control standards. 27 Although ICs are required to 

document the rationales used when skipping applications or funding 

applications as exceptions, the ICs are not required to provide the NIH OD 

with this documentation, and the NIH OD does not collect it. As a result, 

NIH’s OD does not have information on the number of applications 

skipped or funded as exceptions and the reasons for these decisions. 

 

27See GAO, GAO/AIMD-00-21.3.1. The Department of Health and Human Services, of which 

NIH is a component, is required to establish and maintain an effective system of internal 

control, consistent with the standards prescribed by the Comptroller General. 31 U.S.C. 

§ 3512(c), (d). 

Page 14 GAO-09-687 NIH Oversight 
With regard to the design of NIH-wide travel processes, NIH OD has 

established policies and procedures to help ensure that federal travel 

regulations are followed with regard to issues such as premium class 

travel, per diem expenses, and travel paid by third parties. The key control 

and monitoring activities for travel include reviews and approvals which 

take place during two stages—authorization and voucher—of the process. 

During the authorization stage, the traveler receives approval to travel 

based on the supervisor’s approval of the mission-relatedness of the trip 

and an administrative official’s approval of the method of transportation 

used, the cost estimates set forth for travel expenses, and the availability 

of funds for reimbursement to the traveler. During the voucher stage, the 

traveler’s voucher for reimbursement of travel expenses is approved based 

on an administrative official’s review of the voucher package which 

includes the traveler’s certification of the voucher and required receipts 

for travel expenses. 

Design of NIH’s Travel 

and Personnel 

Appointment 

Processes Includes 

Key Control Activities 

and Some Monitoring 

Activities but Lacks 

Systemic Risk-Based 

Monitoring 

The NIH OD has also established policies and procedures to help ensure 

that Title 42 personnel appointment decisions are appropriate. The design 

of NIH-wide key control activities for Title 42 personnel appointments 

includes reviews and approvals which take place during three stages—

resource determination, appointment selection, and compensation—of the 

process. During the resource determination stage, the IC’s selecting 

official identifies a hiring need and the administrative official determines 

whether necessary resources are available to meet the hiring need. During 

the appointment selection stage, the IC completes the recruitment 

including receiving applications, conducting candidate interviews, and 

making a tentative selection. For some positions, the NIH Offices of 

Human Resources, Intramural Research, and Extramural Research also 

play a role in preparing the recruitment and approving the selected 

candidate.28 During the compensation stage, final approval for the 

appointment and compensation is given depending upon the position and 

salary level for the candidate. Specifically, if the proposed compensation is 

below the lowest third of a given position’s salary range, then the IC 

director makes the final approval; if the proposed compensation is above 

the lowest third of a given position’s salary range but still within the range, 

then the IC director makes the final approval based on a recommendation 

from an IC committee; and if the proposed compensation is above the 

 

28Examples of positions that would require more involvement from the NIH-level offices 

include senior-level employees such as tenure-track investigators, senior investigators, 

senior scientists, and senior clinicians. 

Page 15 GAO-09-687 NIH Oversight 
salary range or other specified limits, then the NIH director makes the 

final approval based on recommendations from an IC committee, the IC 

director, and an NIH-wide committee. 

Overall, we found that the design of the controls included in the NIH-wide 

processes over travel and Title 42 personnel appointments included key 

controls necessary to help ensure these activities were being carried out 

appropriately, except in one key area related to the lack of requirements 

for risk-based monitoring. While controls may appear adequate based on 

written policies and procedures, without monitoring actual 

implementation based on the assessed risk levels, NIH does not have 

adequate assurance that controls are operating as intended within those 

areas that pose risk. NIH policy did not require the ICs to perform 

monitoring that includes risk-based control evaluations. Further, although 

NIH policy required a flexible plan for NIH-wide control evaluations that 

would generally target high- and medium-risk areas for review, according 

to NIH OD officials, such reviews have not been performed for over 

3 years because they do not have staff to perform these reviews. 

At NIH OD and two of the three ICs we reviewed—NCI and NIAAA—we 

found that some monitoring activities were performed over travel and Title 

42 personnel appointments. However, these monitoring activities were 

either not part of systemic risk assessment efforts that lead to subsequent 

monitoring based on assessed risk or not performed on a consistent and 

ongoing basis. Specifically, 

• Because of travel issues previously identified by GAO,29 HHS requires each 

of its operating divisions, which includes NIH, to perform quarterly control 

evaluations of travel cards.30 As a result of this requirement, each quarter 

the NIH OD selected a sample of travel transactions from across the ICs 

and tested compliance with federal travel regulations and NIH policies and 

procedures. For example, during each of the first 2 quarters of fiscal year 

2008, the NIH OD found problems with about 20 percent of the 100 sample 

items it tested. During the third quarter, the NIH OD found problems with 

about 30 percent of the 75 sample items it tested. Some of the problems 

found during these quarters included over- or underpayment to travelers, 

 

29GAO, Department of Health and Human Services: Controls Over Travel Program Are 

Generally Effective, but Some Improvements Are Needed, GAO-03-334 (Washington, D.C.: 

Feb. 21, 2003). 

30Travel cards are a type of charge card used for official travel-related expenses. 

Page 16 GAO-09-687 NIH Oversight 
failure of travelers to take advantage of lodging tax exemptions, and 

misuse of travel cards. The NIH OD required follow-up actions such as 

reimbursement of overpayment amounts and issuing additional guidance. 

However, these travel control evaluations were not part of a systemic 

process for assessing risk over operations and subsequently monitoring or 

evaluating controls based on assessed risk levels. 

• In 2008, NCI and NIAAA performed control evaluations over travel, and 

NCI performed a control evaluation of personnel appointments (including 

those under Title 42). These control evaluations were performed in 

response to prior audit findings, to prepare for upcoming audits or 

reviews, or to address concerns regarding process inefficiencies. However, 

they were not incorporated into the design of the processes and therefore 

were not performed on a consistent and ongoing basis. 

One of the three ICs we reviewed—NIDDK—had adopted its own riskbased program which consisted of assessing the risks over operational 

areas, including travel and personnel appointments, and subsequently 

monitoring the controls over those operational areas. The frequency of 

monitoring depended upon the risk level, and high-risk activities at NIDDK 

were scheduled to be monitored more frequently than low- or medium-risk 

activities. The design of NIDDK’s program represents a positive step 

towards an effective risk management program. Further details on a 

framework for an effective risk management program are discussed in the 

next section. 

The design of the Management Control Program provided NIH with a 

limited ability to identify and address risks to the agency’s overall 

operations. Recognizing the need for improvement, in 2006, the NIH OD 

began redesigning its program. However, while an improvement over the 

Management Control Program, the new Enterprise Risk Management 

Program does not fully address all of the components of GAO’s framework 

for effective risk management. Further, NIH’s Enterprise Risk 

Management Program has not been fully implemented, despite an over 

3-year effort, and NIH had not yet established milestones for its full 

implementation. 

NIH’s Management 

Control Program and 

Enterprise Risk 

Management Program 

Do Not Fully Address 

Key Components of 

Effective Risk 

Management 

Page 17 GAO-09-687 NIH Oversight 
NIH’s Management Control 

Program Had Weaknesses 

NIH’s Management Control Program was initially implemented in 1999 and 

updated in 2004. Under the design of this program, risk assessments are 

performed that relate to specific management control areas, such as 

functional areas, systems, or processes (e.g., intramural research 

programs) without relating those areas to potential systemic or 

agencywide risks. If weaknesses are identified within the particular area 

being reviewed, the Management Control Program appropriately requires 

that corrective action plans be developed and implemented to correct the 

weakness and that such actions be monitored after implementation to 

ensure that the weakness has been corrected. 

As designed, NIH’s Management Control Program did not address several 

of the components and related key elements included in GAO’s framework 

for an effective risk management program. An effective risk management 

program should enable management to proactively identify, assess, and 

mitigate risks. Table 2 outlines the seven components of the risk 

management framework and the key elements within each of these 

components. 

Page 18 GAO-09-687 NIH Oversight 
Table 2: GAO’s Risk Management Framework 

Risk management 

component Description Key elements 

Strategic goals, objectives, 

and constraints 

Addresses what the strategic goals and 

objectives are attempting to achieve and the 

steps needed to attain these results, and 

considers the constraints under which an 

agency operates such as statute, higher 

level policy, budget, or other factors beyond 

management’s control that may affect an 

agency’s risk management plans. 

An agency’s risk management program should: 

• Require mission-based strategic goals and 

objectives, that are clearly articulated and 

measurable, to be set as a pre-condition for 

effective risk management. Without clearly 

identified strategic goals and objectives, an agency 

cannot effectively identify and address potential 

risks to its mission, prioritize risk, or identify criteria 

against which to measure performance. 

• Require agencies to identify constraints (e.g., 

legislative requirements or resources) that may limit 

effective risk management. 

Risk assessment Addresses the identification and evaluation 

of potential risks to an agency’s ability to 

achieve its goals and objectives so that 

management can design and implement 

responses to prevent or mitigate identified 

risks. 

An agency’s risk management program should: 

• Identify potential events which may adversely affect 

the agency, called risks, and evaluate the events 

based on likelihood of occurrence and impact. For 

example, an agency may identify and evaluate 

potential risks associated with economic and 

legislative changes, natural disasters, and criminal 

or terrorist activities. 

• Require continuous identification and evaluation of 

potential risks since governmental, economic, 

industry, legislative, and operating conditions 

continually change. 

Alternatives evaluation Addresses the identification and evaluation 

of alternative ways in which the agency can 

act to alter either the likelihood of 

occurrence or the impact of a potential risk. 

An agency’s risk management program should: 

• Identify alternative ways the agency can respond to 

prevent or mitigate an identified risk. For example, 

to comply with new legislation, an agency may 

need to revise existing policy and procedures or 

develop new policies and procedures. 

• Evaluate the alternatives identified to consider the 

effect on likelihood of occurrence and impact of a 

potential risk. 

• Evaluate the alternatives identified to consider the 

costs and benefits. 

Management selection Addresses the selection of a response to 

mitigate an identified risk based on the 

alternatives evaluated and management 

priorities, such as management’s attitude 

towards risk and how limited resources will 

be targeted. 

An agency’s risk management program should: 

• Require management to select and document an 

alternative, such as revising or creating a policy or 

procedure, for addressing an identified risk. 

• Require management to document the rationale for 

selecting the alternative. 

Page 19 GAO-09-687 NIH Oversight 
Risk management 

component Description Key elements 

Implementation and 

monitoring 

Addresses how risk responses will be 

applied and assessed to improve efficiency 

and effectiveness. In addition, addresses 

how the risk management program will be 

assessed to determine whether changes are 

needed to improve efficiency and 

effectiveness. 

An agency’s risk management program should: 

• Implement management’s selected alternative to 

address risk. 

• Periodically assess management’s selected 

alternative to address risk. 

• Periodically assess the efficiency and effectiveness 

of the entire risk management program. 

Internal environment Addresses how management will establish 

and maintain a positive environment that 

sets the tone throughout the agency and is 

the foundation upon which all other 

components of risk management operate. 

An agency’s risk management program should: 

• Include an agency’s risk management philosophy 

to help position the agency so that it can effectively 

recognize and manage risk. 

• Require oversight by a high-level senior body within 

the agency. 

• Incorporate the importance of integrity and ethical 

values to increase the effectiveness of the risk 

management program since the program and its 

results depend upon the personnel who carry out 

risk activities. 

• Include the way management assigns authority and 

responsibility to help ensure that risk 

responsibilities are carried out. 

• Hold managers accountable for their assigned 

duties in the risk management program. 

• Require management to organize its risk structure 

to provide a framework for the agency to plan, 

execute, control, and monitor risk activities. 

• Require management to initially train its personnel 

to help ensure that they have the necessary 

knowledge and skills to perform their assigned 

tasks. 

• Ensure management maintains competence of the 

agency’s personnel by providing for continuous 

training to update personnel on risk management 

practices and techniques. 

Information and 

communication 

Addresses the need to identify and 

communicate pertinent information in a form 

and timeframe that allows personnel to carry 

out their risk management responsibilities. 

An agency’s risk management program should: 

• Require pertinent information to be collected from 

and disseminated to relevant internal stakeholders 

in a form and timeframe consistent with the 

agency’s risk management needs. 

• Require pertinent information to be collected from 

and disseminated to relevant external stakeholders 

in a form and timeframe consistent with the 

agency’s risk management needs. 

Sources: GAO-06-91; The Committee of Sponsoring Organizations of the Treadway Commission, Enterprise Risk Management—

Integrated Framework (Jersey City, N.J.: American Institute of Certified Public Accountants, September 2004); and 

GAO/AIMD-00-21.3.1. 

Page 20 GAO-09-687 NIH Oversight 
The three components of the framework that the Management Control 

Program did not address are strategic goals, objectives, and constraints; 

risk assessment; and information and communication. Specifically, the 

program did not do the following. 

• Link the identification of potential risks with the agency’s strategic goals 

and objectives. The design of the Management Control Program did not 

require strategic goals and objectives to be set as a precondition for risk 

management. Without clearly identified strategic goals and objectives, an 

agency is limited in its ability to effectively identify and address potential 

risks to its mission, prioritize risk, or identify criteria against which to 

measure performance. 

• Require risk assessments be performed to identify and evaluate potential 

risks that could adversely affect NIH’s ability to achieve its objectives. The 

design of the Management Control Program called for evaluating the risks 

within specific functional areas, systems, or processes rather than 

assessing the risks that could adversely affect the agency as a whole. 

• Require pertinent information to be collected from and disseminated to 

relevant internal and external stakeholders in a form and time frame 

consistent with the agency’s overall risk management needs. The design of 

the Management Control Program allowed for inconsistent and 

incomparable information from the ICs, which can prevent management 

from effectively using the information to help ensure that agency 

objectives are met. 

For a number of years, NIH OD officials recognized that weaknesses 

existed in the agency’s Management Control Program, which resulted in a 

lack of sufficient information for effective oversight and agencywide risk 

management. For example, according to NIH OD officials, the program 

(1) did not hold managers responsible for their assigned duties in the risk 

management program, (2) did not require the ICs to communicate 

information in a form that allows NIH to effectively identify and manage 

risk across the agency, and (3) was not overseen by a high-level senior 

body, such as the Steering Committee.31 The three weaknesses NIH 

officials identified in the agency’s Management Control Program 

 

31The Steering Committee, which is chaired by the NIH director and composed of 10 IC 

directors who serve on a rotating basis, is NIH’s most senior-level governing body. The 

Steering Committee is responsible for addressing NIH-wide issues, other than those that 

relate to science. 

Page 21 GAO-09-687 NIH Oversight 
correspond to the following key components of our framework for 

effective risk management: 1) internal environment, 2) information and 

communication, and 3) internal environment, respectively. As a result of 

acknowledged shortcomings, in 2006 NIH began redesigning its risk 

management efforts. According to NIH OD officials, the new risk 

management program will improve the ability of the NIH OD to proactively 

identify and mitigate risks before they become obstacles to the NIH 

mission. However, as discussed later, NIH has not fully implemented the 

new Enterprise Risk Management Program and has encountered several 

obstacles in implementing initial phases of the program. 

NIH’s Enterprise Risk 

Management Program, 

while Improved, Does 

Not Fully Address 

Several Key Components 

of Effective Risk 

Management 

NIH began developing a new risk management program in 2006. The new 

program is designed to consist of a formal six-step methodology for 

managing risks.32 The six steps include: 

• Organize – Identify and train those charged with carrying out risk 

management activities, and define the risk management structure.33

• Identify and Score34 – Identify and score risks, review risks for quality and 

accuracy, and develop the risk baseline. 

• Assess – Document, analyze, and test processes and controls. 

• Remediate – Develop, review, approve, and execute corrective action 

plans. 

• Monitor – Monitor the risk baseline. 

• Report – Report risk information and results. 

 

32NIH Office of Management Assessment, NIH Enterprise Risk Management Program, 

Enterprise Risk Management Guidebook: A Step-By-Step Guide (March 2009, Draft). 

33Risk management structure is a segmentation of discrete, mission-oriented subsets of an 

organization to facilitate risk management activities at the lower level. 

34Scoring risks includes assessing the risk based on likelihood of occurrence and impact. 

Based on the assessment, risks are assigned a points value, which allows for quantitative 

comparison and ranking of risks across NIH. 

Page 22 GAO-09-687 NIH Oversight 
The design of the Enterprise Risk Management Program represents an 

improvement over the 2004 NIH Management Control Program in several 

key areas. Specifically, the new program will allow for improved 

identification, assessment, and mitigation of risks agencywide because it 

includes the following: 

• Risk assessments: The new program requires the identification of potential 

events that could adversely affect the agency and the evaluation of those 

events based on likelihood of occurrence and impact. 

• Oversight by a high-level senior body: The design requires the Steering 

Committee to oversee the new risk management program. 

• Information and communication: The design requires that pertinent 

information be collected from and disseminated to relevant internal 

stakeholders in a form and time frame consistent with NIH’s risk 

management needs. For example, the program requires a consistent 

methodology for identifying, assessing, and communicating risks across 

NIH, which will allow for consistent, comparable information from each of 

the ICs. 

However, the Enterprise Risk Management Program still does not fully 

address all of the components that we have identified for an effective risk 

management framework. As discussed below, further consideration of the 

risk management framework could significantly improve the design of 

NIH’s new risk management program, which, if effectively implemented, 

could assist management in maintaining effective control over the 

agency’s decentralized and diverse activities. 

Strategic Goals, Objectives, and Constraints. The Enterprise Risk 

Management Program does not require the NIH OD or ICs to set missionbased strategic goals and objectives as a precondition for risk 

management. This is a critical shortcoming because although the risk 

design requires risks to be assessed on the basis of their impact on NIH’s 

mission, there is not an NIH-wide strategic plan against which to assess 

risks. Further, while some ICs and NIH OD offices have strategic plans for 

their organizations, the risk management program as designed does not 

call for risks to be assessed on the basis of their impact on IC- or NIH OD 

office-level missions. 

Alternatives Evaluation. Although the Enterprise Risk Management 

Program identifies four different responses the agency can select to 

prevent or mitigate identified risks (creating a new policy, procedure, or 

Page 23 GAO-09-687 NIH Oversight 
control; revising an existing policy, procedure, or control; streamlining or 

automating an existing policy, procedure, or control; or redesigning the 

process), the program does not require management to evaluate the risk 

responses identified to consider (1) the effect on the likelihood of 

occurrence and impact of a potential risk and (2) the costs and benefits. 

These types of evaluations could assist management in making an 

informed decision within an environment that includes constrained 

resources. 

Management Selection. The design of the Enterprise Risk Management 

Program does not require management to document the rationale for 

selecting a particular risk response. Such documentation could help 

improve accountability and facilitate analysis of the effectiveness of 

actions taken. 

Implementation and Monitoring. Although the design of the Enterprise 

Risk Management Program requires periodic assessments of the overall 

efficiency and effectiveness of the risk management program, it does not 

offer any detail regarding how these assessments will be performed. For 

example, the program does not provide details such as the frequency, 

scope, or methodology for these reviews. Further, the design does not 

require periodic assessments of implemented risk responses. These types 

of monitoring activities are critical in helping management to identify 

problems with the overall risk management program and to determine 

whether risk responses are preventing or mitigating risks and operating as 

intended. 

Internal Environment. The Enterprise Risk Management Program includes 

many of the elements that define this component. However, the design 

could be improved by (1) incorporating the importance of ethical values 

into the risk management program and (2) ensuring management 

maintains the competence of its personnel by providing for continuous 

training to update personnel on risk management practices and 

techniques. 

Information and Communication. The design of NIH’s Enterprise Risk 

Management Program does not require the collection and dissemination of 

pertinent information to relevant external stakeholders in a form and time 

frame consistent with NIH’s risk management needs. For example, 

although the design requires annual reporting, in aggregate, to HHS on the 

adequacy of internal control, it does not require communication with other 

external stakeholders, such as congressional oversight committees. 

Page 24 GAO-09-687 NIH Oversight 
Implementation of the 

Enterprise Risk 

Management Program Has 

Been Hampered by Lack of 

Milestones 

The design and implementation of NIH’s new risk management program is 

not yet completed, despite an over 3-year effort. Without a sound risk 

management program NIH cannot be reasonably assured that it will be 

able to effectively and proactively identify, assess, and mitigate risks 

before they become problems that affect NIH’s ability to achieve its 

mission. During fiscal year 2008, the NIH OD implemented the first two 

steps of the six steps in its new risk management program. NIH had 

(1) organized the risk structure at the NIH OD and ICs, and identified and 

trained personnel responsible for managing risks within NIH OD, and 

(2) identified and scored risk at the NIH OD. NIH OD officials said they 

planned to complete the IC-level implementation of these two steps by the 

end of fiscal year 2009. 

The NIH OD is responsible for the design and implementation of the new 

program, and it has developed a time line with milestones for 

implementing some steps of the program. However, the timeline’s 

milestones are not firm, and the NIH OD has revised the timeline to 

accommodate delays. According to NIH OD officials, they have 

experienced delays in designing and implementing the new program 

because of a change in contractors, balancing limited staff resources with 

competing demands, and underestimating the amount of time necessary 

for implementing specific steps of the program. As of the completion of 

our draft report, the NIH OD had not set a date for fully implementing the 

program agencywide. However, in providing written comments on a draft 

of this report, HHS indicated that the Enterprise Risk Management 

Program at NIH was scheduled for full implementation throughout NIH by 

June 2010. (See agency comments and our evaluation for additional 

details.) 

While NIH’s decentralized structure allows NIH to address a wide range of 

research areas, it also creates significant oversight challenges. The ICs 

operate largely independently—each with its own budget, mission, and 

staff—making it vitally important that NIH and especially the OD have the 

means to ensure that all the ICs operate in accordance with NIH’s policies 

and mission. With an annual budget of nearly $30 billion, plus an 

additional $10 billion in funding available in 2009 and 2010 through the 

American Recovery and Reinvestment Act of 2009, the financial stakes are 

high. 

Conclusions 

We found gaps in NIH’s ability to monitor key aspects of its extramural 

funding process. Specifically, NIH’s OD does not monitor extramural 

funding decisions in which IC directors exercise their discretion to skip 

Page 25 GAO-09-687 NIH Oversight 
applications and make exceptions, even though information on these 

decisions is collected at the IC level. Without routine monitoring, which is 

consistent with federal internal control standards, NIH does not have the 

information to be reasonably assured that these decisions are appropriate 

and support the agency’s mission. Appropriate funding decisions are 

critical to ensuring an effective use of taxpayer dollars and supporting 

NIH’s reputation as the premier federal medical research agency in the 

United States. In reviewing selected administrative operations, we also 

found a key weakness in the design of the controls the NIH OD has 

established for oversight of travel and Title 42 personnel appointments. 

Without internal controls that include risk-based monitoring of the 

controls’ actual implementation, NIH cannot be reasonably assured that 

these controls are effective and operating as intended in areas identified as 

posing potential risks to NIH. 

Given these issues, a comprehensive risk management program could help 

ensure that such monitoring gaps are identified and addressed. NIH has 

recognized the importance of risk management to its organization and has 

taken steps towards implementing its new Enterprise Risk Management 

Program. Specifically, NIH has organized the risk structure at the NIH OD 

and ICs, identified and trained personnel responsible for managing risks 

within the NIH OD, and made progress in identifying and scoring risks at 

both the NIH OD and the ICs, which represent important steps. However, 

the design of the Enterprise Risk Management Program lacks several key 

components identified in our framework as necessary for effective risk 

management and the program has not yet been fully implemented 

throughout NIH. 

To ensure effective oversight of extramural funding decisions, we 

recommend that the Director of NIH establish a process for routine 

monitoring of the extramural funding decisions in which the IC directors 

use their discretion to skip applications or fund applications as 

exceptions. 

Recommendations for 

Executive Action 

To help ensure that NIH has a comprehensive program to effectively 

address potential risks to the agency’s mission, including those related to 

the monitoring of extramural research funding decisions, travel, and 

personnel appointments, we recommend that the Director of NIH take two 

actions to complete the design and implementation of NIH’s Enterprise 

Risk Management Program: 

Page 26 GAO-09-687 NIH Oversight 
• Add key components and related elements needed to achieve 

comprehensive and effective agencywide risk management to the design 

of NIH’s Enterprise Risk Management Program, including: 

• mission-based strategic goals and objectives as a precondition for risk 

management and risks to be assessed on the basis of their impact on 

the achievement of these goals and objectives; 

• the evaluation of risk responses to consider the effect on the likelihood 

of occurrence and impact of a potential risk and the costs and benefits; 

• the documentation of the rationale for selecting risk responses; 

• additional detail regarding how the assessments of the overall 

efficiency and effectiveness of the risk management program will be 

performed; 

• periodic assessments of implemented risk responses; 

• the importance of ethical values; 

• continuous training to maintain the competence of personnel carrying 

out risk management duties; and 

• communication with relevant external stakeholders. 

• Identify major milestones, including a final implementation date, to help 

ensure that NIH completes and implements the Enterprise Risk 

Management Program in a reasonable time frame. 

The Department of Health and Human Services provided written 

comments on a draft of this report, which are reprinted in appendix III. In 

responding to our draft report, HHS disagreed with the first 

recommendation and partially concurred with the second 

recommendation. In response to the third recommendation, HHS provided 

new information. The following sections summarize HHS’s comments on 

each of our three major findings and related recommendations and 

provide our responses. 

Agency Comments 

and Our Evaluation 

Page 27 GAO-09-687 NIH Oversight 
OD Oversight of 

Extramural Funding 

Decisions 

HHS disagreed with our recommendation that the Director of NIH should 

establish a process for routine monitoring of the extramural funding 

decisions in which the IC directors use their discretion to skip applications 

or fund applications as exceptions. In its written comments, HHS stated 

that we implied an inappropriate role for the NIH OD. Specifically, HHS 

said that the OD’s role was not to provide input on the scientific reasoning 

for making skips and exceptions, which should be left to the judgment of 

the scientific officials who understand the current trends in science and 

the institute research portfolios. HHS further stressed that the ICs are 

required to document the reasons for these decisions and that the 

documents are available for review by the OD upon request. 

Our work shows there would be benefit for the Office of the Director of 

NIH, as part of its responsibility to oversee IC operations, to routinely 

monitor the extent to which IC directors use their discretion to skip 

applications and fund applications as exceptions. This monitoring can be 

consistent with NIH’s stated reliance on scientific reasoning and the 

judgment of the scientific officials in making these decisions. As we noted 

in our draft report, when IC directors decide to skip applications and fund 

applications as exceptions, they do so by considering factors other than 

the science-based priority scores originally assigned to each application by 

NIH’s initial peer review groups and advisory councils. There can be good 

reasons for the decision to skip an application or fund an application as an 

exception, such as the desire to maintain a diverse portfolio of work. 

Routinely monitoring the extent to which IC directors use their discretion 

to skip applications and fund applications as exceptions would position 

the Director of NIH to help ensure that these decisions are consistent with 

NIH policy goals and are therefore appropriate. Such routine monitoring 

would also enable the Director of NIH to identify instances in which 

further review by appropriate officials may be desirable. Further, the 

routine monitoring we recommended is consistent with other efforts by 

the OD to monitor extramural funding decisions. As we noted in our draft 

report, the NIH OD already collects certain information related to 

extramural funding decisions, such as the priority rankings of funded 

applications and the number of extramural grants awarded to new 

investigators in response to an NIH-wide initiative. Finally, NIH OD 

monitoring activities would be consistent with federal internal control 

standards. 

In related comments, HHS drew attention to our finding that the share of 

RO1 grants awarded outside the payline (as exceptions) increased 

substantially from fiscal year 2003 through fiscal year 2007, and noted that 

this increase resulted largely from a corresponding increase in the number 

Page 28 GAO-09-687 NIH Oversight 
of RO1 grants awarded to new investigators. We agree with HHS, and 

noted in our draft report that our analysis of NIH’s records showed that 

the NIH-wide initiative to fund new investigators was one of the most 

frequently cited reasons for funding an application as an exception. HHS 

further stated that it would like to review our methods for quantifying the 

number of extramural grants funded as exceptions. As we indicated in the 

scope and methodology section of our draft report, we based our analysis 

on data provided by NIH. We noted that NIH provided us with information 

about the payline established by each of the 24 ICs for each fiscal year 

from 2003 through 2007, and the number of RO1 grant applications funded 

relative to each IC’s payline for each year. 

Design of Controls Over 

NIH’s Travel and Personnel 

Appointment Processes 

HHS concurred with our finding that the design of NIH’s Title 42 personnel 

appointment process included key control activities and some monitoring 

but lacked systematic risk-based monitoring. HHS said that it intends to 

incorporate risk-based monitoring into the Title 42 personnel appointment 

process. HHS also commented that NIH has identified and scored the 

agency travel process within its Enterprise Risk Management Program 

(discussed in the next section) but that it will reassess the travel risk 

levels to ensure that they are appropriate. 

Design of NIH’s 

Management Control 

Program and Enterprise 

Risk Management Program 

In response to our recommendation that NIH add key components to the 

design of its Enterprise Risk Management Program to achieve 

comprehensive and effective agencywide risk management, HHS agreed 

with some of our specific recommendations and disagreed with others. We 

identified eight specific items in this area; HHS agreed with four, partially 

agreed with one, and disagreed with three. 

• HHS agreed that the design of NIH’s Enterprise Risk Management Program 

should be modified to include the evaluation of risk responses to consider 

the effect on the likelihood of occurrence and impact of a potential risk 

and the costs and benefits. HHS noted that NIH will modify its Enterprise 

Risk Management Guidebook to reflect this recommendation. 

• HHS agreed that the design of NIH’s Enterprise Risk Management Program 

should be modified to include documentation of the rationale for selecting 

risk responses. HHS noted that it appreciated the feedback and has 

incorporated this element into NIH’s processes and amended the NIH 

Enterprise Risk Management Guidebook. 

Page 29 GAO-09-687 NIH Oversight 
• HHS agreed that the design of NIH’s Enterprise Risk Management Program 

should be modified to include periodic assessment of implemented risk 

responses. 

• HHS agreed that the design of NIH’s Enterprise Risk Management Program 

should be modified to include additional detail regarding how the 

assessments of the overall efficiency and effectiveness of the risk 

management program will be performed. However, HHS noted that the 

NIH Enterprise Risk Management Program has already undergone 

incremental evaluation during implementation. HHS also noted that NIH 

plans to develop a program evaluation process and conduct periodic 

reviews of the program in fiscal year 2011. 

• HHS partially agreed that the design of NIH’s Enterprise Risk Management 

Program should be modified to include communication with relevant 

external stakeholders. HHS noted that NIH promptly responds to all 

requests for information from external stakeholders. However, HHS also 

noted that the Enterprise Risk Management Program will include external 

communications as it matures. 

• HHS did not agree that the design of NIH’s Enterprise Risk Management 

Program should be modified to include mission-based strategic goals and 

objectives as a precondition for risk management and to assess risks on 

the basis of their impact on the achievement of these goals and objectives. 

HHS said that NIH’s Enterprise Risk Management Program is designed to 

identify and manage risks before they become obstacles to the NIH 

mission and noted that the ICs establish their own strategic goals and 

objectives. As we noted in the draft report, the design of the program does 

not require the NIH OD or ICs to set mission-based strategic goals and 

objectives as a precondition for risk management, nor does the design call 

for risks to be assessed on the basis of their impact on IC- or NIH OD-level 

missions. We continue to believe that a clear and explicit link to strategic 

goals and objectives would help ensure that risks are routinely assessed 

based on their potential impact to achieving NIH’s mission and would 

identify criteria against which to measure performance. 

• HHS did not agree that the design of NIH’s Enterprise Risk Management 

Program should be modified to include the importance of ethical values. 

HHS said that NIH’s risk management program already operates within the 

context of a positive environment in which integrity and ethical values 

play a key role. However, HHS said that NIH would modify the design of 

the Enterprise Risk Management Program as we recommended, by 

amending the Enterprise Risk Management Guidebook to include specific 

language addressing the importance of ethics at NIH. 

Page 30 GAO-09-687 NIH Oversight 
• HHS did not agree that the design of NIH’s Enterprise Risk Management 

Program should be modified to include continuous training to maintain the 

competence of personnel carrying out risk management duties. 

Nevertheless, HHS stated that NIH has provided training to over 400 

individuals who hold significant risk management roles and noted that 

NIH plans to develop continuous training for all employees on risk 

management. Moreover, HHS said that NIH will modify the design of the 

Enterprise Risk Management Program as we recommended, by modifying 

the Enterprise Risk Management training plan to incorporate ongoing 

training such as training updates and refreshers. 

In response to our recommendation that the Director of NIH should 

identify major milestones, including a final implementation date, to help 

ensure timely implementation of the Enterprise Risk Management 

Program, HHS identified a final implementation date of June 2010. 

Although HHS asserted that NIH’s Enterprise Risk Management Program 

is fully functional because NIH has implemented all six steps of the 

program at some level, as we noted in our draft report and as HHS 

confirmed in its written comments, several elements of the program have 

not been implemented across all of NIH. For example, HHS stated that 

steps one and two (identify and score risks) have been implemented 

across all of NIH—including the OD and the ICs—but that steps three and 

four (assess and remediate risks) have been implemented at the OD level 

but not across the ICs. If NIH proceeds with the actions and time frames 

outlined in HHS’s comments, it should meet the intent of our 

recommendation. 

HHS stated that the prior risk management program—which our draft 

referred to as the “current” program—was discontinued in 2006. This 

statement is not consistent with the information we gathered during the 

time of our review nor with the policy manual posted on the NIH Web site, 

which states that the Management Control Program was “temporarily 

rescinded effective June 24, 2009,”—1 day after HHS received our report 

for review and comment—and that replacement guidance has not been 

issued. If the prior program has been discontinued and the final 

implementation date for the new program is scheduled for June 2010, NIH 

may have been operating without a fully functioning risk assessment 

program in place, which is a key element of a system of internal control. 

Although we believe our draft report correctly characterized the status of 

NIH’s Management Control Program and Enterprise Risk Management 

Program at the time of our review, in response to HHS’s comments we 

revised the wording in our report to more clearly distinguish between the 

Page 31 GAO-09-687 NIH Oversight 
new Enterprise Risk Management Program and the Management Control 

Program it is replacing. 

In commenting on our evaluation of the NIH Enterprise Risk Management 

Program, HHS questioned the criteria we used in our evaluation. HHS 

stated that it defines risk management as synonymous with internal 

control and that the NIH Enterprise Risk Management Program was 

developed based on the Standards for Internal Control in the Federal 

Government and the Office of Management and Budget (OMB) Circular A123. Thus, HHS suggested that we should revise our report using different 

criteria. We believe that our criteria are appropriate for the evaluation. As 

noted in the draft report, GAO developed the framework based on 

authoritative literature and standards, as well as previous GAO reports and 

testimonies. We consulted the Government Performance and Results Act 

(GPRA) of 1993; the Government Auditing Standards, 2003 Revision; 

GAO’s Standards for Internal Control in the Federal Government

(November 1999); guidance from OMB; the work of the President’s 

Commission on Risk Management; consulting papers; and the enterprise 

risk management approach of the Committee of Sponsoring Organizations 

(COSO) of the Treadway Commission. We also reviewed numerous risk 

management frameworks from industry, government, and academic 

sources. 

Furthermore, our draft report noted the relationship between internal 

control and risk management. Specifically, risk management is a 

continuous process through which an organization identifies, assesses, 

and mitigates risks, and through risk management, an organization can 

identify the most significant areas in which to place or enhance internal 

control. Systems of internal control may help an organization prevent or 

reduce risks, such as fraud, waste, abuse, or mismanagement. Internal 

control standards, therefore, provide an important tool for use in risk 

management. For example, in response to our draft report, NIH pointed 

out a variety of management oversight mechanisms, as discussed below. 

Those mechanisms could be considered part of NIH’s internal controls, 

but are not part of its risk management program. We believe that the 

framework we used to evaluate NIH’s risk management program was 

appropriate. 

In addition, HHS commented that our report implied that the risk 

management program is the sole management oversight mechanism at 

NIH and that we failed to acknowledge other oversight bodies and 

functions. We agree that NIH has many mechanisms for managerial 

oversight and accountability and we cited some of the mechanisms HHS 

Page 32 GAO-09-687 NIH Oversight 
specified in the draft report, such as oversight of travel and Title 42 

personnel appointments. However, it was beyond the scope of our report 

to evaluate the full spectrum of NIH’s oversight and accountability 

mechanisms. Further, regardless of the number or type of the other 

oversight mechanisms in place at NIH, these do not in any way diminish 

NIH’s need to make its risk management program fully functioning, 

comprehensive, and effective. 

HHS also provided us with technical comments, which we incorporated as 

appropriate. 

As arranged with your office, unless you publicly announce its contents 

earlier, we plan no further distribution of this report until 30 days after its 

issue date. At that time, we will send copies of this report to other 

interested congressional committees, the Secretary of HHS, and the 

Director of NIH. This report will also be available on the GAO Web site at 

http://www.gao.gov. 

If you or your staff have any questions regarding this report, please 

contact Linda T. Kohn at (202) 512-7114 or kohnl@gao.gov or Susan 

Ragland at (202) 512-8486 or raglands@gao.gov. Contact points for our 

Offices of Congressional Relations and Public Affairs may be found on the 

last page of this report. Key contributors to this report are listed in 

appendix IV. 

Sincerely yours, 

rector 

Health Care 

Susan Ragland, Director 

Financial Management and Assurance 

Linda T. Kohn, Di

Page 33 GAO-09-687 NIH Oversight 
Appendix I: Scope and Methodology Appendix I: Scope and Methodology 

To gain an understanding of the process used to make extramural research 

funding decisions, we reviewed the laws and regulations governing the 

funding process and National Institutes of Health (NIH) policies related to 

each stage of the process. We also interviewed NIH officials with 

responsibility for establishing these policies and overseeing the institutes 

and centers’ (IC) implementation of this process. In addition, to develop a 

detailed understanding of how the 24 ICs that fund extramural research 

implement the process, we selected 3 of the 24 ICs for a more detailed 

review: the National Cancer Institute (NCI), the National Institute on 

Alcohol Abuse and Alcoholism (NIAAA), and the National Institute of 

Diabetes and Digestive and Kidney Diseases (NIDDK). These ICs were 

chosen because they vary in budget size and focus on different diseasespecific research missions. We also included the Center for Scientific 

Review (CSR), which does not fund extramural research but is responsible 

for implementing the initial steps in the extramural research funding 

process, including receipt of all grant applications. 

At the IC level, we reviewed IC policies and guidance for implementing the 

extramural research funding process and interviewed officials at each of 

the 3 selected ICs plus CSR about their roles in receiving applications, 

facilitating peer review of the applications, and making final funding 

decisions. We also interviewed members of the NCI, NIAAA, and NIDDK 

advisory councils about their role in reviewing and making 

recommendations regarding extramural grant applications. In addition, we 

analyzed selected data from the 24 ICs that fund extramural research 

regarding funding decisions for grants in NIH’s R01 category, which is the 

most common of NIH’s various grant categories. The R01 grant is the 

original and historically oldest grant mechanism used by NIH. This type of 

grant is awarded to organizations of all types (universities, colleges, small 

businesses, for-profit, foreign and domestic, etc.) to support a discrete, 

specified project to be performed by a named investigator or investigators. 

Specifically, we requested information about the paylines each of the 24 

ICs established during fiscal years 2003 through 2007 to be used when 

making funding decisions. (The payline roughly corresponds with the 

number of extramural grant applications an IC will be able to fund each 

year and is based on projections of the total funding available for grants at 

the IC that year, the average dollar amount expected to be awarded per 

application, and the number of applications coming to an IC.) We also 

requested data about the number of R01 grant applications received, 

scored, and recommended by the peer review groups; the total number of 

grant applications funded; and the number of grant applications funded 

relative to each IC’s payline that year. We used the data to analyze trends 

in funding decisions over the 5-year period. In order to analyze the reasons 

Page 34 GAO-09-687 NIH Oversight 
Appendix I: Scope and Methodology 

the ICs cited when funding applications as exceptions to the payline for 

R01 grants, we collected IC documentation for fiscal years 2006 and 2007 

from NCI, NIAAA, and NIDDK. Because the total number of exception 

decisions made by NCI and NIDDK were large during this time frame, we 

analyzed documents for a random sample of the grants awarded as 

exceptions to the main payline. We also reviewed IC documentation 

related to applications with priority scores above the main payline that 

were not funded by NCI and NIDDK. NIAAA did not choose to skip any 

applications during these fiscal years. 

To ensure that the IC data were sufficiently reliable for our analyses, we 

conducted detailed data reliability assessments of the data that we used. 

We assessed the reliability of the IC data by reviewing existing information 

about the data and the system that produced them and interviewing 

agency officials knowledgeable about the data. We determined that the 

data were sufficiently reliable for the purposes of this report. 

To gain an understanding of the design of control and monitoring activities 

over travel and Title 42 personnel appointments, we reviewed relevant 

NIH policies and guidance.1

 To further our understanding of control and 

monitoring activities, we also performed walkthroughs of the travel and 

Title 42 personnel appointment processes at three ICs—NCI, NIDDK, and 

NIAAA; these were the three ICs selected for our review of the extramural 

research funding process. 

During our walkthroughs of the travel process, we reviewed 

authorizations, vouchers, and supporting receipts for travel transactions at 

each of the selected ICs. During our walkthroughs of the Title 42 

personnel appointment process, we reviewed checklists showing 

documents included in the appointment packages, routing slips showing 

who received the appointment packages, and memos documenting 

approvals for Title 42 personnel appointments at each of the selected ICs. 

We interviewed key officials from the NIH OD and the ICs, including the: 

 

1

For travel, we reviewed NIH manual chapters 1500-01: Introduction to Official 

Government Travel (Jan. 5, 2004), 1500-02: Traveler Responsibilities (May 13, 2008), and 

1500-08: Acceptance of Payment from a Nonfederal Source to Cover Travel Expenses 

[Sponsored Travel] (Jan. 23, 2006). For Title 42 personnel appointments, we reviewed Title 

42 Pay Model—NIH (Dec. 21, 2004) and NIH manual chapter 2300-575-2: Title 42 

Recruitment and Retention Incentives (May 4, 2000). 

Page 35 GAO-09-687 NIH Oversight 
Appendix I: Scope and Methodology 

• NIH Deputy Director and the NIH Deputy Director for Management to 

clarify our understanding of the differences between the roles of the NIH 

OD and the ICs in the travel and personnel appointment processes and the 

associated control and monitoring activities; 

• NIH Director of Financial Management and the NIH Director and Deputy 

Director of the Office of Human Resources to gain an understanding of the 

control and monitoring activities that the NIH OD performs over travel and 

Title 42 personnel appointments; and 

• IC Executive Officers (the highest level officials at the ICs that oversee 

administrative activities) and other specialists within the ICs to clarify our 

understanding of control and monitoring activities in the travel and Title 

42 personnel appointments at the IC level. 

We compared the design of the processes to GAO’s Standards for Internal 

Control in the Federal Government2

 to determine if the processes as 

designed included appropriate control and monitoring activities. While the 

design of control activities is based on NIH-wide policies and procedures, 

monitoring activities vary at the individual ICs. Therefore, our review of 

monitoring activities for travel and Title 42 personnel appointments at 

these selected ICs cannot be generalized to the other ICs. The scope of our 

audit did not include testing the implementation of controls over travel 

and Title 42 personnel appointments. 

To gain an understanding of the design of the NIH Management Control 

Program, we reviewed relevant NIH policy and supporting documentation. 

Specifically, we reviewed relevant NIH policies3

 and the NIH OD’s fiscal 

year 2008 guidance to the ICs on reporting risk management activities. To 

gain an understanding of the design of the Enterprise Risk Management 

Program we reviewed NIH draft guidance.4

 We also reviewed the time 

lines for implementing the Enterprise Risk Management Program to 

determine the estimated implementation dates. We interviewed key 

officials from the NIH OD including the: 

 

2

GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1

(Washington, D.C.: November 1999). 

3

NIH Manual Chapter 1750 – Management Control Program (Nov. 15, 2004). 

4

NIH Office of Management Assessment, NIH Enterprise Risk Management Program, 

Enterprise Risk Management Guidebook: A Step-By-Step Guide (March 2009, Draft). 

Page 36 GAO-09-687 NIH Oversight 
Appendix I: Scope and Methodology 

• NIH Deputy Director and the NIH Deputy Director for Management to gain 

a high-level understanding of how the Enterprise Risk Management 

Program will address recent oversight issues at NIEHS and help NIH to 

better manage its decentralized organization; 

• NIH Director of Financial Management to understand the risk activities 

NIH performed for fiscal year 2008 as part of the NIH Management Control 

Program; and 

• NIH Director for the Office of Management Assessment—the office with 

primary responsibility for designing and implementing the new risk 

management program—to understand current risk activities at NIH, to 

clarify the design of the new risk management program, and to further our 

understanding of the implementation time line for the new risk 

management program as well as the cause for delays in implementation. 

We compared elements of the NIH Management Control Program and the 

Enterprise Risk Management Program to our risk management 

framework5

 to determine if the designs contain the key components of an 

effective risk management program. We did not review the implementation 

of either the NIH Management Control Program or the Enterprise Risk 

Management Program because, at the time of our review, NIH did not plan

to continue the Management Control Program and the Enterprise Risk 

Management Program was not yet fully implemented. 

 

We conducted this performance audit from March 2008 to September 2009, 

in accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

5

See table 2. GAO developed the framework based on authoritative literature and 

standards, as well as previous GAO reports and testimonies. We consulted the Government 

Performance and Results Act (GPRA) of 1993; the Government Auditing Standards, 2003 

Revision; GAO’s Standards for Internal Control in the Federal Government (November 

1999); guidance from the Office of Management and Budget (OMB); the work of the 

President’s Commission on Risk Management; consulting papers; and the enterprise risk 

management approach of the Committee of Sponsoring Organizations (COSO) of the 

Treadway Commission. We reviewed numerous frameworks from industry, government, 

and academic sources. GAO, “Appendix I: A Risk Management Framework” of Risk 

Management: Further Refinements Needed to Assess Risks and Prioritize Protective 

Measures at Ports and Other Critical Infrastructure, GAO-06-91 (Washington, D.C.: 

Dec. 15, 2005). 

Page 37 GAO-09-687 NIH Oversight 
Appendix I: Scope and Methodology 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives. 

Page 38 GAO-09-687 NIH Oversight 
Appendix II: NIH Organization and Mission Appendix II: NIH Organization and Mission 

As the primary federal agency for supporting medical research in the 

United States, the National Institutes of Health’s (NIH) mission is “science 

in pursuit of fundamental knowledge about the nature and behavior of 

living systems and the application of that knowledge to extend healthy life 

and reduce the burdens of illness and disability”. NIH is headed by a 

Director who is supported by 11 staff offices and 1 program office within 

the NIH Office of the Director (OD) and 27 institutes and centers (IC). 

Figure 3 depicts the organizational structure of NIH. 

Page 39 GAO-09-687 NIH Oversight 
Appendix II: NIH Organization and Mission 

Figure 3: Organizational Structure of NIH 

Source: NIH.

Immediate Office of the Director

Office of the Director Program Office:

Division of Program Coordination, Planning, and Strategic Initiatives

Office of the Director Staff Offices:

Office of Extramural Research

Office of Intramural Research

Office of Management/Chief Financial Officer

Office of Science Policy

Office of Communications and Public Liaison

Office of Equal Opportunity and Diversity Management

Office of Legislative Policy and Analysis

Executive Office

Office of the Ombudsman/Center for Coorperative Resolution

NIH Ethics Office

Office of the Chief Information Officer

National Cancer 

Institute

National Eye Institute National Heart, Lung, 

and Blood Institute

National Human 

Genome Research 

Institute

National Institute on 

Aging

National Institute on 

Alcohol Abuse and 

Alcoholism

National Instutite 

of Allergy and 

Infectious Diseases

National Institute of 

Arthritis and 

Musculoskeletal and 

Skin Diseases

National Institute of 

Biomedical Imaging 

and Bioengineering

Eunice Kennedy Shriver 

National Institute of 

Child Health and 

Human Development

National Institute on 

Deafness and Other 

Communication 

Disorders

National Institute 

of Dental and 

Craniofacial 

Research

National Institute 

of Diabetes and 

Digestive and 

Kidney Diseases

National Institute 

on Drug Abuse

National Institute 

of Environmental 

Health Sciences

National Institute 

of General Medical 

Sciences

National Institute 

of Mental Health

National Institute 

of Neurological 

Disorders and Stroke

National Institute of 

Nursing Research

National Library 

of Medicine

John E. Fogarty 

International Center 

for Advanced Study in 

the Health Sciences

National Center for 

Complementary and 

Alternative Medicine

National Center on 

Minority Health and 

Health Disparities

National Center for 

Research Resources

Clinical Center Center for Information TechnologyCenter for Scientific 

Review

Page 40 GAO-09-687 NIH Oversight 
Appendix II: NIH Organization and Mission 

The ICs, which were established over time, each have an explicit mission 

focused on a particular disease or organ system, an area of human h

and development, or aspects of research support.

ealth 

ring, 

al 

extramural research. Table 3 depicts a time line of the establishment of the 

7 ICs and their respective missions and fiscal year 2008 appropriations. 

 

1

 The first institute, the 

National Cancer Institute, was established in 1937, and the newest 

institute, the National Institute of Biomedical Imaging and Bioenginee

was established in 2000. Research funded by NIH can be conducted by 

scientists in NIH laboratories and Clinical Center—called intramur

research—or by nonfederal scientists at universities, academic health 

centers, hospitals, and independent research institutions—called 

2

1

Prior to 1985, Congress either created ICs itself or gave others (e.g., the Surgeon General 

or the Secretary of HHS) the authority to create ICs through individual laws. Since 1985, 

the Secretary of HHS has had the authority to establish, reorganize, or abolish ICs. Pub. L. 

No. 99-158, 99 Stat. 820 (1985). 

Page 41 GAO-09-687 NIH Oversight 
Appendix II: NIH Organization and Mission 

Table 3: Overview of ICs Including Establishment Date, Mission, and Fiscal Year 2008 Appropriation 

IC 

Year 

established Mission 

FY 2008 

appropriation 

(in 000s)

National Cancer Institute 1937 Conducts and supports research that will lead to a future in 

which we can prevent cancer, identify cancers that do 

develop at the earliest stage, eliminate cancers through 

innovative treatment interventions, and biologically control 

those cancers that we cannot eliminate so they become 

manageable, chronic diseases. 

$4,830,647

Center for Scientific Review 1946 Conducts initial peer reviews of the majority of research and 

research-training applications submitted to NIH. 

N/Aa

National Institute of Allergy and 

Infectious Diseases 

1948 Leads research that strives to understand, treat, and 

ultimately prevent the myriad infectious, immunologic, and 

allergic diseases that threaten millions of human lives. 

4,583,344

National Heart, Lung, and 

Blood Institute 

1948 Provides leadership for a national program in diseases of the 

heart, blood vessels, lung, and blood; blood resources; and 

sleep disorders. 

2,937,654

National Institute of Dental and 

Craniofacial Research 

1948 Provides leadership for a national research program 

designed to understand, treat, and ultimately prevent the 

infectious and inherited craniofacial-oral-dental diseases and 

disorders that compromise millions of human lives. 

392,233

National Institute of Mental 

Health 

1949 Provides national leadership dedicated to understanding, 

treating, and preventing mental illnesses through basic 

research on the brain and behavior, and through clinical, 

epidemiological, and services research. 

1,412,951

National Institute of Diabetes 

and Digestive and Kidney 

Diseases 

1950 Conducts and supports basic and applied research and 

provides leadership for a national program in diabetes, 

endocrinology, and metabolic diseases; digestive diseases 

and nutrition; and kidney, urologic, and hematologic 

diseases. 

1,715,761

National Institute of 

Neurological Disorders and 

Stroke 

1950 Seeks to reduce the burden of neurological diseases by 

supporting and conducting research, both basic and clinical, 

on the normal and diseased nervous system, fostering the 

training of investigators in the basic and clinical 

neurosciences, and seeking better understanding, diagnosis, 

treatment, and prevention of neurological disorders. 

1,552,113

Clinical Center 1953 Provides the patient care, services, and environment needed 

to initiate and support the highest quality conduct of and 

training in clinical research. 

N/Aa

National Library of Medicine 1956 Collects, organizes, and makes available biomedical science 

information to scientists, health professionals, and the public. 

322,212

National Institute of General 

Medical Sciences 

1962 Supports basic biomedical research that is not targeted to 

specific diseases but rather funds studies on genes, proteins, 

and cells, as well as on fundamental processes like 

communication within and between cells, how our bodies use 

energy, and how we respond to medicines. 

1,946,104

Page 42 GAO-09-687 NIH Oversight 
Appendix II: NIH Organization and Mission 

IC 

Year 

established Mission 

FY 2008 

appropriation 

(in 000s)

Eunice Kennedy Shriver 

National Institute of Child 

Health and Human 

Development 

1962 Leads research on fertility, pregnancy, growth, development, 

and medical rehabilitation that strives to ensure that every 

child is born healthy and wanted and grows up free from 

disease and disability. 

1,261,381

National Center for Research 

Resources 

1962 Provides laboratory scientists and clinical researchers with 

the environments and tools they need to understand, detect, 

treat, and prevent a wide range of diseases. 

1,155,560

Center for Information 

Technology 

1964 Incorporates the power of modern computers into the 

biomedical programs and administrative procedures of NIH 

by focusing on three primary activities: conductingcomputational biosciences research, developing computer 

systems, and providing computer facilities. 

N/Aa

National Eye Institute 1968 Conducts and supports research that helps prevent and treat 

eye diseases and other disorders of vision. 

670,664

John E. Fogarty International 

Center for Advanced Study in 

the Health Sciences 

1968 Promotes and supports scientific research and training 

internationally to reduce disparities in global health. 

66,912

National Institute of 

Environmental Health Sciences 

1969 Reduces the burden of human illness and dysfunction from 

environmental causes by, defining how environmental 

exposures, genetic susceptibility, and age interact to affect 

an individual’s health. 

645,669

National Institute on Alcohol 

Abuse and Alcoholism 

1970 Conducts research focused on improving the treatment and 

prevention of alcoholism and alcohol-related problems to 

reduce the enormous health, social, and economic 

consequences of this disease. 

438,579

National Institute on Drug 

Abuse 

1973 Supports and conducts research across a broad range of 

disciplines and rapid and effective dissemination of results of 

that research to improve drug abuse and addiction 

prevention, treatment, and policy. 

1,006,022

National Institute on Aging 1974 Leads a national program of research on the biomedical, 

social, and behavioral aspects of the aging process; the 

prevention of age-related diseases and disabilities; and the 

promotion of a better quality of life for all older Americans. 

1,052,830

National Institute of Arthritis 

and Musculoskeletal and Skin 

Diseases 

1986 Supports research into the causes, treatment, and prevention 

of arthritis and musculoskeletal and skin diseases, the 

training of basic and clinical scientists to carry out this 

research, and the dissemination of information on research 

progress in these diseases. 

511,291

National Institute of Nursing 

Research 

1986 Supports clinical and basic research to establish a scientific 

basis for the care of individuals across the life span—

including managing patients during illness and recovery to 

reducing risks for disease and disability; promoting healthy 

lifestyles; promoting quality of life in those with chronic 

illness; and caring for individuals at the end of life. 

138,207

Page 43 GAO-09-687 NIH Oversight 
Appendix II: NIH Organization and Mission 

IC 

Year 

established Mission 

FY 2008 

appropriation 

(in 000s)

National Institute on Deafness 

and Other Communication 

Disorders 

1988 Conducts and supports biomedical research and research 

training on normal and disordered processes of hearing, 

balance, smell, taste, voice, speech, and language that affect 

46 million Americans. 

396,234

National Human Genome 

Research Institute 

1989 Supports the NIH component of the Human Genome Project, 

a worldwide research effort designed to analyze the structure 

of human DNA and determine the location of the estimated 

30,000 to 40,000 human genes. 

489,368

National Center on Minority 

Health and Health Disparities 

1993 Promotes minority health and leads, coordinates, supports, 

and assesses NIH efforts to reduce and ultimately eliminate 

health disparities among minority and other medically 

underserved communities. Conducts and supports basic, 

clinical, social, and behavioral research; promotes research 

infrastructure and training; fosters emerging programs; 

disseminates information; and reaches out to minority and 

other medically underserved communities. 

200,630

National Center for 

Complementary and Alternative 

Medicine 

1999 Explores complementary and alternative medical practices in 

the context of rigorous science; trains researchers; and 

disseminates authoritative information. 

122,224

National Institute of Biomedical 

Imaging and Bioengineering 

2000 Improves health by promoting fundamental discoveries, 

design and development, and translation and assessment of 

technological capabilities in biomedical imaging and 

bioengineering. 

300,233

Source: NIH. 

a

The IC does not fund research and does not receive a separate appropriation but rather is funded 

through the NIH Management Fund. 

Page 44 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National Appendix III: Comments from the National Institutes of Health 

Institutes of Health 

Page 45 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 46 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 47 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 48 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 49 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 50 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 51 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 52 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 53 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 54 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 55 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 56 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 57 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 58 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 59 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 60 GAO-09-687 NIH Oversight 
Appendix III: Comments from the National 

Institutes of Health 

Page 61 GAO-09-687 NIH Oversight 
Appendix IV: 

A

GAO Contacts and Staff 

cknowledgments 

Page 62 GAO-09-687 NIH Oversight 

Appendix IV: GAO Contacts and Staff 

Acknowledgments 

Linda T. Kohn (202) 512-7114 or kohnl@gao.gov

Susan Ragland (202) 512-8486 or raglands@gao.gov 

In addition to the contacts named above, Paul Caban and Jenny Grover, 

Assistant Directors; Jehan Abdel-Gawad; Deyanna Beeler; Francine 

Delvecchio; Patrick Frey; Krister Friday; Natalie Herzog; Cynthia Jackson; 

Kelli Jones; Judy Lee; Lisa Motley; Kara Patton; Will Simerl; Jessica Smith; 

and Matt Zaun made key contributions to this report. 

GAO Contacts 

Acknowledgments 

(290691) 
GAO’s Mission The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday afternoon, GAO 

posts on its Web site newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.” 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s Web site, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information. 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, DC 20548 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548 

Public Affairs 

Please Print on Recycled Paper