FEDERAL TRAINING 

INVESTMENTS 

Office of Personnel 

Management and 

Agencies Can Do More 

to Ensure CostEffective Decisions 

A Report to the Chairman, Subcommittee on 

Oversight of Government Management, the 

Federal Workforce, and the District of Columbia; 

Committee on Homeland Security and 

Government Affairs, United States Senate 

September 2012 

GAO-12-878 

United States Government Accountability Office

GAO 
 United States Government Accountability Office 

Highlights of GAO-12-878, a report to the 

Chairman, Subcommittee on Oversight of 

Government Management, the Federal 

Workforce, and the District of Columbia; 

Committee on Homeland Security and 

Governmental Affairs, United States Senate 

September 2012 

FEDERAL TRAINING INVESTMENTS 

Office of Personnel Management and Agencies Can 

Do More to Ensure Cost-Effective Decisions 

Why GAO Did This Study 

OPM and agency CHCOs play an 

important role in ensuring that federal 

training dollars are invested effectively. 

GAO was asked to review the extent to 

which: (1) CHCOs of selected federal 

agencies have established processes 

to set and prioritize training 

investments that are aligned with 

leading practices; and (2) OPM’s 

guidance and assistance for 

developing training investment 

strategies align with these leading 

practices. GAO obtained information 

from 27 CHCOs on their training 

investment practices through a 

questionnaire, and selected four 

agencies—the Departments of Energy 

(DOE), Homeland Security (DHS), the 

Interior (DOI) and Veterans Affairs 

(VA)—to provide illustrative examples. 

We compared both CHCO and OPM 

practices to leading practices, 

identified through past GAO and expert 

studies. 

What GAO Recommends 

GAO recommends, among other 

things, that OPM improve guidance 

and assistance to agencies in 

establishing a process for setting and 

prioritizing training investments; 

improve the reliability of agency 

training investment information; and 

identify the best existing courses that 

fulfill governmentwide training 

requirements, and offer them to all 

agencies through the HR University or 

other appropriate platforms. OPM fully 

or partially concurred with four 

recommendations and did not concur 

with a portion of another. OPM, DOI 

and VA provided technical comments, 

which GAO incorporated, as 

appropriate, into the report. DOE and 

DHS had no comments. 

What GAO Found 

Many Chief Human Capital Officers (CHCOs) reported that they are 

implementing several leading practices important to making strategic decisions 

about training delivery, such as determining the best mix of decentralized and 

centralized training and considering government-wide reform when planning 

training. However, many CHCOs reported they are not implementing some 

practices that support making more cost-effective training investment decisions, 

such as prioritizing training so that the most important needs are met first and 

evaluating the benefits of training. In addition, many CHCOs do not have 

information from component or sub-agency leaders regarding their level of 

investments and priorities. Consequently, some agencies are duplicating internal 

training investments and missing opportunities to leverage economies of scale 

across their agencies. Federal agencies also need reliable information on how 

much they spend on training and for what purposes. However, several CHCOs 

reported they do not completely and reliably track training costs agency-wide. 

The Office of Personnel Management (OPM) provides guidance and assistance 

to agencies on a number of the leading practices, such as evaluating the benefits 

of training in three of its guides and in workshops. In some practice areas that 

are challenges to agencies, such as prioritization of investments and determining 

whether to design training and development programs in-house or obtain these 

services from a contractor, guidance is minimal or absent. OPM also requires 

agencies to submit training investment data and provides guidance on how to do 

so, but considers this data to be unreliable because it is incomplete. However, 

OPM officials have not internally assessed improvements in the completeness of 

the data over the last 3 years or the quality of the data in the six years that 

agencies have been required to submit it, and have only provided agencies with 

one summary of their data for correction. Agencies and OPM reported there are 

also opportunities for OPM to help agencies reduce duplicative investments 

across agencies. For example, currently, agencies independently purchase or 

develop training for the same mandated or common occupational training. 

Agency leaders and OPM recognize that this has led to redundant and inefficient 

federal training investments. According to OPM officials, HR University—which is 

a website currently administered by OPM to provide training for the HR 

community—has already resulted in a cost savings of $14.5 million as a result of 

sharing the best HR training government-wide. Several agencies and OPM 

officials reported that HR University could be expanded to provide mandatory 

training and serve as a model for centralizing training in other occupations or 

functional areas, which could save millions more and help standardize training. 

View GAO-12-878. For more information, 

contact Yvonne J. Jones at (202) 512-2717 or 

Jonesy@gao.gov. 
Page i GAO-12-878 Federal Training Investments 

Letter 1 

Background 5

CHCOs are Implementing Leading Practices Related to Training 

Delivery, but Not Practices that Support Making More CostEffective Training Investments 10

OPM Guidance and Assistance to Agencies on Federal Training 

Investments Addresses Many Leading Practices that We 

Identified, But Could be Improved in Areas That Are Challenges 

to Agencies 29

Conclusions 45

Recommendations for Executive Action 46

Agency Comments and Our Evaluation 48

Appendix I Objectives, Scope, and Methodology 52

Appendix II Table of Statutes, Regulations, and Executive Orders Related to 

Leading Practices 57

Appendix III Illustrations of Building a Business Case for Training from OPM’s 

Guide to Strategically Planning Training and Measuring Results 59

Appendix IV Comments from the Office of Personnel Management 61

Appendix V GAO Contact and Staff Acknowledgments 66

Tables

Table 1: Leading Training Investment Practices 3

Table 2: OPM’s Guidance Documents and Description 6

Table 3: The Leading Training Investment Practices CHCOs 

Reported Implementing 11

Table 4: Leading Training Investment Practices that Many CHCOs 

Reported Not Implementing Routinely 21

Contents
Page ii GAO-12-878 Federal Training Investments 

Table 5: Comparison of Agency Implementation of Leading 

Practices and OPM Guidance and Assistance 29

Table 6: Selection of Case Example Agencies 55

Figures

Figure 1: OPM illustration of questions used to develop a business 

case for training 59

Figure 2: OPM illustration of developing a business case for 

training 60

Abbreviations

HCO chief human capital officer

CLO chief learning officer 

DHS Department of Homeland Security

DOE Department of Energy

EHRI Enterprise Human Resources Integration

GPRA Government Performance and Results Act

GEAR Goals-Engagement-Accountability-Results

GPRAMA GPRA Modernization Act of 2010

HR human resources

DOI Department of the Interior

OPM Office of Personnel Management

TED Training and Executive Development

VA Department of Veterans Affairs

VALU Veterans Affairs Learning University

VBA Veterans Benefits Administration

VHA Veterans Health Adminis

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-12-878 Federal Training Investments 

United States Government Accountability Office

Washington, DC 20548 

September 17, 2012 

The Honorable Daniel K. Akaka 

Chairman 

Subcommittee on Oversight of Government Management, 

 the Federal Workforce, and District of Columbia 

Committee on Homeland Security and Governmental Affairs 

United States Senate 

Dear Chairman Akaka: 

Constrained budgets and the need to address gaps in critical federal skills 

and competencies make it is essential that agencies identify the 

appropriate level of investment and establish priorities for employee 

training and development, so that the most important training needs are 

addressed first.1

 The Office of Personnel Management (OPM) and 

agency chief human capital officers (CHCOs) play important roles in 

ensuring that federal training dollars are invested wisely. OPM provides 

leadership and guidance on the establishment and operation of efficient 

federal training and development programs government-wide and advises 

the President on means for furthering and strengthening federal training 

programs. CHCOs statutorily have the primary role in setting the 

workforce development strategies of their agencies, and advising and 

assisting their agency heads who are responsible for establishing 

priorities for needed training and development, and providing for the use 

of funds and resources in accordance with these priorities. The decisions 

that these key players make in determining how to invest federal training 

dollars will ultimately have an important impact on how well the federal 

government is equipped to address its current and future performance 

and fiscal goals and challenges. 

In order to better understand how federal training investment decisions 

are made and whether improvements are needed, you asked us to review 

the methods that agencies are using to establish their training investment 

strategies and OPM’s training investment guidance to agencies. 

 

1

In the February 2011 update of our High Risk Series, we reported that the most 

significant human capital challenge to the government is to close current and emerging 

critical skills gaps. GAO, High-Risk Series: An Update, GAO-11-278 (Washington, D.C.: 

February 2011). 

 
 

Page 2 GAO-12-878 Federal Training Investments 

Accordingly, this review assesses the extent to which (1) CHCOs of 

selected federal agencies have established processes to set and prioritize 

training investments that are aligned with leading practices and (2) OPM’s 

guidance and assistance for developing training investment strategies 

align with these leading practices.

For the purposes of this review, we define the key terms “training”, 

“development” and “agency-wide” in the following ways:

• Training is making available to employees planned and coordinated 

educational programs of instruction in professional, technical, or other 

fields that are or will be related to the employee’s job responsibilities. 

Training can be accomplished through a variety of approaches, such 

as classroom training, e-learning, and professional conferences that 

are educational or instructional in nature.2

• Development is generally considered to include training, structured 

on-the-job learning experiences, and education. Developmental 

programs can include experiences such as coaching, mentoring, or 

rotational assignments.

• Agency-wide includes all components, sub-agencies or offices within 

a cabinet department or independent agency.

For both objectives of the review, we compared OPM and CHCO 

practices against eight federal training investment leading practices, 

which are based on our prior studies; other expert studies; and statutory, 

regulatory and executive order training requirements.3

 

2

This definition of training includes mandatory training that governs how employees 

conduct themselves when carrying out their responsibilities, such as mandatory Equal 

Employment Opportunity training or Information Technology Security training.

(See table 1). OPM 

reviewed these criteria and agreed that they are practices that agencies 

should be implementing to support effective training investment decisions. 

While some of the leading practices we identified are related to statutory, 

regulatory and executive order training requirements, the leading practice 

questions we examined were not developed to assess whether agencies 

3

All of these practices are described in GAO, Human Capital: A Guide for Assessing 

Strategic Training and Development Efforts for the Federal Government, GAO-04-546G 

(Washington, D.C.: March 2004). We also used several studies from the Corporate 

Leadership Council. 
 

Page 3 GAO-12-878 Federal Training Investments 

are compliant with these requirements.4

Table 1: Leading Training Investment Practices

OPM officials agreed agency 

responses to our questions do not indicate whether or not agencies are in 

compliance with statutory, regulatory, and executive order training 

requirements. Therefore, we do not make determinations regarding 

compliance with these requirements in this review.

Agencies should implement:

Practice 1: (a) Identify the appropriate level of investment to provide for training and 

development efforts and (b) prioritize funding so that the most important 

training needs are addressed first.

Practice 2: Identify the most appropriate mix of centralized and decentralized 

approaches for its training and development programs.

Practice 3: Consider government-wide reforms and other targeted initiatives to improve 

management and performance when planning its training and development 

programs.

Practice 4: Have criteria for determining whether to design training and development 

programs in-house or obtain these services from a contractor or other 

external source.

Practice 5: Compare the merits of different delivery mechanisms (such as classroom or 

computer-based training) and determine what mix of mechanisms to use to 

ensure efficient and cost-effective delivery.

Practice 6: Track the cost and delivery of its training and development programs agencywide. 

Practice 7: Evaluate the benefits achieved through training and development programs, 

including improvements in individual and agency performance:

(a) Has a formal process for evaluating employee satisfaction with training. 

(b) Has a formal process for evaluating improvement in employee 

performance after training. 

(c) Has a formal process for evaluating the impact of training on the agency’s 

performance goals and mission.

Practice 8: Compare training investments, methods, or outcomes with those of other 

organizations to identify innovative approaches or lessons learned.

Source: GAO analysis based on prior GAO reports, other related expert studies, and federal training requirements. 

 

4

Appendix II includes a table that lists the laws, regulations, and Executive Orders that are 

related to the leading practices. 
 

Page 4 GAO-12-878 Federal Training Investments 

To obtain government-wide information on agency training investment 

practices, we obtained high-level information from a Council made up of 

CHCOs representing 27 agencies (known as the CHCO Council) through 

a questionnaire on their training investment practices and processes.5 To 

obtain additional perspective and insights on the training investment 

practices identified in the questionnaire, we discussed the responses with 

the CHCO Council and a Council made up of the Chief Learning Officers 

(CLO) from the 27 agencies (known as the CLO Council). We describe 

these Councils in greater detail in the background of this report. Based on 

the responses to the questionnaire and workforce size, we selected four 

agencies—the Department of Homeland Security (DHS), Department of 

Veterans Affairs (VA), Department of the Interior (DOI),6 and Department 

of Energy (DOE)—from which to obtain illustrative examples of how they 

implemented the training investment practices identified in the 

questionnaire. As part of our review of agency practices, we also obtained 

information on the steps that agencies are taking to identify and prioritize 

investment allocations for training required to implement the GPRA 

Modernization Act of 2010 (GPRAMA).7

To identify and assess OPM’s guidance and assistance to agencies on 

training investment strategies, we reviewed OPM training guidance, and 

relevant documentation on forums, workshops or other assistance and 

activities. In addition, we interviewed officials from the OPM offices with 

primary responsibility for providing training policy guidance and technical 

assistance to agencies. We compared this information to the leading 

practices identified in table 1. We also identified and described the steps 

 

5

We sent a questionnaire to all 27 members of the CHCO Council and received responses 

from all 27 members of the Council. For a full list of Council members, see the full 

discussion of scope and methodology in Appendix I.

6

In a separate review, we are also assessing the Department of the Interior’s human 

capital program, including training requirements for key oil and gas oversight positions and 

to what extent the department has developed and implemented plans to address ongoing 

training challenges.

7

GPRAMA mandated the Director of OPM to: (1) within a year of enactment, identify, in 

consultation with the Performance Improvement Council, the key skills and competencies 

needed by federal government personnel for developing goals, evaluating programs, and 

analyzing and using performance information for the purpose of improving government 

efficiency and effectiveness; (2) within 2 years of enactment, incorporate, as appropriate, 

key skills and competencies into relevant position classifications; and (3) within 2 years of 

enactment, work with each agency to incorporate the key skills identified into training for 

relevant employees at each agency Pub.L. No. 111-352, 124 Stat. 3866 (Jan. 4, 2011)
 

Page 5 GAO-12-878 Federal Training Investments 

that OPM has taken to identify the skills and training needed to implement 

performance management improvements, such as those required by 

GPRAMA, as a foundation for future agency training investments. 

However, we did not assess the effectiveness of OPM’s efforts to identify 

GPRAMA-related skills and actions to develop related training.

Based on information obtained from agencies and OPM, we assessed 

which leading training investment practices were being implemented by 

agencies and addressed by OPM guidance and assistance. We also 

identified the challenges or limitations reported by agencies to 

implementing the practices, and opportunities for improvement in agency 

processes and related OPM guidance.

We conducted this performance audit from December 2011 to September 

2012, in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives. Further details of our 

objectives, scope, and methodology are provided in appendix I.

The federal government has established a policy to develop its 

employees through training programs, to improve public service, increase 

efficiency and economy, and build and retain a force of skilled and 

efficient employees, among other things. In 1967, President Johnson 

signed Executive Order No. 11348, to provide agency heads and OPM 

with presidential direction on how training is to be carried out.8

 

8

Exec. Order No. 11348, “Further Training of Government Employees”, 32 Fed.Reg. 6335 

(April 20, 1967), as amended by Exec. Order No. 12107, 44 Fed. Reg.1055

(Dec 28, 1978).

Under 

Executive Order No. 11348, OPM is responsible for planning and 

promoting the development, improvement, coordination, and evaluation of 

training in accordance with chapter 41 of title 5 of the U.S. Code and the 

established policy. Chapter 41 of title 5 sets forth the statutory framework 

for federal government training and development. The executive order 

further requires OPM to identify functional areas in which new or 

expanded interagency training activity is needed and either conduct such 

training or arrange for agencies having the substantive competence to do 

Background
 

Page 6 GAO-12-878 Federal Training Investments 

so; as well as to coordinate interagency training conducted by and for 

agencies. It also requires OPM to assist agencies in developing sound 

programs and financial plans for training and provide advice, information, 

and assistance to agencies on planning, programming, budgeting, 

operating, and evaluating training programs. In addition to these activities, 

OPM provides advice and assistance to agencies on training and 

development programs. OPM’s Training and Executive Development 

(TED) group (a subcomponent within the Office of Executive Resources) 

is the primary office that provides policy direction and leadership to 

agencies in developing plans and strategies to implement training and 

development programs. It also provides agency guidance to ensure the 

government’s training and development programs support strategic 

human capital investments. The TED group provides assistance through 

two main mechanisms: guidance documents and technical assistance.

OPM has developed five guides that agencies can use as references for 

different aspects of making or reporting training investment decisions in 

the planning, designing, implementation, and evaluation phases of their 

training and development programs (see table 2).

Table 2: OPM’s Guidance Documents and Description 

Guidance document Description

Guide to Human 

Resources Reportinga This guide assists agencies with preparing and submitting human resources, payroll, and training data files to OPM’s 

Enterprise Human Resources Integration (EHRI) data 

warehouse.

(March 2012) b Agencies are required to provide updates of 

their training information to OPM’s electronic data collection 

system, including training costs.

Training Evaluation Field 

Guide (January 2011)

c

This guide provides information on methodologies for 

conducting training evaluations with the goal of identifying 

cost effective training initiatives that maximize mission 

accomplishments. OPM developed this guide based, in 

part, on the statutory requirements related to agency 

evaluation of training programs under 5 U.S.C . § 4103(c) 

as implemented by OPM regulations 5 CFR § 410.202. 

d

Draft Training Policy 

Handbook 

(December 2011)

This guidee

Guide for Collection and 

Management of Training 

Information 

(August 2008)

provides guidance to agencies on implementing 

training activities required by laws and regulations on 

training and development in the federal government. 

This document provides guidance to assist agencies in 

complying with the legal and regulatory instructions for 

collecting and managing federal training information. Under 

5 CFR § 410.601 agencies are required to maintain and 

report to OPM government-wide Electronic Data Collection 

System training data on its employees’ training and 

development activities.
 

Page 7 GAO-12-878 Federal Training Investments 

Guidance document Description

Guide to Strategically 

Planning Training and 

Measuring Resultsf

This document provides guidance to agencies in planning 

the allocation and use of training resources. Among many 

other things, it provides ideas and tools to help agencies 

identify potential training investments that will assist them in 

achieving their goals and accomplishing their mission, and 

provides approaches and models to assist agencies in 

evaluating training’s contribution to the organization. 

 

(July 2000)

Source: GAO analysis of OPM guidance documents.

a

http://main.opm.gov/feddata/ghrr/index.asp

OPM, Guide to Human Resources Reporting, accessed August 6, 2012, 

. 

b

The OPM’s EHRI program’s data warehouse is the government’s premier source for integrated 

federal workforce information. The system currently collects, integrates, and publishes data for 2 

million executive branch employees on a biweekly basis, supporting agency and government-wide 

analytics. 

c

5 C.F.R. § 410.601. 

d

This draft is an updated version of OPM’s Training Policy Handbook: Authorities and Guidelines 

published in May 11, 2007. 

e

OPM is currently updating the Training Policy Handbook to align it with the updated regulations on 

training (5 C.F.R. part 410) and management development (5 C.F.R. part 412). We obtained a draft of 

the updated Training Policy Handbook. 

f

In July 2000, OPM developed this guide in response to Exec.Order No.13111, Using Technology to 

Improve Training Opportunities for Federal Government Employees, 64 Fed. Reg. 2793 

(Jan. 12, 1999). 

The TED group also provides technical assistance on agency training 

investments through facilitating discussions and forums, and providing 

training to agencies’ human resources (HR) staff. For example, the TED 

group uses various web-based mechanisms—such as OPM’s website, 

OPM LISTSERV, OPM Federal Training and Development web site and 

OPM Federal Training and Development Wiki—to facilitate discussions 

between agencies on training investments and to share guidance with 

agencies. In addition to these facilitated discussions and forums, the TED 

group provides training to federal HR professionals in various areas, 

including activities that support making training investment decisions. For 

example, OPM provides training to HR staff through its partnership with 

the CHCO Council to operate HR University, which OPM officials and the 

HR University website report is the federal government’s single “one stop” 

training resource center for the HR professional throughout the federal 

government. HR University is an effort that is intended to achieve 

government-wide savings through pooling and sharing training resources 

and identifying the best HR training across government.9

 

9

HR University was established in 2010, in partnership with CHCO Council.
 

Page 8 GAO-12-878 Federal Training Investments 

Agencies have the primary responsibility for establishing, operating, 

maintaining, and evaluating their training programs in support of 

achieving their mission and goals. OPM regulations specify that agency 

employee developmental plans and programs should be designed to build 

or support an agency workforce capable of achieving agency mission and 

performance goals and facilitating continuous improvement of employee 

and organizational performance.10 Furthermore, Executive Order No. 

1134811

• Review periodically, but not less often than annually, the agency’s 

program to identify training needed to bring about more effective 

performance at the least possible cost;

states that agency heads must undertake several activities in 

support of developing employees, including:

• Conduct periodic reviews of individual employee’s training needs as 

related to program objectives;

• Conduct research related to training objectives required for program 

improvement and effectiveness;

• Plan, program, and evaluate training for both short and long-range 

program needs by occupations, organizations, or other appropriate 

groups;

• Establish priorities for needed training, and provide for the use of 

funds and man-hours in accordance with these priorities;

• Establish training facilities and services as needed;

• Extend agency training programs to employees of other agencies and 

assign his employees to interagency training whenever this will result 

in better training, improved service, or savings to the government.

The CHCO Council, established under the Chief Human Capital Officers 

Act of 2002, provides assistance to OPM and agencies in accomplishing 

federal human capital goals.12

 

105 C.F.R. § 410.201.

The 25-member CHCO Council is 

11Exec. Order No. 11348, §§ 302 and 303 

12Pub. L. No. 107-296, Title XIII, 116 Stat. 2287, 2288-89, § 1303 (Nov. 25, 2002) at 5 

U.S.C. § 1401 note.
 

Page 9 GAO-12-878 Federal Training Investments 

composed of the Director of the OPM, who serves as chairman; the 

Deputy Director for Management of the Office of Management and 

Budget (OMB), who acts as vice chairman; the CHCOs of the 15 

executive departments; and the CHCOs of 8 additional agencies 

designated by the OPM Director.13

• advocate and assure a culture of continuous learning and high 

performance, developing and implementing effective strategies to 

attract, develop, manage, and retain employees with superior abilities;

Additionally, the CHCO Council has an 

Executive Director from OPM who coordinates and oversees the activities 

of the council. The CHCO Council supports OPM in leading federal 

agencies in the strategic management of human capital, providing a 

forum for senior management officials to exchange HR best practices, 

and informing the dialogue on civil service reform in order to build and 

maintain an outstanding Federal workforce for the Nation. According to 

the CHCO Council’s charter, among other purposes, the council is to:

• identify human capital best practices and benchmarks, and apply 

those exemplars to their agencies and the federal government as a 

whole; and

• provide leadership in identifying and addressing the needs of the 

federal government’s human capital community, including training and 

development.

To help CHCOs implement their training goals, many of the 24 Chief 

Financial Officers Act agencies14

 

13The CHCO Council was established with 25 members; however, the Director of OPM 

may designate other members to the council. Such additional members may include, but 

are not limited to: (1) the CHCOs of other executive agencies and (2) members who are 

designated on an ex officio basis and who may be invited to contribute to projects, as 

particular skills and expertise are needed. The Council currently includes a representative 

for Small Agency Council (Corporation for National and Community Service) and the 

Office of the Director or National Intelligence (ODNI), as designated by the OPM Director. 

Although the Office of the Director or National Intelligence is not generally covered under 

title 5 of the U.S. Code, according to OPM, they were still designated to join the Council 

and follow many of the same training practices employed by other agencies.

and smaller agencies established Chief 

Learning Officers (CLOs). These officers subsequently formed an 

informal Chief Learning Officers (CLO) Council, which is a community of 

14 These 24 agencies are subject to the Chief Financial Officers Act. See listing of 

agencies at 31 U.S.C. § 901. 
 

Page 10 GAO-12-878 Federal Training Investments 

practice composed of federal CLOs or their equivalents who meet 

periodically to share best practices and create learning opportunities for 

agencies and organizations.

15 The purpose of the CLO Council is to 

provide a regular forum for CLOs to discuss and collaborate on high-level 

agency strategic and operational issues affecting the Federal learning 

and workforce development community of the federal government. These 

two Councils, in partnership with OPM are to play a key role in assisting 

agencies in the implementation of federal training and development 

efforts.

Many CHCOs reported that they are implementing leading practices we 

identified as being important to making strategic training and development 

investment decisions, especially regarding the delivery of training. These 

practices include determining the best mix of decentralized and 

centralized training, considering government-wide reforms when 

identifying their training needs, and measuring employee satisfaction with 

training, among other things. However, many CHCOs reported that they 

are not implementing the leading practices that would allow them to make 

more cost-effective training decisions, such as having an agency-wide 

process for prioritizing training investments so that the most important 

training needs are addressed first and comparing the merits of different 

delivery mechanisms (e.g. classroom or computer-based training) to 

determine what mix of mechanisms will be most efficient and costeffective. All of these practices are important to ensuring that training 

investments will be both effective and efficient in equipping federal 

employees to accomplish their agencies’ goals.

 

15The CLO Council derives its authority from its members, not their respective agencies. 

Its members, act in a collective capacity and agree to support and further the ends of the 

CLO Council’s vision and mission. CLO Council members are the most senior CLOs or 

equivalent of the federal agencies that make up the cabinet of the President and those 

who hold similar positions in independent agencies, boards, commissions, and the 

intelligence community.

CHCOs are 

Implementing Leading 

Practices Related to 

Training Delivery, but 

Not Practices that 

Support Making More 

Cost-Effective 

Training Investments
 

Page 11 GAO-12-878 Federal Training Investments 

Many CHCOs reported that they are implementing six of the leading 

practices that we identified as being important to making strategic training 

and development investment decisions, especially regarding the delivery 

of training, as shown in Table 3. However, regarding the leading practice 

related to tracking training investments agency-wide, we found that even 

those who reported that they track training agency-wide did not do so 

completely or reliably.

Table 3: The Leading Training Investment Practices CHCOs Reported Implementing

Number of agencies with 

the response

Leading training investment practices Yes No Sometimes

Practice 2: Identify the most appropriate mix of 

centralized and decentralized approaches 

for its training and development programs 

27 0 

Practice 3: Consider government-wide reforms and 

other targeted efforts when planning its 

training and development programs

23 4 

Practice 4: Have criteria for determining whether to 

design training and development programs 

in-house or obtain these services from a 

contractor or other external source 

15 12

Practice 6: Track the cost and delivery of its training 

and development programs agency-wide 

16 11

Practice 7: Evaluate the benefits achieved through 

training and development programs, 

including improvements in individual and 

agency performance 

 

(a) Has a formal process for evaluating 

employee satisfaction with training

25 1 1 

(b) Has a formal process for evaluating 

improvement in employee performance after 

training

13 12 2 

Practice 8: Compare training investments, methods, or 

outcomes with those of other organizations 

to identify innovative approaches or lessons 

learned.

24 3 

Source: GAO summary of CHCO responses to a GAO questionnaire

CHCOs Reported 

Implementing Leading 

Practices that Support 

Making Strategic Training 

Delivery Decisions
 

Page 12 GAO-12-878 Federal Training Investments 

All CHCOs reported that their agencies have implemented this practice. 

We have previously reported that, while neither approach fits every 

situation, agencies need to consciously think about the advantages and 

disadvantages of using centralized and decentralized approaches, 

particularly for the design of training and development programs.16

In the four agencies that we selected for review to obtain illustrative 

examples of how they implemented the training investment practices, the 

CHCOs or their representatives reported that their agencies made a 

decision to have both centralized and decentralized processes because 

they believe that the components or sub-agencies are more 

knowledgeable about their mission-specific training needs, while the 

central human capital staff can add the most value by managing 

investment decisions for more general training across the department.

VA—which was one of the four agencies that we selected—established a 

corporate university known as the Veterans Affairs Learning University 

(VALU) to provide training to all VA employees. VALU provides training 

primarily in general areas such as leadership, management and 

supervision, as well some career and technical training. VALU offers 

Centralizing design can enhance consistency of training content and offer 

potential cost savings. A decentralized approach to training design can 

enable agencies to tailor training programs to better meet local and 

organizational unit needs. Agencies with decentralized approaches often 

embed training representatives within their business lines and field 

structures to assist in coordination of training efforts, including design and 

development. Nineteen of the 27 agencies reported that they have both 

centralized and decentralized training processes, while eight reported 

having completely decentralized training processes. Most of these 

agencies reported that their CHCOs or CHCO staff typically make 

centralized training decisions, while the leadership within the 

components, subagencies or offices make mission-specific training 

decisions. In the questionnaire responses, CHCOs identified a range of 

officials who are involved in making training investment decisions at the 

corporate and sub-agency level, including CHCOs and their staff, chief 

management officers, chief executive officers, budget officers, chief 

information officers, and others. A number of agencies also reported that 

advisory or oversight boards or training universities within their agency 

are involved in making training investment decisions.

 

16GAO-04-546G. 

Identifying the Most 

Appropriate Mix of Centralized 

and Decentralized Approaches 

for its Training and 

Development Programs 
 

Page 13 GAO-12-878 Federal Training Investments 

training to the administrations and staff offices through a request process 

that is based on the training needs that the administrations and staff 

offices identify. Those training needs are required to be aligned to VA 

critical training areas.17 An Enterprise Training Advisory Board, 

established in April 2012, also advises the Dean of VALU on the impact of 

training, potential training development, and methods of delivery.

18

 

17 On June 5, 2012 an enhanced training requirements process was launched with an 

emphasis on closing identified skill gaps based on training needs assessments. 

According to VA officials, this process incorporated lessons learned, best practices, and 

prioritization criteria to align training with transformational goals and the VA Strategic plan. 

However, another tier of training is also provided within VA’s three 

administrations—the Veterans Health Administration (VHA), Veterans 

Benefits Administration (VBA), and National Cemetery Administration. 

Each administration independently makes training investment decisions 

and provides training to its employees in mission-specific and some 

general and mandatory areas. The leadership of each administration 

makes decisions about the level and prioritization of these training 

investments. For example, at the VHA the Associate Deputy Under 

Secretary for Health (or equivalent) and, subsequently, the Deputy Under 

Secretary for Health assess the training requested by their offices against 

various criteria, including whether training requests are aligned with and 

support VA and VHA strategic goals, objectives, strategies, initiatives, 

and performance improvement goals. During each review these officials 

prioritize the requests through a voting process, and forward selected 

training to the next level. Ultimately, the training is sent for approval to the 

Under Secretary for Health and the VA Chief of Staff. The other three 

agencies that we met with also reported having both centralized and 

decentralized processes for making mission-specific training investment 

decisions. However, most often, decentralized training decisions were not 

required to be vetted with department level leadership for these three 

agencies.

18According to the Enterprise Training Advisory Board charter, which was signed on April 

10, 2012, the group is an enterprise training and development management advisory 

board. It serves as a high-level executive review and recommendation group to advise the 

Dean of VALU on business requirements and on the impact of planned policies and 

procedures for leadership and mission-critical, non-clinical career technical training and 

development. The Enterprise Training Advisory Board shall provide advice to the Dean of 

VALU the impact of training and support communication regarding requested or 

recommended training plans and projects. However, it does not have authority to approve 

training investment decisions.
 

Page 14 GAO-12-878 Federal Training Investments 

Nearly all CHCOs in our review reported that they have a process for 

these considerations. We have previously reported that when planning 

training and development efforts, agencies should look to the actions of 

the administration, Congress, and internal and external auditors by 

considering administration priorities, legislative reforms, and major 

management challenges that might shape agency priorities and strategies 

for training and development.19

At DOE, another agency we selected for review to obtain illustrative 

examples, officials reported that they have identified the training that the 

department currently offers and will need to offer to implement GPRAMA. 

The Secretary issued a memo to DOE employees on GRPAMA’s 

implementation and is holding town hall meetings on improving 

organizational performance. Another effort that DOE expects to support 

the implementation of GPRAMA is the Goals-Engagement-AccountabilityResults (GEAR) model that OPM and OMB are helping to pilot in DOE 

and four other federal agencies, which includes efforts to improve 

employee performance, among other things.

As an administration focuses its efforts on 

addressing its priorities, agencies can benefit by having mechanisms or 

processes for considering whether and to what extent these initiatives 

could be linked to employees’ skills and competencies and the related 

training and development approaches that might be needed. Twentythree of the 27 CHCOs who responded to our questionnaire reported 

having such a process in place. For example, 16 of the CHCOs reported 

that they are already setting investment allocations or training priorities to 

implement GPRAMA.

20

 

19

According to DOE officials 

and related documentation, DOE’s GEAR implementation plan includes 

aligning employee performance management with organization 

performance management and developing training to support these 

goals, which along with initiating knowledge sharing activities, will 

promote improvement of DOE’s organizational performance.

GAO-04-546G. 

20Beginning in late May 2011, a workgroup of the National Council on Federal LaborManagement Relations (LMR) partnered with members of the CHCO Council to develop a 

new model of employee performance management, referred to as GEAR. GEAR focuses 

on articulating a high-performance culture, aligning employee performance engagement 

with organizational performance management, implementing accountability at all levels, 

and create a culture of engagement. OPM is piloting GEAR at five agencies—the Housing 

and Urban Development, the DOE, the Coast Guard, OPM and VA. 

Considering Government-wide 

Reforms and Other Targeted 

Initiatives When Planning its 

Training and Development 

Programs
 

Page 15 GAO-12-878 Federal Training Investments 

Many of the CHCOs in our review reported having criteria for this 

purpose. Training can be provided by the agency itself, another 

government agency, a school, a manufacturer, a professional association, 

or other competent persons or groups in or outside of government. To aid 

in making these decisions, agencies should try to develop clear criteria for 

determining when to contract for training and development services. We 

have previously stated that factors that agencies should consider in these 

decisions include the capability of in-house staff to develop and 

implement the training; the prior experience, capability, and stability of 

possible providers in the marketplace; and agency limitations on cost, 

time, and resources21

One agency that we selected for review to obtain illustrative examples 

was DOI, which reported implementing this practice, however, the extent 

to which this decision-making process is implemented agency-wide is 

unclear. In its questionnaire response, DOI’s CHCO reported that DOI’s 

Office of Strategic Employee and Organizational Development has 

responsibility for offering corporate training through DOI’s university. This 

office decides whether to “make or buy” departmentwide training. When 

we met with DOI officials in the course of our review, they explained that 

although almost all courses are delivered by vendors because DOI has 

no internal trainers, they do have a small cadre of instructional designers 

who can develop some e-Learning courses. Decisions on whether to 

develop the courses internally are based on various criteria, including 

whether a course can be developed quickly, does not require a significant 

amount of content development, and subject matter experts can be 

provided to support the course development. Although this department 

level process is useful, DOI officials did not know if the bureaus within the 

department consistently use a “make or buy” approach. They reported 

that the larger bureaus have some capacity for in-house development 

while the smaller bureaus do not have this capability.

Of the 27 CHCOs included in our questionnaire, 15 

reported that they have criteria for determining whether to design training 

and development programs in-house or obtain these services from a 

contractor or other external source.

Many CHCOs reported that their agencies implement this practice, 

although most of the CHCOs who reported that they do not track training 

investments agency-wide were leaders of the agencies with the largest 

 

21GAO-04-546G. 

Have Criteria for Determining 

Whether to Design Training and 

Development Programs inhouse or to Obtain These 

Services from a Contractor or 

Other External Source

Track the Cost and Delivery of 

its Training and Development 

Programs Agency-wide
 

Page 16 GAO-12-878 Federal Training Investments 

workforces.22 We have previously reported that to obtain a 

comprehensive determination of the costs of these initiatives, agencies 

need to find ways around barriers that prevent them from fully and 

accurately identifying the expenses associated with all components of 

their training and development processes.23

In our review of agencies to obtain illustrative examples of practices, two 

of the selected agencies, DOE and VA , were among the CHCOs who 

reported in their questionnaire that they track training investments 

agency-wide. However, our review found both these agencies may not do 

so completely or reliably. Both agencies identified the same limitations 

and challenges as the agencies whose CHCOs reported that they do not 

have a process to track training agency-wide. For example, DOE and VA 

officials reported experiencing challenges in tracking training costs 

agency-wide because of their multiple learning management systems and 

inconsistent practices in tracking training data.

These costs can include 

expenses for instructional development; participant and instructor 

attendance; facility, material, and equipment costs, and travel and per 

diem expenses. To track the cost and delivery of training and 

development programs, agencies need credible and reliable data from 

learning management systems as well as accounting, financial, and 

performance reporting systems. To the extent possible, agencies also 

need to ensure data consistency across the organization (such as having 

data elements that are pulled from various systems representing the 

same type of information). Variation in the methods used to collect data

can greatly affect the analysis of uniform, quality data on the cost and 

delivery of training and development programs. In response to our 

questionnaire, 16 CHCOs reported that they track training investments 

agency-wide.

24

 

22The eleven agencies that reported that they do no track training investments agencywide represented 65 percent of the federal workforce. This figure is based on CRS 

calculations of the size of each agency’s workforce, as of December 2010.

Officials from DHS and 

DOI, who reported in the questionnaire that they could not track training 

investments agency-wide, described similar challenges to those 

experienced by DOE and VA. Moreover, officials from all four agencies 

were unaware of the total amount their agencies invest in federal training 

23GAO-04-546G. 

24A learning management system is a software application that automates the 

administration, tracking, and reporting of training events.
 

Page 17 GAO-12-878 Federal Training Investments 

and stated that they could not provide reliable training data to OPM, 

which requests these data to address its government-wide training 

responsibilities. Under OPM regulations, agencies are required to 

maintain data on training activities and expenditures and submit these 

data to OPM.

As an example of challenges tracking training investments, DHS reported 

that it is unable to track or better leverage training investments across the 

department because of the nine, major incompatible Learning 

Management Systems that it uses to track training throughout the agency.

We highlighted these same challenges in a 2005 report on DHS training, 

noting that the lack of common management information systems and the 

absence of commonly understood training terminology across 

components, among other things, may impede the agency’s ability to 

achieve its training goals.25 According to more recent documentation on 

the limitations of DHS’ tracking systems, the components’ disparate 

systems currently limit them from sharing useful training information 

across the department, effectively aggregating training data agency-wide, 

and reporting complete training investment information to OPM. As a 

result, DHS is seeking to purchase a single learning management 

system.26

Even when agencies had a single training information system, the 

components may not consistently use them to track training investments 

because of inconsistent coding schemes for tracking similar training 

activities. For example, even though DOI has a single system for tracking 

training information, officials reported that their human capital office must 

rely on employees or data stewards27

 

25GAO, Department of Homeland Security-Strategic Management of Training Important 

for Successful Transformation, 

to input their training data and 

some cost data may not be included, such as training travel costs, or 

GAO-05-888 (Washington, D.C.: Sept. 23, 2005). 

26In addition to challenges tracking training data across components, a recent GAO report 

highlighted that a DHS component, the Federal Emergency Management Agency (FEMA) 

does not track how much it spends within its own component on the Disaster Assistance 

Employee (DAE) training, which hinders FEMA’s ability to plan for future training. See 

GAO, DISASTER ASSISTANCE WORKFORCE-FEMA Could Enhance Human Capital 

Management and Training, GAO-12-538 (Washington, D.C.: May 25, 2012). 

27Data Stewards are those individuals ultimately responsible for the definition, 

management, control, integrity or maintenance of a departmental or Enterprise data 

resource.
 

Page 18 GAO-12-878 Federal Training Investments 

certain types of training may not be entered, such as conferences. Such 

costs are sometimes paid directly by an employee’s immediate office 

using a government credit card and are not tracked as training. In 

addition, learning management system training data are often not 

reconciled with DOI’s financial expenditure data because, until recently, 

their financial systems have not captured education tuition and training 

fees, and are still unable to track training travel costs. Therefore DOI’s 

cost data are most likely incomplete. Similarly, officials from DOE, DHS, 

and VA reported that they are aware of some inconsistencies in whether 

some types of training, such as conferences, are entered into their 

learning management systems. They also stated that there are 

inconsistencies in how agency components capture and code workforce 

training into their system because they lack a common definition for what 

types of activities should be considered training or have varying coding 

schemes or tools for capturing the cost. For example, some organizations 

use a procurement request or obtain a contractor to deliver training and 

do not document these costs using the standard government form for 

tracking data or in their learning management systems. In other cases, 

training investment data are captured using different coding in various 

learning management systems and financial systems. Some officials 

report that reconciling these data would be difficult. For example, DOE’s 

chief financial officer reported that it takes a couple of months to gather 

training investment data from DOE’s various systems, partly because the 

systems have inconsistent coding for these data. DOE’ officials reported 

that changing financial codes to reconcile training data would be time 

consuming and expensive because their financial systems are 20 years 

old. However, after years of highlighting this challenge, they are seeking 

approval to make such changes. Officials from the four agencies 

generally reported that, as a result of all of these factors, there is no 

overarching awareness or oversight of how much is spent on training 

investments and for which activities .

Nearly all CHCOs reported having a formal process to evaluate employee 

satisfaction with training, but fewer had processes to evaluate the impact 

of training on employees or agency performance. We have previously 

reported that it is increasingly important for agencies to be able to 

evaluate their training and development programs and demonstrate how 

these efforts help develop employees and improve the agencies’ 

performance because it can aid decision makers in managing scarce 

resources, and provide credible information on how training and 

Evaluate the Benefits Achieved 

Through Training and 

Development Programs, 

Including Improvements in 

Individual and Agency 

Performance 
 

Page 19 GAO-12-878 Federal Training Investments 

development programs have affected organizational performance.28 To 

do so agencies need to develop evaluation processes that systematically 

track the cost and delivery of training and development efforts and assess 

the benefits of these efforts. Training and development programs can be 

assessed by measuring (1) participant reaction to the training program, 

(2) changes in employee behavior or performance; and (3) the impact of 

the training on program or organizational results, which may include a 

return on investment assessment that compares training costs to derived 

benefits.29

Officials from the four agencies that we interviewed reported that they all 

assess employee’s reaction to training and sometimes assess changes in 

employee performance. For example, officials from DOE reported that 

they evaluate all the training that they offer by surveying participants’ 

reactions to the training—which can include their feedback on the 

effectiveness of the instructor, the topics, the presentation style, the 

schedule, audiovisuals, and other subjects—and use this information to 

make revisions to the program courses. Documents that we reviewed on 

training evaluations identified updates or revisions made to course 

materials and tests to improve their effectiveness, based on training 

Some of these methods can help provide better value through 

identifying areas for continuous improvement in training programs. We 

consider the processes for conducting these evaluations to be formal 

when they are systematically conducted throughout the agency, have 

established guidelines and criteria that govern how they are implemented 

and are documented. However, CHCOs may also have other criteria for 

determining what is considered a formal process, based on their 

agencies’ environment. We asked CHCOs about their formal processes 

for conducting the three levels of evaluation listed earlier, which are the 

common types of evaluations. Many CHCOs reported routinely 

implementing the first two, but not the third (which we discuss later in this 

report). Twenty-five of the 27 CHCOs included in our questionnaire 

reported that they measure employee satisfaction; and a little more than 

half reported that they measure improvement in employee performance.

 

28GAO-04-546G. 

29The Kirkpatrick model of evaluation is a popular model of evaluation that includes 

measuring (1) participant reaction to the training program; (2) changes in employee skills, 

knowledge, or abilities; (3) changes in employee behavior or performance; and (4) the 

impact of the training on program or organizational results, which may be split into a fifth 

level that includes a return on investment assessment that compares training costs to 

derived benefits. For purposes of our review, we focused on three of these areas.
 

Page 20 GAO-12-878 Federal Training Investments 

feedback and policy updates. As an example of evaluating the impact of 

training on employee performance, DOI officials stated that, while they do 

not have an agency-wide process, some of their organizations—such as 

those within the Bureau of Land Management and National Park Service 

use an online evaluation tool to assess the impact of training courses on 

employees’ abilities to perform tasks, about 6 weeks after a course has 

been completed. According to the official, at this time, the process is not 

used department-wide, but the agency is looking into how it may be able 

to do so starting in fiscal year 2013. According to DOI’s CLO, establishing 

this link between training, employee competencies and mission critical 

occupation work is one that DOI is targeting for improvement.

Nearly all CHCOs reported that they implement this practice. We have 

previously reported that there are many ways to help improve 

performance, so it is important for agencies to continually look to others to 

identify innovative approaches that may relate to their training and 

development efforts. Within the context of that agency’s unique 

environment and situation, an agency can compare its investments, 

approaches, and outcomes with those of public and private organizations 

that are undertaking notably innovative and effective training and 

development efforts. Agencies can uncover weaknesses in their training 

and development strategies that need improvement and identify new 

ideas, mechanisms, and metrics that they could employ.30

Officials from the two agencies we asked to provide examples of this 

practice described this process as occurring informally through 

interactions with other CLO Council members. For example, DHS officials 

that we met with reported that they meet with other agencies to share 

best practices and recommend vendors during breaks or after the CLO 

meetings. The officials said that examples of sharing ideas on new 

training programs included recent discussions by OPM and agencies on 

the GEAR pilot program lessons learned and new courses for developing 

supervisors. While two agencies reported having informal interactions 

with other agencies to share and compare training information, none of 

the agencies that we met with described efforts to benchmark their 

Twenty-four of 

the 27 CHCOs included in our questionnaire reported that they compare 

training investments, methods, or outcomes with those of other 

organizations to identify innovative approaches or lessons learned.

 

30GAO-04-546G. 

Compare training investments, 

methods, or outcomes with 

those of other organizations to 

identify innovative approaches 

or lessons learned
 

Page 21 GAO-12-878 Federal Training Investments 

practices with agencies or other relevant entities. We have previously 

reported that benchmarking can help agencies determine who is the very 

best, who sets the standard, and what that standard is.31

Many CHCOs reported that they are not implementing the leading 

practices that would allow them to make more cost-effective training 

decisions, as shown in Table 4.

Table 4: Leading Training Investment Practices that Many CHCOs Reported Not 

Implementing Routinely

Leading training investment practices

Number of CHCOs with 

the response

Yes No Sometimes

Practice (1a): Identify the appropriate level of investment to 

provide for training and development efforts

11 16

Practice (1b): Prioritize funding so that the most important 

training needs are addressed first

12 15

Practice 5: Compare the merits of different training delivery 

mechanisms and determine what mix to use to ensure 

efficient and cost-effective delivery

11 16

Practice 7(c): Have a formal process for evaluating the impact 

of training on the agency’s performance goals and mission

8 a 8 11

Source: GAO analysis of CHCO responses to a GAO questionnaire. 

a

One agency that reported “I don’t know” was included in the category “No”. 

Many CHCOs included in our review reported that they have not 

implemented this practice. We have previously stated that, to determine

the best ways to leverage investments and establish priorities, agencies 

can develop an annual training plan that targets developmental areas of 

greatest need and that outlines the most cost-effective training 

approaches to address those needs.32

 

31

When assessing investment 

opportunities for its training plan, the agency ought to consider the 

competing demands confronting the agency, the limited resources 

available, and how those demands can best be met with available 

resources. If training is identified as a solution to improve agency 

GAO-04-546G. 

32GAO-04-546G. 

Many CHCOs are Not 

Implementing the Leading 

Practices that Support 

More Cost-Effective 

Training Investment 

Decisions

Identify the Appropriate Level 

of Investment to Provide for 

Training and Development 

Efforts and Prioritize Funding 

So That the Most Important 

Training Needs are Addressed 

First
 

Page 22 GAO-12-878 Federal Training Investments 

performance, agencies can prioritize training using criteria, such as 

expected demand for the investment from internal sources, availability of 

resources to support the effort, potential for increased revenue, and risk 

of unfavorable consequences if investments are not made. Given current 

budget constraints, agencies may also want to prioritize training that has 

the potential to improve their efficiency. Developing a business case for 

training and development that includes this information sets forth the 

expected costs and benefits of the investments and provides decision 

makers with essential information they need to allocate necessary 

resources. Furthermore, under Executive Order No.11348 and OPM 

regulations, agencies are to establish training priorities, although 

agencies are not specifically instructed to establish an agency-wide 

process to do so.33

In our meetings with officials from the DOE, DOI, DHS, and VA as well as 

the CLO council, agency officials cited several reasons for why they do 

not establish a level of training investment agency-wide or prioritize 

training agency-wide. Some of the reasons were described as purposeful 

decisions not to do so and other reasons were described as limitations in 

their ability to do so. First, CHCOs elect to establish and prioritize training 

investments for centralized training and are often not involved in the 

investment decisions made for specific training within the components or 

offices, as we previously described. In addition, large components or subagencies often have autonomy over their training budgets because the 

budgets are appropriated directly to them from Congress. As a result, 

CHCOs and their staff are often unaware of how much these components 

spend for training and do not have input into these decisions. Component 

and sub-agency heads often act autonomously and are not required to 

communicate with the CHCO about these decisions. Further, because of 

limitations in internal tracking systems for training (which we discussed 

earlier in this report), CHCOs do not have information on all of the training 

that is completed in their agency and the related costs.

Of the 27 CHCOs questionnaire responses, 16 

CHCOs reported that they do not set a level of investment agency-wide 

and 15 CHCOs reported that they do not prioritize training agency-wide.

Officials from various agencies involved in the CLO Council and three of 

the four agencies that we individually met with reported having a lack of 

 

33Section 303(e) of Exec. Order No.11348 and 5 C.F.R. § 410.201(c) requires agency 

heads to establish priorities for training and allocate resources according to those 

priorities. 
 

Page 23 GAO-12-878 Federal Training Investments 

visibility into the prioritization and level of training investments throughout 

their agencies, which they reported limits their ability to better leverage 

and reduce duplication in training investments their agencies. Officials in 

the agencies that we met with reported that, although they believe that 

their components or organizational elements are more capable of making 

training decisions related to their specific missions, the lack of 

coordination and communication on training investments and priorities 

has led to some duplicative and ineffective training investments in their 

departments. For example, senior human capital officials in DOI reported 

that the department’s leadership, including the CHCO are not aware of 

the department’s overall training investments agency-wide and have no 

formalized mechanism for ensuring accountability for how the funds are 

used. They are aware that bureaus are buying duplicative training or 

offering similar training classes that are of varying effectiveness—which is 

resulting in inefficient training investments. For example, one bureau 

recently independently contracted with an external provider for mid-level 

manager leadership training that was already offered at DOI’s university 

and paid $50,000 more than DOI University charges. According to 

officials, this is a common problem. In addition to duplicative training 

courses, in some cases, bureaus are duplicating the creation of new 

training facilities. For example, a regional director of an DOI bureau built a 

training classroom with a computer lab, despite having access to existing 

computer labs within the complex where he worked and also at DOI 

facilities a few miles way. Further, according to the officials, because it is 

common practice for each bureau to independently secure training, there 

is no consistency, little quality control, and no maximization of 

procurement tools (such as blanket purchase agreements) across DOI.34

 

34A blanket purchase agreement is a simplified acquisition method that government 

agencies use to fill anticipated repetitive needs for supplies or services. Essentially, 

blanket purchase agreements are like “charge accounts” set up with trusted suppliers. 

According to the General Services Administration, BPAs offer an excellent option for 

federal agencies, providing convenience, efficiency, and reduced costs. 

In order to address these challenges, DOI has formed a one-time 

departmentwide task force known as the Department Innovation and 

Efficiency Team for Training. This task force was expected to identify: 

potential duplication in training, funds expended in training delivery, and 

the cost of travel and facilities, among other things. In July 2012, the 

committee made recommendations to the CLO on opportunities to 

generate efficiencies and savings in training operations. DOI’s Office of 
 

Page 24 GAO-12-878 Federal Training Investments 

Strategic Employee and Organization Development is developing action 

plans to address the committee’s recommendations.

Officials from DHS also reported experiencing similar challenges with 

duplicative or ineffective training investments in their agencies. Some of 

these challenges are long standing. For example, seven years ago we 

reported that DHS’s two-tier training process (component and 

departmentwide) and lack of communication throughout the department 

on the availability of some training programs and resources were 

challenges that could impede its ability to achieve departmental training 

goals and efficiencies.35

Many CHCOs that responded to our questionnaire reported that they do 

not compare the merits of different training delivery mechanisms. Our 

past research and that of others has shown that agencies should 

deliberatively consider the options for delivering training and consider 

essential issues, such as the goals and objectives for the training, the 

type of audience intended for the training, the nature of the training 

DHS is still taking steps to address this on-going 

challenge. In June 2005, DHS formally chartered a Training Leaders 

Council (TLC) and recently revised its charter in June 2011. The TLC is 

made up of senior training leaders from each component, and 

representatives from headquarters to serves as an advisory and 

collaborative community of practice to promote effective and efficient 

training, education, and professional development opportunities to DHS 

employees. According to DHS’ Human Capital leaders, while this group 

does not set or prioritize training investments, it provides a forum for 

exchanging useful information about common challenges and training 

practices, which helps in making more efficient use of existing agency 

resources. DHS also established the Human Resource Information 

Technology Executive Steering Committee, made up of management 

chiefs and HR and information technology leadership across DHS in 

2010, and included TLC leadership as members in July 2011. This group 

makes some funding decisions related to some training investments, such 

as their recent decision to fund the purchase of a single learning 

management system for the entire department. However, according to 

DHS officials, because DHS has multiple congressional committees and 

subcommittees from which the components receive funding and training 

direction, coordinating training investments remains challenging.

 

35See GAO-05-888. 

Compare the Merits of 

Different Training Delivery 

Mechanisms and Determine 

What Mix to Use to Ensure 

Efficient and Cost-Effective 

Delivery 
 

Page 25 GAO-12-878 Federal Training Investments 

content, the availability of technology and resources, and the timing for 

delivering the training.36

In our meetings with DHS and VA to obtain illustrative examples, DHS 

officials reported that their current learning management systems do not 

allow them to mine information on the different delivery mechanisms used 

throughout the department and to assess and compare their 

effectiveness. According to the officials, they could obtain this information 

manually, but it would be a very labor intensive process. Therefore, it is 

not done. In contrast, VA officials informed us that they are assessing 

different delivery mechanisms for training and conferences offered by 

VALU because they recognize that opportunities exist to offer more 

efficient mechanisms (such as e-learning). Moreover, VHA, which has the 

largest workforce in the department, builds into its initial investment 

decision-making process considerations of which delivery methods will be 

most effective and efficient, and subsequently evaluates employee 

satisfaction with the various delivery methods to inform future investment 

decisions. Without processes such as these, agencies that do not 

compare the merits of different training delivery mechanisms have limited 

information for determining what mix of methods provides the most 

efficient and effective delivery of federal training.

Agencies can use a variety of instructional 

approaches to achieve learning—in the classroom, through distance 

learning, or in the workplace. When warranted, agencies should also 

consider blended learning that combines different teaching methods (e.g. 

Web-based and instructor-led) within the same training effort and provide 

trainees with the flexibilities to choose among different training delivery 

methods while leveraging resources in the most efficient way possible. 

When assessing delivery options, agencies can try to achieve economies 

of scale and avoid duplication of effort by taking advantage of existing 

course content or training, such as sharable on-line courses or 

multiagency training programs. However, In the responses to our 

questionnaire, 16 of the 27 CHCOs reported that they do not compare the 

merits of the different training delivery mechanisms in their agency.

 

36See GAO-04-546G and Corporate Leadership Council Learning and Development: LDR 

a Guide to Selecting the Most Suitable Development Method (Arlington, VA: Corporate 

Executive Board, 2011). 
 

Page 26 GAO-12-878 Federal Training Investments 

Most CHCOs reported that their agencies do not have a routine formal 

process to implement this practice. As we previously mentioned, it is 

increasingly important for agencies to be able to evaluate their training 

and development programs and demonstrate how these efforts help to 

improve the agencies’ performance, and to assist them in making more 

effective decisions about how to allocate scarce resources. Agencies are 

required by statute and OPM implementing regulations to evaluate how 

well training programs contribute to mission accomplishment and meet 

organizational performance goals.37 We have identified having a formal 

process for this evaluation as a leading practice. However, there are 

some understandable limitations to regularly and formally implementing 

this practice. For example, some agency officials that we met with 

reported that the cost and time required can be significant for obtaining 

results of evaluations of training that measure the impact on agency 

performance goals. As a result, they can only conduct this level of review 

for training that they identify as highly important to key areas of their 

mission. We have previously reported that not all training and 

development programs require, or are suitable for, higher levels of 

evaluation.38

 

375 U.S.C. § 4103(c) requires such evaluation on a regular basis whereas OPM’s 

implementing regulations specify that such evaluations are to be conducted annually. 5 

C.F.R. 410.202. 

For example, it may be ineffective to try to measure the 

impact of training in an area that is still undergoing other significant 

changes that could affect relevant performance goals, such as changes in 

related policy and management structure. We recognize that higher levels 

of evaluation (such as evaluating the impact on organizational 

performance or return on investment) can be challenging to conduct 

because of the difficulty and costs associated with data collection and the 

complexity in directly linking training and development programs to 

improved individual and organizational performance. Factors to consider 

when deciding the appropriate level of evaluation include estimated costs 

of the training effort, size of the training audience, management interest, 

program visibility, and the anticipated “life span” of the effort. Each 

agency will need to consider the feasibility and cost-effectiveness of 

conducting these in-depth evaluations, along with budgetary and staffing 

circumstances that may limit the agency’s ability to complete such 

evaluations. Given the current budget constraints that agencies face, 

making thoughtful tradeoffs regarding how to target costly evaluation 

38GAO-04-546G. 

Have a Formal Process for 

Evaluating the Impact of 

Training on the Agency’s 

Performance Goals and 

Mission
 

Page 27 GAO-12-878 Federal Training Investments 

reviews is a sensible approach. While it is important to prioritize reviews 

of training, 8 of the 27 CHCOs that responded to our questionnaire 

reported that they do not have a formal process for evaluating the impact 

of their training on their agency’s performance.

For example, the CHCO at DOE reported in our questionnaire that DOE 

does not implement this practice. We met with the CLO from DOE who 

informed us that DOE does not have a formal process for implementing 

this practice because the agency does not have a systematic 

documented approach for conducting this level of review. Moreover,

evaluation data are not collected in a way that allows it to be aggregated 

into a comprehensive assessment of its impact on the agency’s overall 

mission. For example, different organizations within DOE conduct reviews 

to assess the impact of training on their goals, but they are not captured 

in an automated system and the methodologies that DOE organizations 

use to conduct these reviews vary. As an illustration, DOE organizations 

that work with nuclear material evaluate the technical training that they 

provide to their employees against required certification and mission 

goals. However, the organizations conduct these evaluations differently, 

and because of these varied methodologies and lack of automated results 

data, it is difficult to aggregate the reviews into an assessment of how 

training has affected DOE’s overall training and mission goals. Similarly, 

the CLO’s office evaluates cross-cutting training for employee satisfaction 

and employee performance at DOE, but does not effectively or 

consistently evaluate its impact on agency goals. According to the CLO, 

to assist them in developing a more systematic formal process, they are 

participating in OPM training on developing training evaluations and in the 

GEAR pilot program—which is intended to better link employee 

performance to organizational goals.

In contrast, VA’s training review processes illustrate that agencies that 

have a formal process for assessing the impact of training on their 

performance mission and goals can use it to make better training 

investment decisions. VA recently assessed the return on investment of 

its corporate training and, the department’s Administrations recently 

evaluated the impact of mission-specific training on their performance 

goals. In January 2012, VA evaluated the monetary and mission-related 

benefits of training that was implemented under its Human Capital 
 

Page 28 GAO-12-878 Federal Training Investments 

Investment Plan.39 According to the return on investment assessment and 

report developed by VALU and VHA’s National Center for Organization 

Development, VA’ s two-year $577 million investment in training and 

development under VA’s Human Capital Investment Plan has resulted in 

$604 million dollars in savings that are tied to reductions in costly VA 

turnovers, fewer overdue accounts renewable, and fewer equal 

employment opportunity complaints.40 The report also states that VA has 

gained non-financial returns, such as faster benefits processing, 

increased veteran hiring programs, and improved patient satisfaction.

According to VALU officials, they have used details in this report along 

with other factors to make decisions about future training and 

development investments. Similarly, for its mission-specific training, VHA 

recently conducted an in-depth review of training provided to Patient 

Aligned Care Teams to improve their collaborative delivery of care to 

patients. The evaluation assessed the training participants’ satisfaction, 

skill acquisition, application on the job, and impact on VHA’s business. 

The assessment ultimately determined that the training was successful in 

addressing the desired behavior changes in the work place and that key 

organizational results were influenced by the training, but it also identified 

some improvements that VHA could make.41

 

39According to documentation on VA’s Human Capital Investment Plan, the Plan is a 

human resource management transformational initiative to develop the department’s 

human capital into a proactive, forward looking, well trained and professional workforce 

focused on delivery of high quality service to Veterans and their families. The plan 

includes a focus on improving the recruitment, hiring and retention of VA employees; 

investing in people development (e.g., leadership training); supporting and developing the 

capabilities of VA’s Senior Executive Service (SES) employees; establishing a Veteran’s 

Employment initiative to support veteran’s recruitment, retention and reintegration; and, 

striving to build a broad set of human resources capabilities to support the delivery of 

quality care and benefit services to Veterans and their families. 

40Department of Veterans Affairs’ Human Capital Investment Plan (HCIP) Impact 

Assessment /Year One: HCIP Monetization Summary, January 2012.

41We did not independently verify the reported cost savings or performance improvements 

reported in the VALU or VHA studies.
 

Page 29 GAO-12-878 Federal Training Investments 

OPM guidance and assistance to agencies on federal training 

investments are in line with five of the eight leading practices, but OPM 

lacks guidance and assistance in some areas that are challenges to 

agencies, as shown in table 5.

Table 5: Comparison of Agency Implementation of Leading Practices and OPM 

Guidance and Assistance

Leading Practice

Addressed 

by OPM 

guidance or 

assistance

Not Addressed 

by OPM 

guidance or 

assistance

Practice 1: (a) Identify the appropriate level of 

investment to provide for training and development 

efforts and (b) prioritize funding so that the most 

important training needs are addressed firsta

X 

Practice 2: Identify the most appropriate mix of 

centralized and decentralized approaches for its 

training and development programs 

X 

Practice 3: Consider government-wide reforms and 

other targeted initiatives to improve management and 

performance when planning its training and 

development programs

X 

Practice 4: Have criteria for determining whether to 

design training and development programs in-house or 

obtain these services from a contractor or other 

external source 

X 

Practice 5: Compare the merits of different delivery 

mechanisms (such as classroom or computer-based 

training) and determine what mix of mechanisms to use 

to ensure efficient and cost-effective delivery 

X 

Practice 6: Track the cost and delivery of its training 

and development programs 

X 

Practice 7: Evaluate the benefits achieved through 

training and development programs, including 

improvements in individual and agency performance

(a) Has a formal process for evaluating employee 

satisfaction with training

(b) Has a formal process for evaluating improvement 

in employee performance after training

(c) Have a formal process for evaluating the impact 

of training on the agency’s performance goals and 

mission 

X 

Practice 8: Compare training investments, methods, or 

outcomes with those of other organizations to identify 

innovative approaches or lessons learned

X 

Source: GAO analysis of OPM guidance and assistance tools. 

OPM Guidance and 

Assistance to 

Agencies on Federal 

Training Investments 

Addresses Many 

Leading Practices that 

We Identified, But 

Could be Improved in 

Areas That Are 

Challenges to 

Agencies
 

Page 30 GAO-12-878 Federal Training Investments 

OPM guidance or assistance to agencies on federal training investments 

addresses five of the eight leading practices for federal agency training 

investment decision-making processes. OPM’s five primary guidance 

documents that relate to making training investment decisions include the 

Guide to Human Resources Reporting, Training Evaluation Field Guide, 

Draft Training Policy Handbook, Guide for Collection and Management of 

Training Information, and Guide to Strategically Planning Training and 

Measuring Results. (See table 2 for a brief description of these guides). In 

addition, OPM provides technical assistance to agencies via facilitated 

forums, discussion, and training.

Our review of OPM’s guidance documents and assistance shows that 

OPM has provided some technical assistance to agencies on this 

practice, although OPM does not have guidance documents that provide 

specific advice on this topic. For example, because of requirements in 

GPRAMA, OPM is providing assistance to agencies in considering this 

government-wide reform when planning their training and development 

programs. GPRAMA required OPM to identify the competencies needed 

to perform the following three functions: developing goals, evaluating 

programs, and analyzing and using performance information for the 

purpose of improving government efficiency and effectiveness. OPM, 

working with subject matter experts developed a competency model for 

the three new roles required by GPRAMA—performance improvement 

officer, performance improvement staff, and goal leader. Earlier this year 

OPM advised agencies that it would provide guidance on how to 

incorporate the skills and competencies into these position descriptions, 

as specified in the GPRAMA. The Director of OPM stated that the agency 

would work with the CLOs to incorporate the key skills and competencies 

into agency training programs.42

OPM has begun providing this assistance to agencies by facilitating 

sessions for agencies to develop training requirements for implementing 

the new positions and roles required by GPRAMA. For example, OPM 

worked with OMB to gather information on existing training, provide 

learning opportunities, and consolidate new and existing training courses 

and materials to support this effort. Using this information, OPM and OMB 

led two working group meetings with agencies to discuss GPRAMA 

 

42OPM’s Director made these statements in a January 3, 2012 memo to agency CHCOs 

entitled “Government Performance and Results Act Modernization Act of 2010 Functional 

Competencies”.

OPM Provides Guidance or 

Assistance that Addresses 

Some Leading Training 

Investment Practices

Consider Government-wide

Reforms and Other Targeted 

Initiatives to Improve 

Management and Performance 

When Planning its Training and 

Development Programs 
 

Page 31 GAO-12-878 Federal Training Investments 

training needs and next steps. In a working group meeting in February 

2012, OPM and agencies discussed which competencies identified in 

GPRAMA could be improved readily through training. OPM provided 

participants with a chart developed from an OPM and the Merit System 

Protection Board 2011 Trainability Study on which competencies for the 

three new roles required by GPRAMA were highly trainable versus those 

that were less trainable.43

• create a common competency assessment tool to assess competency 

gaps within agencies;

After the discussion, OPM and participants 

identified the most critical and manageable next steps, including:

• create a course on writing results-oriented goals and standards—

while also gathering existing training;

• create a working group to assess the needs and create a solution to 

satisfy the training requirement for the Organizational Performance 

Analysis, Planning and Evaluating, and Performance Measurement 

competencies and to collect relevant case studies, as well as to 

identify opportunities to leverage agency resources;

• identify existing subject matter experts in the agencies and create 

forums, workshops, training sessions, etc. where they can share their 

expertise and possibly engage in peer-to peer coaching;

• create working groups where necessary; and

• consider the development of a career path after OPM’s classification 

study.

In addition to this assistance, although not specific to government-wide 

reforms, OPM’s Training Policy Handbook advises agencies to conduct a 

training needs assessment that includes an evaluation of organization 

 

43According to OPM documents, “Highly trainable” competencies consist primarily of 

knowledge and can be improved readily through training. “Moderately trainable” 

competencies have both a skills and ability component. The skills can be improved thru 

training, but improvement in ability may be limited by the degree of the employees’ natural 

talent. These competencies are only partially responsive to training. “Less Trainable” 

competencies are identified as “trait” competencies that have personal characteristics that 

resist change. These competencies make a difference on the job, but are frequently not 

responsive to improvement through training.
 

Page 32 GAO-12-878 Federal Training Investments 

needs, which should take into consideration changing demographics, 

political trends, technology, and the economy.

OPM’s TED group advises agencies in its Training Policy Handbook and 

2000 Guide to Strategically Planning Training and Measuring Results to 

use multiple delivery methods, or combine them, when providing training 

to employees. For example, the Training Policy Handbook maintains that 

agencies should decide which delivery option is best to achieve the 

instructional goals of the training, highlighting that some methods are 

more effective for certain courses. It states that a performance 

management course may include role play scenarios which may not be 

suited for an e-learning course. Further, the guide states that agencies 

need to develop training delivery mechanisms that effectively limit 

unnecessary overlap and duplication of efforts. Similarly, we have 

previously reported that agencies need to consider essential issues such 

as the goals and objectives for the training, the type of audience for which 

the training is intended, the nature of the training content, the availability 

of technology and resources, and the timing for delivering the training 

when identifying the most effective and efficient delivery mechanism.44

 

44

Agency officials who have implemented this practice reported seeing 

positive results. For example, VHA officials that we met with and agencies 

that have publically discussed their efforts to assess the different delivery 

mechanisms at a March 2012 Partnership for Public Service Forum on: 

Going Virtual- Maximizing the Return On Investment of Online Training

reported significant savings and increases in the effectiveness of their 

training by assessing and changing their training delivery mechanisms. 

Specifically, VHA officials reported achieving several non-financial and 

financial benefits as a result of moving from in-person meetings and audio 

and video conferencing to providing training on-line for one of its 

leadership training programs. According to a VHA assessment report, the 

benefits included: consistent curriculum across eight medical centers in 

three states; easier accessibility to course materials and job aids; 

immediate access to feedback on courses from learners; easier 

reproduction of courses for instructors; a return on investment of 140 

percent since implementation; and $116,000 saved in travel costs, 

facilitation, and facilities; among other things.

GAO-04-546G. 

Compare the merits of different 

delivery mechanisms (such as 

classroom or computer-based 

training) and determine what 

mix of mechanisms to use to 

ensure efficient and costeffective delivery 
 

Page 33 GAO-12-878 Federal Training Investments 

OPM guidance informs agencies that they should implement this practice.

However, the guidance does not include methodologies for how to 

implement it. Officials from DHS—an agency that reported that it does not 

implement this practice—stated that tools provided by OPM could be 

strengthened to assist them in comparing training delivery mechanisms.

For example, DHS officials reported that they have difficulty implementing 

this practice partly because their components do not track comparative 

data on the different delivery mechanisms. According to the DHS officials, 

the standard government form for tracking training data (Standard Form182) does have a category for tracking training delivery type, but filling out 

this block is not mandatory and is often not used.45

OPM’s TED group provides guidance and assistance to agencies on 

tracking and reporting the cost and delivery of training and development 

programs in four of its five guides. For example, OPM’s 2000 Guide to 

Strategically Planning Training and Measuring Results advises agencies 

to calculate the cost of the expenses associated with designing, 

developing, implementing, and evaluating their training programs and 

provides a list of the most common types of training costs. OPM’s Guide 

for Collection and Management of Training Information also outlines 

agency requirements to track various types of data training and provides 

a list of several data sources (e.g. Standard Form-182, agency personnel 

records, procurement documents, financial and performance records, 

training evaluation forms, etc.) that agencies could use to collect this 

information. Similarly, the Training Policy Handbook also incorporates 

guidance on tracking the cost and delivery of agencies’ training and 

development. In the more recent 2012 Guide to Human Resources 

Reporting, OPM outlines requirements for agencies to track training data 

and describes the requirement to use certain standard tracking forms, 

The DHS officials 

reported that an OPM requirement to capture these data would improve 

their ability to gather the information needed from DHS components to 

effectively implement this practice. As noted earlier, 15 of the 27 CHCOs 

included in our review, reported that they do not implement this practice, 

which indicates that they may also benefit from additional guidance and 

tools on ways to do so.

 

45The Standard Form-182 is the government-wide training form used to request, approve, 

and certify completion of training courses, conferences, seminars, symposia and 

academic courses. The primary purpose of this form is to document and track employee 

training, especially training paid with agency funds. The Standard Form-182 also requires 

that each training record should include all of the mandatory data. 

Track the cost and delivery of 

its training and development 

programs
 

Page 34 GAO-12-878 Federal Training Investments 

such as the standard “Authorization, Agreement, and Certification of 

Training” (Standard Form-182) to track data. The guide also instructs 

agencies to provide all training information included in this form for 

submission to OPM’s Enterprise Human Resources Integration (EHRI) 

database systems.

Although OPM provides several guidance documents and assistance on 

tracking the cost and delivery of training, we found that this practice 

continues to be a challenge for many agencies to implement. Agency 

officials that we met with reported that they could benefit from additional 

assistance from OPM in developing a common definition of what should 

be tracked as training, developing policies to strengthen the utilization of 

Standard Form -182 to document and report all training costs, and 

encouraging agencies through guidance and technical assistance, to 

routinely report training cost data to agency learning management 

systems.

In addition, to providing guidance on tracking data, OPM facilitates the 

collection of federal training data government-wide. Executive Order No. 

11348 requires OPM to develop, install, and maintain a system to provide 

the training data needed to carry out its own functions and to provide staff 

assistance to the President. OPM’s EHRI is the government-wide 

repository for these training data.46

 

46OPM was also required by statute (5 U.S.C. § 4118) to prescribe regulations providing 

for the maintenance of necessary information concerning the conduct of training activities

of agencies to enable the President and Congress to discharge effectively their respective 

duties and responsibilities for supervision, control, and review of these training programs. 

OPM implementing regulations require agencies to maintain records of training activities 

and expenditures for submission to OPM. 5 C.F.R. § 410.601(a). As previously noted, 

OPM’s EHRI Program’s Data Warehouse is the government’s premier source for 

integrated Federal workforce information. The system is intended to collect, integrates, 

and publishes data for 2.0 million Executive Branch employees on a bi-weekly basis, 

supporting agency and government-wide analytics in various areas, including agency 

training data.

As noted above, agencies have been 

required since 2006 to report training data to OPM monthly via this 

system. However, according to OPM officials, they consider the data to 

be unreliable because they are incomplete. Therefore, OPM officials have

not used it to inform their training guidance and assistance to agencies, to 

counsel heads of agencies and other agency officials regarding federal 

training needs or investments or to assist agencies in developing sound 

programs and financial plans for training programs.
 

Page 35 GAO-12-878 Federal Training Investments 

According to OPM officials and documents, OPM should assess EHRI 

training data for technical compliance and data quality validation. 

Technical compliance is the testing and approving of agency systems for 

data quality (i.e. correct formatting, adherence to edit rules). Once 

systems are technically compliant, agencies are required to send monthly 

data feeds of completed training events to OPM.47

We believe that the current reliability of agency training investment data is 

unknown because OPM officials have not internally assessed 

improvements in the completeness of the data over the last 3 years or the 

quality of the data in the six years that agencies have been required to 

submit it. The two internal reviews that OPM conducted of training data 

were in 2008 and September 2009. In the 2009 review, OPM reported 

that there was an increase from fiscal year 2008 in the amount of training 

data being reported by agencies, but that the quality of the data was still 

less than what was necessary to provide an accurate picture of federal 

training investments. According to the 2009 report, over half of all 

agencies were reporting data for the entire agency, 86 percent were 

reporting on a regular basis, but only 7 percent were reporting cost data.

The report identified several of the same reasons that we previously 

described as limitations to agencies reporting training investment data.

Although the report stated that OPM would continue to work with 

agencies to assess the quality/validity of training investment data and 

Once agencies are 

reporting these data for all major components, all employees, all types of 

training (e.g. conferences, on-line, classroom), and training cost data, 

OPM are to evaluate the data quality to determine if it presents an 

accurate picture of all training in the agency. However, OPM officials told 

us that they have not assessed the quality of data or developed a report 

on its reliability because no agency is sending information on all training 

events. According to OPM officials, when agencies request assistance or 

when OPM finds that an agency has been grossly delinquent in providing 

data, OPM officials will inquire further and offer assistance to the 

agencies. However, they typically do not document reliability issues or the 

agreed upon action plans to address the problems. The officials agreed 

that this is a problem, but stated that they would need more staff 

resources to provide this level of assistance and oversight.

 

47OPM has an EHRI technical office that collects the data from agencies and informs TED 

officials about which agencies are providing data each month and whether there are 

apparent problems with the data (e.g. missing or miscoded data). OPM requires agencies 

to provide a single source of data that represents training throughout their organization.
 

Page 36 GAO-12-878 Federal Training Investments 

determine whether agencies are reporting all training events, as noted 

above, OPM officials informed us that they have not assessed the quality 

of the data because the data are not 100 percent complete.

While it is important to have complete data, we do not believe that having 

incomplete data necessarily prevents OPM from assessing the overall 

reliability of the data, if it meets standards for sufficiency. In our guidance 

on assessing the reliability of computer based data, we have stated that 

agencies can assess data if it is sufficiently complete.48

OPM also previously identified several steps that its officials would take to 

assist agencies in improving their data, but have not yet implemented all 

of them. According to OPM’s 2009 report assessing EHRI data, OPM 

planned to assist agencies in improving training investment data, by: (1) 

working with agencies to fully report all training investment data—

including costs; (2) working with agencies to decrease errors in reporting; 

and (3) providing individual agencies with summary reports of the data 

that they submitted to OPM for their review and verification.

Data are 

sufficiently reliable when testing and reviews of the existing information 

provide assurance that (1) the likelihood of significant errors or 

incompleteness is minimal and (2) the use of the data would not lead to 

an incorrect or unintentional message. Further, we consider the data not 

to be sufficiently reliable when there are significant errors or 

incompleteness in some of or all the key data elements and if using the 

data would probably lead to an incorrect or unintentional message. 

Because OPM has not conducted an assessment of improvements in 

agency training data in three years, it is unknown whether it is currently 

complete enough to test other aspects of its quality and reliability. 

According to the officials, although they have not conducted a formal 

review of the data, they are able to visually look at the EHRI data base 

and tell that the data are significantly more complete than in past years.

49

 

48Data validity refers to whether the data actually represent what we think is being 

measured. Data reliability refers to the accuracy and completeness of computerprocessed data, given the uses they are intended for. Computer-processed data may be 

data (1) entered into a computer system or (2) resulting from computer processing. GAO, 

Assessing the Reliability of Computer-Processed Data, 

We found 

GAO-09-365G (Washington, D.C.: 

February 2009). 

49According to the 2009 analysis, the agency reports were intended to include data on the 

percent of managers receiving training, average cost spent per employee training, and 

percent of salary spent on training per agency, among other things. 
 

Page 37 GAO-12-878 Federal Training Investments 

that OPM has initiated some related efforts, but has not fully addressed 

two of these issues. In order to decrease errors in reporting, OPM officials 

and EHRI reports show that OPM has worked with agencies to identify 

technical errors in their training data submission. However, to improve 

reporting on cost data—which is currently a challenge for agencies—

OPM held one focus group with agencies in 2007, which it used to 

updated it’s guidance on tracking training data in 2008.50 OPM also has 

not followed through on plans to annually provide agencies with reports of 

their training data for verification and correction. According to OPM, the 

purpose of the training data report is to (1) inform agencies of the training 

data OPM had received (2) offer them the opportunity to work closely with 

OPM in correcting any identified deficiencies and (3) to make note of the 

progress they have made in addressing OPM’s training reporting 

requirement. OPM officials said that they sent one report to agencies (in 

fiscal year 2010) summarizing their training data and requesting 

verification and this report was provided in response to expectations that 

the data would be posted on the government-wide website Data.gov.

51

Although OPM has only provided one summary of EHRI data to agencies, 

agency officials that we met with stated that they could benefit from using 

this type of data summary to improve their training data. Further, OPM 

officials stated that using these summaries to improve EHRI data could 

help agencies measure the return of investment on their training and 

assist agencies’ stakeholders in making more informed decisions on the 

best use of training dollars. During our review, OPM officials reported that 

they began developing a report to send to agencies with fiscal year 2011 

training data.

In 

our review of examples of agency responses, we found that agencies 

identified important discrepancies in their data, including significant 

underestimates of the costs spent on training, and reported that they 

would take steps to address incorrect data. However, OPM officials 

informed us that they do not have a process for documenting whether 

agencies have taken steps to correct their data.

 

50OPM officials reported that they used this focus group to revise OPM’s 2008 Guide for 

Collection and Management of Training Information.

51 Accessed August 7, 2012.http:www.data.gov/. 
 

Page 38 GAO-12-878 Federal Training Investments 

The TED group provides guidance and assistance to agencies in 

evaluating training programs through three of its guides and in 

workshops. OPM’s Training Evaluation Field Guide is the primary guide 

through which OPM advises agencies on how to evaluate training. The 

guide instructs agencies to define the results they want to achieve from 

training and to align training with the agency mission and goals. Further, 

the Guide discusses useful models for evaluating training and describes 

Return on Expectations52 as the ultimate indicator to demonstrate the 

training value to stakeholders.53 The guide also provides information on 

evaluation requirements outlined in laws and regulations, and provides 

practical instruction by identifying common challenges and solutions 

related to identifying the most cost-effective methods to train and develop 

employees. In addition to this guidance, OPM’s Training Policy Handbook 

instructs agencies to evaluate all training to determine whether or not it 

provides meaningful contributions to agency results. Similarly, in its Guide 

for Collection and Management of Training Information, OPM highlights 

the importance of collecting accurate, comprehensive training information 

and making it available to decision-makers and others who have a vested 

interest in the training activities of the federal government. This guide 

discusses the two basic types of performance measures for measuring 

training and development program effectiveness: process indicators and 

outcome indicators.54

In addition to its guides, OPM has made evaluation tools available to 

agencies on its website and held workshops on training evaluation in 

order help agencies identify and share best practices on evaluating 

 

52OPM’s 2011Training Evaluation Field Guide: Demonstrating the Value of Training at 

Every Level defines return on expectations as the ultimate indicator of value. It is the 

conversion of stakeholders’ expectations into observable, measureable outcomes of 

success for the training. 

53The guide discussed the Kirkpatrick Four-Level model (reaction, learning, performance 

and impact of training), the updated New World Kirkpatrick Four Levels and the Kirkpatrick 

Business Partnership Model as useful models for evaluating training.

54Process indicators measure performance or key processes. They track such things as 

number of employees trained, number of training courses completed, or number of hours 

employees spent in training. Outcome indicators contain measures of service or 

performance results. These measure such things as an increase in employee and/or 

customer satisfaction, increase in employee job performance, and/or increase in 

organizational performance. Outcome indicators may require prolonged periods of 

measurement and multiple types and levels of measurements to include surveys, training 

impact measures, and return on investment analysis. 

Evaluate the benefits achieved 

through training and 

development programs, 

including improvements in 

individual and agency 

performance
 

Page 39 GAO-12-878 Federal Training Investments 

training. For example, OPM’s website contains a Training Evaluation Tool 

that describes the levels of training evaluation and provides agencies with 

evaluation questions to be answered in each of the four levels and the 

types of information typically collected.

As previously mentioned, some agency officials reported that it is difficult 

to conduct these reviews because their cost and time demands can be 

significant. As a result, some agencies only conduct them for the most 

critical training and others reported that they do not have a formal process 

for conducting these reviews at all. While we agree that it is appropriate to 

target costly evaluations to the most important training, those who do not 

implement this practice at all could benefit from using OPM’s 

comprehensive guidance and assistance on training evaluations.

OPM does not have a guidance document that advises agencies on how 

to compare training investment methods and outcomes with other 

agencies, but provides some support to agencies in this area through 

technical assistance. For example, the TED group uses various webbased mechanisms such as OPM’s website, OPM LISTSERV, OPM 

Federal Training and Development web site and OPM Federal Training 

and Development Wiki to facilitate discussions between agencies on 

training investments. We observed the exchange and sharing of 

information among agencies through OPM’s LISTSERV, which is used by 

950 employees from various federal agencies to share training practices 

and advice. At times, agencies requested and shared information with 

each other on the most effective or efficient ways to implement specific 

training programs or requested models for which to compare their 

activities. Similarly, OPM’s wiki page contains examples and models of 

training programs for others to use when developing their training 

programs. According to OPM officials, the TED group also provides best 

practice forums on topics when they believe agencies need additional 

assistance. For example, OPM officials reported that they have held 

forums with agencies on the Training Evaluation Field Guide to share 

best practices and tools among agencies. OPM is working on a similar 

forum for developing supervisory training.

OPM does not have guidance and assistance for three leading training 

investment practices, two of which are areas in which agencies reported 

experiencing challenges. We examined the five guidance documents that 

OPM provides related to making training investment decisions and 

documentation on OPM’s technical assistance and did not find support for 

Compare training investments, 

methods, or outcomes with 

those of other organizations to 

identify innovative approaches 

or lessons learned

OPM Does Not Provide 

Guidance and Assistance 

on Three Leading Training 

Investment Practices
 

Page 40 GAO-12-878 Federal Training Investments 

the following three practices. OPM officials confirmed that that the agency 

does not provide direct guidance or assistance in some of these areas. 

In our review of OPM guidance and documentation on its technical 

assistance, we found that OPM provides some guidance to agencies 

regarding steps to identify training and development investment needs 

and related training strategies, but we did not find guidance on prioritizing 

these investments so that the most important training needs are 

addressed first. According to TED officials, each agency must assess its 

own needs as the primary driver for investment determinations. To that 

end, OPM officials provide guidance and tools for conducting training 

needs assessments in OPM’s training policy guide and on OPM’s website 

and also direct agencies to review benchmarks in American Society for 

Training & Development’s State of the Industry reports. OPM officials also 

reported that they use the Human Capital Assessment Accountability 

Framework and related efforts to emphasize the importance of 

considering training as a solution to addressing mission critical 

competencies and skill gaps, but acknowledged that OPM does not 

provide specific guidance on prioritizing training investments through 

these processes.55

OPM officials identified guidance that they believe addresses the leading 

practice of prioritizing training investments; however, we found that the 

guidance does not address prioritization. As we previously mentioned, it 

is a leading practice for agencies to prioritize their training investments 

using criteria, such as expected demand for the training from internal 

sources, availability of resources to support the effort, potential for 

increased revenue, and risk of unfavorable consequences if investments 

are not made, or their potential to improve their efficiency. TED officials 

identified OPM’s 2000 Guide to Strategically Planning Training and 

Measuring Results as the source of guidance to agencies on this 

 

55Officials from OPM’s Agency Program Review office oversee OPM’s Human Capital 

Assessment Accountability Framework program. According to OPM documents, the 

Human Capital Assessment Accountability Framework consists of five human capital 

systems that together provide a consistent, comprehensive representation of human 

capital management for the federal government: strategic alignment, leadership and 

knowledge management, results-oriented performance culture, talent management, and 

accountability. 

Identify the appropriate level of 

investment to provide for 

training and development 

efforts and prioritize funding so 

that the most important 

training needs are addressed 

first
 

Page 41 GAO-12-878 Federal Training Investments 

practice.56

In our review of OPM guidance and documentation on its technical 

assistance, we did not identify specific guidance or assistance to 

agencies on this practice. As we previously noted, while neither approach 

fits every situation, agencies need to consciously think about the 

advantages and disadvantages of using centralized and decentralized 

approaches, particularly for the design of training and development 

programs. Although OPM officials confirmed that they do not provide 

guidance and assistance to agencies in this area, OPM officials agreed 

with this leading practice. We found that most agencies included in our 

review reported that they already implement the practice, so additional 

guidance may not be necessary.

In our review of this guide, we did not find any explicit 

guidance to agencies on prioritizing training investments. Instead, the 

guide advises agencies to build a business case for their training 

strategies. OPM defines a business case as a method for projecting and 

documenting the benefits to be gained as a result of investing resources 

in a training intervention. The guide encourages agencies to consider 

questions that are important to building a business case and provides an 

example of how to build a business case using this information (see 

Appendix III for diagrams from the guide on building a business case for 

training). These steps are consistent with our identified leading practice. 

However, the guide does not take the additional step of advising agencies 

on how to prioritize training investments selected from their business 

case(s) relative to each other. We have previously reported that, when 

budgets are constrained, training is often one of first investments that 

agencies reduce. Therefore, it is increasingly important for agencies to 

prioritize their selected training activities, so that the most important 

training is identified. Moreover, they need to communicate those priorities 

agency-wide, in order to identify common needs and potential areas for 

consolidated investments. As previously noted in our review, this is a 

practice that most CHCOs reported that they do not implement, which, as 

illustrated by our case example agencies, has resulted in costly 

duplicative and inefficient training investments at some agencies.

 

56The guide recommends that agencies identify their agency-wide competency gaps, 

assess whether training will address the gaps, identify and weigh the cost and benefits of

training, and then build a business case to support their selected training strategy. 

Identify the most appropriate 

mix of centralized and 

decentralized approaches for 

its training and development 

programs.
 

Page 42 GAO-12-878 Federal Training Investments 

TED officials agreed that they do not provide guidance or assistance to 

agencies on this practice. TED officials stated that agencies need to 

incorporate this leading practice into their training investment decisionmaking process. As we previously noted, once an agency has identified 

its training and development needs, it should make informed decisions 

about whether to design and develop training and development programs 

in-house or buy these services from a contractor or other external source. 

Factors that they should consider include the capability of in-house staff 

to develop and implement the training; the prior experience, capability, 

and stability of possible providers in the marketplace; and agency 

limitations on cost, time, and resources. As previously mentioned, 12 of 

the 27 CHCOs reported that they do not implement this leading practice, 

and our discussion with DOI officials, one of the four agencies that we 

interviewed for this review, illustrates that even those agencies that 

reported implementing this practice may not be doing so for all training in 

the agency.

Agency officials reported they could use more OPM assistance in 

leveraging federal training investments across the government. Part of 

OPM’s role is to identify functional areas in which new or expanded 

interagency training activity is needed and either conduct such training or 

arrange for agencies having the substantive competence to do so; as well 

as to coordinate interagency training conducted by and for agencies. 

Members of the CLO council emphasized that they could benefit from 

more OPM assistance in achieving greater interagency collaboration on 

training to reduce duplicative training investments. All four agencies that 

we interviewed reported concerns similar to the Council’s. For example, 

DOI officials noted that OPM’s knowledge and expertise could help 

agencies identify one basic approach to competency management (e.g., 

establishing levels of proficiency and competency validation processes) 

that can be used across government rather than using multiple 

approaches at various agencies. At the present, each agency individually 

identifies training needed for these competencies, which results in 

duplication and variation in the quality of training provided throughout the 

government. DOI officials stated it could be more efficient if agencies 

would use a standard set of Knowledge, Skills, and Abilities to hire and 

identify training and development investment priorities. The officials also 

suggested that OPM’s HR University could be used to provide training for 

other mission-critical occupations. Further, VA officials stated that it would 

Have criteria for determining 

whether to design training and 

development programs inhouse or obtain these services 

from a contractor or other 

external source. 

Agency Officials Identified 

a Need for OPM to Provide 

More Assistance in 

Improving the 

Effectiveness and 

Efficiency of Training and 

Development Investments 

Government-wide
 

Page 43 GAO-12-878 Federal Training Investments 

assist agencies if OPM established government-wide courses for 

mandatory training and cross-cutting areas.57 As an example, the officials 

stated the federal government has 17 different versions of No Fear Act 

training.58

In contrast, the shared training efforts that are being implemented by the 

Federal Healthcare Training Partnership collaborative, which consists of 

14 federal agencies that provide clinical health care or related training to 

support their mission, illustrate the potential magnitude of savings that 

could be achieved by leveraging training across agencies. The Federal 

Healthcare Training Partnership was created by its members to share 

training programs and resources across the agencies to speed up the 

provision of employee learning and reduce training costs. According to 

VA officials—who lead the effort, the agencies formed this group because 

they saw the unaddressed need and opportunity to save costs in common 

training areas. Documentation provided by VA on the collaborative group 

states that in fiscal 2011, Federal Healthcare Training Partnership partner 

organizations shared more than 2,300 programs, generating a total cost 

avoidance of more than $82 million.

The officials suggested that OPM could establish one 

government wide training for such subjects which would help agencies 

save federal time and money. Officials from DHS and DOE expressed 

similar views.

59

 

57We are conducting a review of agency actions to address critical skill gaps, which 

include training and also have work underway assessing training in cross-cutting 

occupations, such as grant management and acquisition.

They did so by utilizing the partner 

organizations’ existing learning systems to share training that was 

originally developed for a single agency’s internal use and making it 

available to all federal learners, as well as by coordinating the joint 

development or purchase of training needed by two or more partner 

agencies. VA officials stated that, while this has been a valuable effort to 

specifically improve healthcare-related training investments for the 

agencies involved, all federal agencies would benefit from an expansion 

of leveraging training investments across the government.

58Under the Notification and Federal Employee Antidiscrimination and Retaliation Act of 

2002 (No Fear Act) and OPM’s implementing regulations, an agency must provide training 

to its employees regarding the rights and remedies available under the employment 

discrimination and whistleblower protection laws at least every two years. See, section 

202(c) of Pub. L. No. 107-174, 116 Stat. 566, 569 (May 15, 2002) and 5 C.F.R. § 724.203.

59We did not verify the costs savings identified in the FHTP’s report.
 

Page 44 GAO-12-878 Federal Training Investments 

OPM officials agreed that increased coordination of mandatory and 

common training across the government could reduce duplication and 

improve the efficiency of federal training investments. The officials 

reported that OPM has already engaged in some efforts to partner with or 

support CLO and CHCO Council efforts to share specific training across 

agencies. For example, the officials worked with the Social Security 

Administration to share a “Plain-Language” writing course developed by 

the Social Security Administration with other agencies, by placing it on 

OPM’s Training and Development Wiki page. In addition, OPM officials 

stated that in 2010, the CHCO Council and OPM collaborated to establish 

HR University, which is aimed at addressing the competency and skill 

gaps within the HR community and achieve savings government-wide by 

identifying and sharing the best HR training with all agencies.60 While the 

system was initially designed to provide training to the HR community, it 

has also been used to provide some mandatory training and HR training 

to other supervisors and managers. For example, officials reported that 

they recently added a mandated Uniformed Services Employment and 

Reemployment Rights Act training to HR University. The CHCO Council 

and OPM have also developed a formula to calculate cost savings 

resulting from the shared courses, which agencies can use to track their 

savings and return on investment. According to the OPM Executive 

Director of the CHCO Council and HR University’s website, in its first 

year, HR University has saved the government $14.5 million as a result of 

the shared training, and OPM officials expect that it could produce 

significantly more savings, when other courses are added.61 According to 

OPM officials, while HR University primarily serves the needs of the HR 

community, OPM would support using the HR University model to 

centralize training in other occupations or functional areas.

 

60HR University was initially funded by the CHCO Council and operated in coordination 

with OPM. However, OPM officials reported that they are currently working on a MOU with 

the CHCO Council to transfer more ownership and operation of HR University to OPM, 

with the CHCO Council as an advisor.

61According to the Executive Director of the CHCO Council, they developed this formula in 

consultation with the CLO Council. We did not independently assess the formula or verify

the reported savings.
 

Page 45 GAO-12-878 Federal Training Investments 

The federal government’s efforts to build and retain a workforce of skilled 

and efficient employees are essential to addressing skill gaps in critical 

fields and effectively and efficiently deliver services to the public. Training 

and development programs play a vital role in fulfilling these goals. 

However, agency leaders need to be as strategic about how they invest 

resources in this area as they are in other key areas of agency 

operations. Training investment decisions should be based on an 

assessment of the appropriate level of training investments and the 

prioritization of those investments, as well as an evaluation of the most 

cost-effective delivery mechanisms, and the known costs and benefits of 

their training investments. CHCOs and OPM each play a vital role in 

ensuring that these investment decisions are effectively made.

While CHCOs report that they are implementing leading practices that 

support the successful delivery of training, they could do more to ensure 

that these investments are more cost effective. Because many CHCOs do 

not have the information that they need from component or subagency 

leaders regarding the level of training investments and mechanisms for 

setting priorities agency-wide, their agencies are duplicating some 

internal training investments and missing opportunities to leverage 

economies of scale and share the most effective training across their 

agencies. Many CHCOs are also limiting their opportunities to make 

training more cost effective and accessible because they are not 

comparing the merits of different training delivery mechanisms. OPM’s 

guidance and assistance in these three areas are minimal or absent and 

could be strengthened to assist agencies in implementing these leading 

practices. In addition to these limitations, some CHCOs do not have a 

formal process to evaluate the impact of training on their mission. While 

not all training and development programs require, or are suitable for, 

higher levels of evaluation, those who do not implement this practice for 

any training are missing information that could help them make more 

effective investment decisions and could benefit from using OPM’s 

existing guidance and assistance on conducting such evaluations.

Federal agencies and OPM also need reliable information on how much 

agencies spend on training and for what purposes, in order to make 

effective training investment decisions. However, CHCOs do not 

completely and reliably track training costs agency-wide and, therefore, 

are unable to provide OPM with the reliable information that it needs to 

more effectively guide government-wide training policies. OPM has 

responsibility for providing regulations for maintenance of agency training 

data, assessing the completeness and quality of those data when 

agencies submit it, and using it to target its assistance to agencies. But 

Conclusions
 

Page 46 GAO-12-878 Federal Training Investments 

OPM does not know the extent of the reliability of federal training 

investment data because they have not compared improvements in the 

completeness of the data over last 3 years and determined if it meets the 

standards of sufficiency for data assessment and have not assessed the 

quality of the data in the 6 years that agencies have been required to 

submit it.

Given the fiscal challenges facing the nation, the federal government 

needs to take advantage of every opportunity to better leverage 

resources and investments across agencies. However, at present many 

agencies independently purchase or develop training for the same 

government-wide mandated courses. OPM has an opportunity to reduce 

duplicative and inefficient training investments by leveraging existing 

training resources government-wide. Agency leaders and OPM recognize 

that this has led to redundant and inefficient federal training investments. 

HR University—the one-stop-shop training platform administered by OPM 

for many courses mostly related to the HR community—provides a model 

that can result in cost savings and help standardize some mandatory 

training courses across government.

To improve federal training investment decision-making processes, the 

Director of OPM should take the following five actions:

1. Include in existing or new OPM guidance or technical assistance 

additional information in the following areas:

• Steps agencies should take and factors they should consider 

when prioritizing federal training investments agency-wide, 

including developing a process to rank training using criteria, such 

as expected demand for the investment from internal sources, 

availability of resources to support the effort, potential for 

increased revenue, and risk of unfavorable consequences if 

investments are not made.

• Steps agencies should take and factors they should consider for 

comparing the merits of different delivery mechanisms and 

determining the mix of mechanisms to use, in order to ensure 

efficient and cost-effective delivery of federal training. Such 

guidance could include requesting that agencies consistently 

utilize Standard Form-182 to document and report training costs 

associated with the different delivery mechanisms employed.

Recommendations for 

Executive Action
 

Page 47 GAO-12-878 Federal Training Investments 

2. In line with statutory and regulatory provisions on maintenance and 

reporting of training information, work with the CHCO Council to 

improve the reliability of agency training investment information by:

• ensuring that agencies are familiar with and follow guidance 

outlined in OPM’s Guide for the Collection and Management of 

Training Information regarding which training events should be 

documented as training and reported to OPM;

• developing policies to strengthen the utilization of Standard Form182 to document and report training costs;

• encouraging agencies through guidance and technical assistance, 

to develop policies that require consistent reporting of training 

data to their learning management systems; and

• encouraging each agency to assess its existing training 

information system(s) and identify whether it is providing complete 

and reliable data and, if not, to develop approaches to improve the 

system(s), in order to do so.

3. Provide regular report summaries to agencies on EHRI training 

investment data and its reliability, in order to improve the transparency 

and reliability of federal training investment data.

4. Once federal training data reliability has been sufficiently improved, 

consistent with Executive Order No. 11348, use EHRI data to:

a) counsel heads of agencies and other agency officials on the 

improvement of training, and

b) assist agencies in developing sound programs and financial plans 

for training and provide advice, information, and assistance to 

agencies on planning and budgeting training programs.

In collaboration with the CHCO and CLO Councils, identify the best 

existing courses that fulfill government-wide training requirements, such 

as mandatory Equal Employment Opportunity training, or training in 

common federal occupations, such as basic training in financial 

management, and offer them to all agencies through HR University or 

other appropriate platform to reduce costly and duplicative federal training 

investments.
 

Page 48 GAO-12-878 Federal Training Investments 

We provided a draft of this report to the departments of OPM, DHS, DOE, 

DOI, and VA for review and comment. OPM commented on our five 

recommendations to their agency, concurring with one recommendation, 

partially concurring on three recommendations and not concurring with a 

portion of one recommendation. OPM’s official comments are reprinted in 

appendix IV. OPM, DOI, and VA provided technical comments, which we 

incorporated into our report, as appropriate. DOE and DHS had no 

comments.

OPM partially concurred with our first recommendation that it should 

provide in existing or new guidance information on prioritizing federal 

training investments agency-wide and factors agencies should consider 

for comparing the merits of different delivery mechanisms. OPM stated 

that its publications mentioned in our report already provide guidance on 

necessary steps and specific factors agencies should consider when 

prioritizing training investments. However, none of the guides that we 

obtained or that OPM provided for our review contain a specific 

discussion about ranking training investments based on key factors that 

should be considered, such as expected demand for the investment from 

internal sources, availability of resources to support the effort, potential 

for increased revenue, risk of unfavorable consequences if investments 

are not made or the potential to improve efficiency. OPM stated that, as 

part of its effort to revise the Human Capital Assessment Framework 

resources that it provides to agencies, OPM plans to include tools and 

guidance on steps agencies can take to prioritize learning investments as 

part of its strategic human capital planning. We did not change our 

recommendation, which is based on OPM’s current guidance and 

assistance. OPM’s reported future plan to provide more specific guidance 

on prioritization has the potential to address our recommendation, when 

implemented. OPM also agreed to provide further guidance regarding 

what steps agencies should take and what factors they should consider in 

comparing the merits of different delivery mechanisms and determining 

the mix of mechanisms to use to ensure efficient and cost-effective 

delivery of federal training.

OPM did not concur with the portion of our second recommendation 

regarding working with the CHCO council to improve the reliability of 

agency training investments by developing a common definition of what 

should be documented as training. OPM stated that the definition of 

training is clearly stated in 5 U.S.C Chapter 41 and OPM’s Draft Training 

Policy Handbook and Guide for the Collection and Management of 

Training Information outlines which training events should be documented 

as training and reported to OPM. Consequently, OPM recommended that 

Agency Comments 

and Our Evaluation
 

Page 49 GAO-12-878 Federal Training Investments 

we delete this task for OPM. OPM’s Guide for the Collection and 

Management of Training Information states that all courses, workshops 

and conferences paid for by the government; all federally mandated 

training; and, all agency required training should be reported to OPM’s 

EHRI system. It further states that agencies do not have to report training 

that occurs spontaneously or casually/incidentally (e.g., reading a book, 

having a discussion, web casts, briefings, etc.); training that has no 

specified training goals; training where there are no ways to evaluate if 

the training improved knowledge, skills, abilities or competencies; and, 

training that was not paid for by the government. We agree that this 

guidance should assist agencies in knowing which training to track and 

report, and therefore have removed this task from the recommendation. 

However, given the concerns raised by officials in our case example 

agencies regarding inconsistencies in whether conferences and other 

trainings are actually tracked, and recent events regarding spending at 

such training, we modified our recommendation to suggest that OPM 

work closely with CHCOs to ensure that this guidance is followed as it 

addresses the other actions we recommend to improve reliable reporting. 

OPM concurred with the other actions identified in the recommendation 

which included working with the CHCO council to: develop policies to 

strengthen the utilization of Standard Form-182 to document and report 

training costs; encourage agencies through guidance and technical 

assistance, to develop policies that require consistent reporting of training 

data to their learning management systems; and encourage each agency 

to assess its existing training information system(s) and identify whether it 

is providing complete and reliable data and, if not, to develop approaches 

to improve the system(s), in order to do so.

OPM partially concurred with our third recommendation that it should 

provide regular report summaries to agencies on EHRI training 

investment data and its reliability, in order to improve the transparency 

and reliability of federal training investment data. OPM stated that it will 

provide regular summaries to agencies on the training investment data 

submitted to OPM to improve transparency. However, OPM stated that 

these summaries will not directly lead to improved reliability of the data 

because agencies must take action to improve the data in order to have 

an effect on data reliability. OPM also noted that agencies currently have 

the option of working with OPM to secure a subscription to Business 

Objects—a reporting tool that will allow agencies to run reports of the 

data they have provided to OPM and determine whether those data 

accurately reflect what is occurring in their agencies. OPM recommended 

that we revise our recommendation to read, “Provide regular report 

summaries to agencies on EHRI training investment data in order to 
 

Page 50 GAO-12-878 Federal Training Investments 

improve the transparency of federal training investment data.” We agree 

that agencies are ultimately responsible for making changes to their data 

to improve its reliability. However, OPM plays an important role in the first 

step of that process by reporting the current information that it has, so 

that agencies can make corrections. We believe that this 

recommendation along with our prior recommendation on steps OPM and 

CHCOs can take to improve reliability will contribute to improving the 

transparency and reliability of agency training data. Therefore, we did not 

make changes to this recommendation.

OPM concurred with our fourth recommendation that it should counsel 

heads of agencies and other agency officials on the improvement of 

training; assist agencies in developing sound programs and financial 

plans for training; and provide advice, information, and assistance to 

agencies on planning and budgeting training programs using EHRI data, 

once federal training data reliability has been sufficiently improved. OPM 

stated that it will consult with agencies on possible improvements and 

assistance on planning training programs once federal training data are 

reliable.

OPM partially concurred with our fifth recommendation that it should, in 

collaboration with the CHCO and CLO Councils, identify the best existing 

courses that fulfill government-wide training requirements, such as 

mandatory Equal Employment Opportunity training, or training in common 

federal occupations, such as basic training in financial management, and 

offer them to all agencies through HR University or another appropriate 

platform to reduce costly and duplicative federal training investments. 

OPM stated that it agrees and is already collaborating with the CHCO and 

CLO Councils to identify, collect, and share existing mandatory courses 

that fulfill government-wide training requirements (e.g., Plain Writing, 

Telework. USERRA, Veterans Employment, Constitution Day) through 

HR University or on OPM’s Federal Training and Development Wiki. 

Therefore, OPM recommended that we revise the recommendation to 

recognize that the expansion of mandatory training by HR University 

would be a continuation of efforts they have started. We have revised the 

recommendation to reflect this comment.
 

Page 51 GAO-12-878 Federal Training Investments 

We are sending copies of this report to the Director of OPM. In addition, 

this report will be available at no charge on the GAO website at 

www.gao.gov. If you have any questions about this report, please contact 

me at 202-512-2717 or jonesy@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page 

of this report. GAO staff who made key contributions to this report are 

listed in appendix V.

Sincerely yours,

Yvonne D. Jones

Director

Strategic Issues
Appendix I: Objectives, Scope, and 

Methodology

Page 52 GAO-12-878 Federal Training Investments 

To better understand how federal training investment decisions are made 

and whether improvements are needed, you asked us to review the 

methods that agencies are using to establish their training investment 

strategies and Office of Personnel Management’s (OPM) training 

investment guidance to agencies. Accordingly, this review assesses the 

extent to which (1) chief human capital officers (CHCOs) of selected 

federal agencies have established processes to set and prioritize training 

investments that are aligned with leading practices; and (2) OPM’s 

guidance and assistance for developing training investment strategies 

align with these leading practices.

For the purposes of this review, we define the key terms “training”, 

“development” and “agency-wide” in the following ways:

• Training is making available to employees planned and coordinated 

educational programs of instruction in professional, technical, or other 

fields that are or will be related to the employee’s job responsibilities. 

Training can be accomplished through a variety of approaches, such 

as classroom training, e-learning, and professional conferences that 

are educational or instructional in nature.1

• Development is generally considered to include training, structured 

on-the-job learning experiences, and education. Developmental 

programs can include experiences such as coaching, mentoring, or 

rotational assignments.

• Agency-wide includes all components, sub-agencies or offices within 

a cabinet department or independent agency.

 

1

This definition of training includes mandatory training that governs how employees 

conduct themselves when carrying out their responsibilities, such as mandatory equal 

employment opportunity training or information technology security training.

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology

Page 53 GAO-12-878 Federal Training Investments 

For both objectives of the review, we compared OPM and CHCO practices 

against eight federal training investment leading practices, which are based 

on our prior studies, other expert studies, and statutory, regulatory, and 

executive order training requirements.2 (See table 1 at the beginning of this 

report). OPM reviewed these criteria and agreed that they are practices 

that agencies should be implementing to support effective training 

investment decisions. They also informed us that, while some leading 

practices are related to training program requirements contained in 

statutory, regulatory, or executive order provisions, responses to our 

questions about the leading practices are not an indication of whether 

agencies are in compliance with these laws and regulations.3

To obtain government-wide information on agency training investment 

practices, through a questionnaire on their training investment practices 

and processes, we obtained high-level information from members of the 

27 agencies represented on the CHCO Council.4

 

2

All of these practices are described in 

We provided a standard 

set of questions to each CHCO to ensure we consistently captured their 

responses to our questions on their training investment practices. We 

then analyzed the results of the questionnaire to identify the main themes 

and develop summary findings. Two of our analysts conducted this 

analysis, placed CHCO responses into categories, and tallied the number 

of responses in each category. A third analyst traced the responses back 

to the original questionnaire and verified the appropriate categorization of 

CHCOs’ responses. To characterize CHCOs views throughout this report, 

we defined modifiers (e.g., “many”) to quantify users’ views as follows:

GAO-04-546G. We also used several studies from 

the Corporate Leadership Council. 

3

Appendix II includes a table that lists the laws, regulations, and executive order 

provisions that are related to the leading practices. 

4

The CHCO Council was established with 25 members. However, the Director of OPM 

may designate other members of the Council. Such additional members may include, but 

are not limited to: (1) the CHCOs of other executive agencies and (2) members who are 

designated on an ex officio basis and who may be invited to contribute to projects, as 

particular skills and expertise are needed. The council agencies include the Departments 

of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, 

Homeland Security, Housing and Urban Development , Interior, Justice, Labor, State, 

Treasury, Transportation, and Veterans Affairs; Environmental Protection Agency: 

General Services Administration: National Aeronautics and Space Administration: National 

Science Foundation: Nuclear Regulatory Commission: Office of Management and Budget: 

OPM: Office of the Director of National Intelligence: Small Agency Council: Small 

Business Administration: Social Security Administration: and U.S. Agency for International 

Development. CHCOs from all 27 agencies responded to our questionnaire.
Appendix I: Objectives, Scope, and 

Methodology

Page 54 GAO-12-878 Federal Training Investments 

• “nearly all” users represents 23 to 27 users,

• “most” users represents 18 to 22 users,

• “many” users represents 13 to 17 users,

• “several” users represents 8 to 12 users,

• “some” users represents 3 to 7 users, and

• “few” users represents 0 to 3 users.

To obtain additional perspective and insights on the training investment 

practices identified in the questionnaire, we discussed the responses with 

CHCO and chief learning officers (CLO) councils. In addition, based on 

the responses to the questionnaire and workforce size, we selected four 

agencies (the Department of Homeland Security, Department of Veterans 

Affairs, Department of the Interior, and Department of Energy) from which 

to obtain illustrative examples of how they implemented the training 

investment practices identified in the questionnaire. (See table 6 for 

selection traits). As part of our review of agency practices, we also 

obtained information on the steps that agencies are taking to identify and 

prioritize investment allocations for training required to implement the

GPRA Modernization Act of 2010 (GPRAMA).5

 

5

GPRAMA mandated the Director of OPM to: (1) within 1 year of enactment, identify, in 

consultation with the Performance Improvement Council, the key skills and competencies 

needed by federal government personnel for developing goals, evaluating programs, and 

analyzing and using performance information for the purpose of improving government 

efficiency and effectiveness; (2) within 2 years of enactment, incorporate, as appropriate, 

key skills and competencies into relevant position classifications; and (3) within 2 years of 

enactment, work with each agency to incorporate the key skills identified into training for 

relevant employees at each agency. Pub.L. No. 111-352, 124 Stat. 3866 (Jan. 4, 2011). 
Appendix I: Objectives, Scope, and 

Methodology

Page 55 GAO-12-878 Federal Training Investments 

Table 6: Selection of Case Example Agencies

Agency 

selected Workforce size

Training 

oversight 

structure

Sets a specific 

level of training 

investment a

Agency-wide 

process for 

prioritizing 

training 

investments

Centralized or 

decentralized 

training investment 

processes

Agency-wide 

process for 

tracking cost 

and delivery 

of training 

DHS Greater than 

100,000

None No Yes both No

VA Greater than 

100,000

Oversight 

committee and 

learning board

Yes Yes both Yes

DOE Less than 

100,000

Oversight 

committee and 

learning board

No Yes decentralized Yes

DOI Less than 

100,000

Oversight 

committee 

No No decentralized No

Source: GAO analysis of CHCO responses to a GAO questionnaire.

a During our meetings with DHS, VA, DOI and DOE, officials clarified or changed their answers to 

some of the other responses regarding their oversight structure as well as prioritizing and tracking 

training investments agency-wide. Similarly, VA, DOE, and DOI reported having decentralized 

processes in their questionnaire. However, when we met with officials in the agency, they described 

have both some centralized and decentralized training investment processes. 

To identify and assess OPM’s oversight and guidance to agencies on 

training investment strategies, we reviewed OPM training guidance, 

relevant documentation on forums, workshop or other assistance, and 

oversight activities. In addition, we interviewed officials from OPM offices 

with primary responsibility for providing training policy guidance and 

technical assistance to agencies. We compared this information to the 

leading practices identified in table 6. We also identified and described 

the steps, that OPM has taken to identify the skills and training needed to 

implement performance management improvements, such those required 

by GPRAMA, as a foundation for future agency training investments. 

However, we did not assess the effectiveness of OPM’s efforts to identify 

GPRAMA-related skills and actions to develop related training

Based on information obtained from agencies and OPM, we assessed 

which leading training investment practices were being implemented by 

agencies and addressed by OPM guidance and assistance. We also 

identified the challenges or limitations reported by agencies to 

implementing the practices, and opportunities for improvement in agency 

processes and related OPM guidance.
Appendix I: Objectives, Scope, and 

Methodology

Page 56 GAO-12-878 Federal Training Investments 

We conducted this performance audit from December 2011 to September 

2012, in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.
Appendix II: Table of Statutes, Regulations, 

and Executive Orders Related to Leading 

Practices 

Page 57 GAO-12-878 Federal Training Investments 

Appendix II: Table of Statutes, Regulations, 

and Executive Orders Related to Leading 

Practices 

Leading Training Investment Practices Related Statutes, Executive Orders, and OPM Regulations a

1. Agencies should (a) Identify the appropriate level 

of investment to provide for training and 

development efforts and (b) prioritize funding so 

that the most important training needs are 

addressed first

• Exec. Order No. 11348, section 303(e) requires agency heads to 

establish priorities for needed training, and provide for the use of funds 

and man-hours in accordance with these priorities.

• 5 C.F.R. § 410.201(c), in implementing the E.O., requires agency heads 

(or designee(s)) to establish priorities for training employees and allocate 

resources according to those priorities.

2. Agencies should identify the most appropriate 

mix of centralized and decentralized approaches 

for its training and development programs 

There are no statutory, regulatory, or Executive Order requirements 

directly related to this practice.

3. Agencies should consider government-wide 

reforms and other targeted initiatives to improve 

management and performance when planning its 

training and development programs 

There are no statutory, regulatory, or Executive Order requirements 

directly related to this practice.

4. Agencies should have criteria for determining 

whether to design training and development 

programs in-house or obtain these services from 

a contractor or other external source 

There are no statutory, regulatory, or Executive Order requirements 

directly related to this practice.

5. Agencies should compare the merits of different 

delivery mechanisms (such as classroom or 

computer-based training) and determine what 

mix of mechanisms to use to ensure efficient and 

cost-effective delivery 

There are no statutory, regulatory, or Executive Order requirements 

directly related to this practice.

6. Agencies should track the cost and delivery of its 

training and development programs

• 5 U.S.C. 4118 authorizes OPM to prescribe regulations—and in doing 

so—to specifically provide for

• the maintenance of necessary information concerning the general 

conduct of the training activities of each agency, and such other 

information as is necessary “to enable the President and Congress 

to discharge effectively their respective duties for supervision, 

control and review of these training programs.” 

• The submission of reports by agencies on results and effects of 

training programs and plans and economies resulting therefrom, 

including estimates of costs of training.

• 5 C.F.R. § 410.601(a) requires agencies to maintain records of 

training plans, expenditures, and activities in such form and manner 

as necessary to submit to OPM. Subsection (b) provides that 

beginning December 31, 2006, agencies are to report training data 

at such times and in such form as required for OPM’s governmentwide Electronic Data Collection System.
Appendix II: Table of Statutes, Regulations, 

and Executive Orders Related to Leading 

Practices 

Page 58 GAO-12-878 Federal Training Investments 

Source: GAO Analysis of relevant laws, regulations, and Executive Orders.

a

These key training investment practices are part of the framework outlined in the GAO’s guide GAO, 

Human Capital: A Guide for Assessing Strategic Training and Development Efforts for the Federal 

Government, GAO-04-546G (Washington, D.C.: March 2004). This guide summarizes attributes of 

effective training and development programs and it is based on the GAO analysis of prior work, other 

related expert studies, and federal training requirements. 

b

5 USC 4103(a) requires agency head to establish, operate, maintain, and evaluate a program or 

programs, and a plan or plans there under, for the training of employees. 

Agencies should evaluate the benefits achieved 

through training and development programs, including 

improvements in individual and agency performance

• 5 U.S.C . § 4103(c) requires the head of an agency to evaluate, on a 

regular basis, each program or plan established, operated, or maintained 

under subsection (a)b

• Exec. Order. No. 11348, section 303(a) requires agency heads to 

review at least annually the agency’s program to identify training needed 

to bring about more effective performance at the least possible cost.

with respect to accomplishing specific performance 

plans and strategic goals in performing the agency mission and modify 

such program or plan as needed to accomplish such plans and goals.

• 5 C.F.R . § 410.202–provides that agencies must evaluate their training 

programs annually to determine how well such plans and programs 

contribute to mission accomplishment and meet organizational 

performance goals.

• 5 C.F.R. 410.201(d)(4), requires heads of agencies (or their designee(s)) 

to develop and maintain plans and programs that assess periodically, but 

not less often than annually, the overall agency talent management 

program to identify training needs within the agency as required by 

section 303 of E.O. 11348. 

Agencies should compare training investments, 

methods, or outcomes with those of other 

organizations to identify innovative approaches or 

lessons learned 

There are no statutory, regulatory, or Executive Order requirements directly 

related to this practice.
Appendix III: Illustrations of Building a 

Business Case for Training from OPM’s Guide 

to Strategically Planning Training and 

Measuring Results 

Page 59 GAO-12-878 Federal Training Investments 

Figure 1: OPM illustration of questions used to develop a business case for training

Appendix III: Illustrations of Building a Business Case 

for Training from OPM’s Guide to Strategically Planning 

Training and Measuring Results
Appendix III: Illustrations of Building a 

Business Case for Training from OPM’s Guide 

to Strategically Planning Training and 

Measuring Results 

Page 60 GAO-12-878 Federal Training Investments 

Figure 2: OPM illustration of developing a business case for training
Appendix IV: Comments from the Office of 

Personnel Management

Page 61 GAO-12-878 Federal Training Investments 

Appendix IV: Comments from the Office of 

Personnel Management 
Appendix IV: Comments from the Office of 

Personnel Management

Page 62 GAO-12-878 Federal Training Investments 
Appendix IV: Comments from the Office of 

Personnel Management

Page 63 GAO-12-878 Federal Training Investments 
Appendix IV: Comments from the Office of 

Personnel Management

Page 64 GAO-12-878 Federal Training Investments 
Appendix IV: Comments from the Office of 

Personnel Management

Page 65 GAO-12-878 Federal Training Investments 
Appendix V: GAO Contact and Staff 

Acknowledgments

Page 66 GAO-12-878 Federal Training Investments 

Yvonne J. Jones, Director (202) 512-2717 or Jonesy@gao.gov

In addition to the contact named above, William Doherty (Assistant 

Director), Latesha Love, Angela Leventis, and Karin Fangman made key 

contributions to this report. Also contributing to this report were Benjamin 

Crawford, Eric Gorman, and Natalie Maddox.

Appendix V: GAO Contact and Staff 

Acknowledgments 

GAO Contact

Staff 

Acknowledgments

(450948)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our . Podcasts

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.