a

GAO

United States General Accounting Office

Report to Congressional Requesters

June 2004 INFORMATION 

TECHNOLOGY

Training Can Be 

Enhanced by Greater 

Use of Leading 

Practices

GAO-04-791
www.gao.gov/cgi-bin/getrpt?GAO-04-791.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Dave Powner 

at (202) 512-9286 or pownerd@gao.gov. 

Highlights of GAO-04-791, a report to

congressional requesters 

June 2004

INFORMATION TECHNOLOGY 

Training Can Be Enhanced by Greater Use 

of Leading Practices

Although federal agencies differ widely in how much IT training they 

provide, their use of the 22 leading IT training practices that we identified 

was generally not extensive. Of these practices, only 5 were in use to a great 

or very great extent in a majority of agencies (see figure below). In 

particular, of the three practices we identified in the area of evaluating 

training, none was widely used. For example, for the practice of collecting 

information on how job performance is affected by training, only three 

agencies reported use to a great or very great extent. 

The most commonly cited obstacles to effective IT training were funding and 

the time training takes away from work. To address these obstacles, 

agencies are looking at ways to reduce training time and costs—for example, 

by greater use of e-learning. 

OPM has made limited progress in issuing policies or performing evaluations 

regarding IT training. The E-Gov Act sets requirements for agency IT training 

programs. To provide oversight, it requires OPM to issue policies to promote 

the development of performance standards for training, and to evaluate 

agency implementation of the act’s IT training provisions. These policies are 

particularly important in view of the lack of extensive agency use of the 

evaluation practices we identified. OPM has begun drafting guidance, but it 

has not issued policies or evaluated agency implementation of the act. Until 

policies are issued and progress is measured, oversight of federal IT training 

will continue to fall short of what the act calls for.

Percentage of Agencies Using Leading Practices Extensively 

Effective training of information 

technology (IT) staff, as called for 

in the E-Government (E-Gov) Act 

of 2002, is essential to developing 

and retaining a qualified workforce. 

In an earlier report (GAO-03-390), 

we identified 22 leading practices, 

grouped into 5 key training 

management processes, used by 

private-sector companies to 

implement effective IT training. 

These practices suggest 

approaches that government 

agencies could consider. 

To assess IT training in the federal 

government, including its use of 

leading practices, we were asked to 

determine, among other things, to 

what extent federal agencies use 

our leading practices, the major 

obstacles in providing effective IT 

training and how agencies address 

them, and the progress the Office 

of Personnel Management (OPM) is 

making in issuing policies and 

performing evaluations to 

encourage agencies to provide 

effective IT training. 

GAO is making recommendations 

to OPM, including that it issue IT 

training policies and evaluate 

implementation of the training 

provisions of the E-Gov Act. 

In response to a draft of this report, 

OPM disagreed with the 

conclusions on its role and efforts 

with respect to federal IT training. 

Most other agencies generally 

agreed with the report or had no 

comment. 
Page i GAO-04-791 Information Technology

Contents

Letter 1

Recommendations for Executive Action 3

Agency Comments and Our Evaluation 3

Appendixes

Appendix I: Information Technology: Federal IT Training 7

Appendix II: Comments from the Office of Personnel Management 70

Appendix III: Comments from the Department of Veterans Affairs 72

Appendix IV: Comments from the Department of Health and Human 

Services 74
Contents

Page ii GAO-04-791 Information Technology

Abbreviations

APPL Academy of Program and Project Leadership 

BLM NTC Bureau of Land Management National Training Center 

CIO Chief Information Officer

DHS Department of Homeland Security 

E-Gov Act E-Government Act of 2002

GoLearn Gov Online Learning Center 

GSA General Services Administration

HHS Department of Health and Human Services

IDP Individual Development Plan

IRMC National Defense University Information Resources 

Management College 

IT Information Technology

NASA National Aeronautics and Space Administration 

NIH National Institutes of Health

OMB Office of Management and Budget

OPM Office of Personnel Management

PMDP Project Management Development Process 

STAR Strategic and Tactical Advocates for Results 

USDA U.S. Department of Agriculture

VA Department of Veterans Affairs

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately.
Page 1 GAO-04-791 Information Technology

United States General Accounting Office

Washington, D.C. 20548

A

June 24, 2004 Letr

The Honorable Tom Davis

Chairman

Committee on Government Reform

House of Representatives

The Honorable Jim Turner

House of Representatives

The rapid pace of technological change, with its potential to transform the 

way the government delivers services, makes information technology (IT) 

human capital a critical issue for federal agencies.1

 To ensure that the 

federal government meets its human capital needs, effective training of IT 

staff, as called for in the E-Government (E-Gov) Act of 2002,2

 is essential to 

developing and retaining a qualified workforce. At your request, in an 

earlier report, we examined IT training practices of leading private-sector 

companies. We found 22 existing and emerging practices (see attachment 1 

in app. I) that are used to implement effective IT training. The practices and 

associated case studies that we presented suggest approaches that 

government agencies could consider. In addition, we noted several critical 

issues (e.g., funding constraints and demonstrating return on investment) 

that should be considered in implementing these practices. More recently, 

we issued a final version of a guide for assessing strategic training efforts 

that provides a framework that federal agencies can use to ensure that their 

training investments are targeted strategically.3

To assess IT training in the federal government including its use of leading 

practices, you asked us to determine (1) how federal agencies provide ITrelated training and to what extent they use the leading practices that we 

identified, (2) the major obstacles in providing effective IT training and 

how agencies address them, (3) the factors agencies should consider in 

preparing for future IT training, and (4) the progress the Office of 

1

We have identified strategic human capital management as a high-risk area. See U.S. 

General Accounting Office, High Risk Series: Strategic Human Capital Management, 

GAO-03-120 (Washington, D.C.: January 2003).

2

Pub. L. 107-347 (Dec. 17, 2002).

3

U.S. General Accounting Office, Human Capital: A Guide for Assessing Strategic 

Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, 

D.C.: March 2004). 
Page 2 GAO-04-791 Information Technology

Personnel Management (OPM) is making in issuing policies and performing 

evaluations to encourage agencies to provide effective IT training. To 

accomplish these objectives, we conducted about 100 structured 

interviews with executives, training managers, and training recipients at 26 

agencies and departments. We administered a survey to agencies and asked 

them to provide basic statistics and to self-assess the extent to which they 

were using the leading practices we had identified. We followed up with 

agencies to verify or correct data, but we did not independently verify 

survey responses. We researched existing reports, including previous 

reports by us and other agencies; reviewed related training legislation and 

guidance; and conducted interviews with experts on federal IT training. To 

evaluate OPM’s progress in implementing provisions of the E-Government 

Act, we analyzed OPM’s responses to written questions and held 

discussions with OPM training officials.

Earlier this year, we provided briefing slides to your staff on the results of 

our study. These results, updated to include additional information 

requested, are included as an appendix to this letter. The purpose of this 

letter is to officially transmit the information to you.

In brief, federal agencies provide IT training in a variety of ways. They use 

multiple delivery methods, acquire training from multiple sources, and 

differ widely in the amount of training they provide and the amount they 

spend. Agencies reported using leading IT training practices, but most were 

not used to a great or very great extent. Of the 22 practices we identified, 

only 5 were in use to a great or very great extent in a majority of agencies. 

Increased use of these practices can result in more effective training 

management. We also noted that several federal agencies and other entities 

have programs or initiatives to provide IT training or training resources to 

employees governmentwide. 

Agencies reported that the most common obstacles to effective training are 

funding and the time that training takes away from normal work hours. To 

address these obstacles, agencies are looking at ways to reduce the time 

and costs associated with training—for example, by increasing the use of elearning. 

Major factors in preparing for future IT training cited by agencies included 

rapidly changing technologies and the loss of skills and expertise as the 

aging workforce retires. Further, federal officials emphasized the need for 

training to be focused on areas such as contract management, project 

management, and information security.
Page 3 GAO-04-791 Information Technology

OPM has made limited progress in issuing policies or performing 

evaluations regarding IT training. The E-Gov Act requires agencies to have 

IT training programs that are developed and applied according to rigorous 

standards. To provide oversight, the act requires OPM, in coordination with 

the Office of Management and Budget (OMB), to issue policies to promote 

the development of performance standards for training, and to evaluate 

agency implementation of the act’s IT training provisions. Such policies are 

particularly important in view of the lack of extensive agency use of the 

leading training evaluation practices that we described. In May, OPM told 

us that it plans to consider incorporating our leading practices in future 

guidance. OPM has begun drafting guidance and is in the final stages of 

issuing a report on governmentwide IT training, but it has not issued 

policies or evaluated agency implementation of the act. Until policies are 

established that promote the development of performance standards for 

training, and progress is measured against these standards, agency and 

OPM oversight of federal IT training programs will continue to fall short of 

the expectations established in the act with respect to standards.

Recommendations for 

Executive Action

We recommend that, as part of OPM’s oversight responsibilities under the 

E-Gov Act, the Director of OPM 

• issue governmentwide IT training policies, consistent with our strategic 

training guide, that promote development of performance standards for 

training and encourage agencies to use the leading practices that we 

identified; and

• set specific milestones for evaluating agency implementation of the 

provisions cited above.

Agency Comments and 

Our Evaluation

We sent a draft of this report to all 24 agencies4 that provided information 

and to OMB. We received written, oral, or e-mail comments from 9 

agencies; 11 agencies responded that they did not have comments; 5 

agencies did not respond.

4

Although we counted Department of Defense components separately in our data, Defense 

handled agency comments centrally as a single response.
Page 4 GAO-04-791 Information Technology

In a written response, OPM disagreed with our conclusions about its role 

and efforts with respect to federal IT training. OPM stated that it has not 

fallen short of the expectations established by the E-Gov Act, noting that 

the draft report focuses on only one of six statutory responsibilities under 

the act and that we fail to acknowledge that OPM is to work with the CIO 

Council and OMB in addressing federal IT personnel needs. Our report 

focuses on policies because we believe these are a foundation for 

exercising oversight of the other areas of responsibility mentioned. We cite 

OPM’s plans to coordinate with OMB and the CIO Council on slide 37 

(p. 43), and we note that OMB generally agreed with the findings and 

conclusions in our report. We adjusted our report to clarify that OPM 

oversight will fall short of the expectations of the E-Gov Act with respect to 

standards.

With respect to issuing policies, OPM states that the report does not fully 

and fairly inform the reader as to the status of the policies. In its response, 

OPM refers to an unpublished report in final clearance that will 

recommend a governmentwide IT training framework and announces that 

it has developed milestones for issuing policies. We mention the 

unpublished report on slide 37, but it has not been provided to us, so we are 

unable to comment on it. We have modified our report to reflect OPM’s 

announcement of milestones. Setting milestones is a positive step toward 

issuing policies.

OPM also took exception to our report’s statement that it has made little 

progress in evaluating agency implementation of the E-Gov Act provisions. 

The response states that OPM will incorporate a review of agency use of 

the Governmentwide IT Training Framework and standards in its regular 

review of agencies’ mandated training carried out under its Human Capital 

Assessment and Accountability Framework. Since the IT Training 

Framework is to be set forth in the unpublished report referred to in the 

previous paragraph, it is clear that no specific evaluation of IT training has 

yet been done.

OPM also objected to “the use of GAO’s 22 standards as a mandatory 

benchmark.” We believe this is a misreading of our recommendation. We 

cite 22 leading practices, not standards, and we recommend that OPM 

encourage their use. We believe, and experts agree, that greater use of 

these practices could result in better IT training management.

OPM’s comments are reproduced in appendix II.
Page 5 GAO-04-791 Information Technology

Eight other agencies submitted comments:

• In oral comments, representatives of the Office of Management and 

Budget Personnel Policy Branch and Office of E-Government and 

Information Technology generally agreed with the findings and 

conclusions in the report and also provided some technical comments, 

which were addressed as appropriate.

• In written comments, the Department of Veterans Affairs (VA) generally 

agreed with our conclusions and provided additional information about 

its training programs. VA’s comments are reproduced in appendix III.

• The GAO Liaison for the Department of Transportation indicated via email that Transportation generally concurred with our report.

• The Acting Principal Deputy Inspector General of the Department of 

Health and Human Services (HHS) characterized our report as 

“generally on the mark” but had some concerns about how our leading 

practices—particularly those on identifying training needs and 

evaluating training—would apply to professional researchers at the 

National Institutes of Health (NIH). The response states that because 

our practices represent conventional wisdom in training theory, they 

might not apply to creative programs that meet unique training needs, 

such as those at NIH. We agree that medical researchers at NIH are not 

typical of the federal IT workforce. However, the leading practices are 

general in nature, are widely accepted, and allow for flexibility in 

application. HHS’s comments are reproduced in appendix IV.

• The GAO Review Liaison of the Nuclear Regulatory Commission 

indicated via e-mail that the commission supports the effort to 

encourage greater use of leading practices across federal agencies.

• The Deputy Associate CIO for Business and Information Management of 

the Department of Energy, commenting via e-mail, agreed with the need 

for policies. She also recommended that we list all reported obstacles to 

effective training and that agencies’ successes in dealing with obstacles 

should be shared across the government. In appendix I, slide 43, we list 

methods of addressing the two most important obstacles—time and 

funding, which were cited by almost all agencies. To promote sharing of 

successes, we include case studies of effective agency training programs 

in appendix I, attachment 4. Accordingly, we did not modify the report.
Page 6 GAO-04-791 Information Technology

• The Department of Defense GAO Liaison sent technical comments via email, which were addressed as appropriate.

• The Department of Justice GAO Liaison sent technical comments via email, which were addressed as appropriate.

Unless you publicly announce the contents of this report earlier, we plan no 

further distribution until 30 days from the report date. At that time, we will 

send copies of this report to interested congressional committees. In 

addition, copies will be made available to others upon request. Copies of 

this report will also be available at no charge on GAO’s Web site at

www.gao.gov.

If you or your offices should have any questions concerning this report, 

please contact me at (202) 512-9286. I can also be reached by e-mail at 

pownerd@gao.gov. Key contributors to this report were Barbara Collier, 

Megen Davis, Pam Greenleaf, Kush Malhotra, David Noone, John Ortiz, 

Tomás Ramirez Jr., Jamie Pressman, Glenn Spiegel, and Angela Watson. 

David A. Powner 

Director, Information Technology Management Issues
Page 7 GAO-04-791 Information Technology

Appendix I

Information Technology: Federal IT Training ApApenednxidexiIs

1

Information Technology:

Federal IT Training

Updated Briefing to the Committee on Government Reform

and to the Honorable Jim Turner

House of Representatives
Appendix I

Information Technology: Federal IT Training

Page 8 GAO-04-791 Information Technology

2

Table of Contents

Introduction

Objectives, Scope, and Methodology

Results in Brief

Background

How Agencies Provide Training

Obstacles to Effective IT Training and Strategies for Dealing with Obstacles

Factors to Consider in Preparing for Future IT Training

OPM Progress in Issuing Governmentwide Policies

Conclusions

Recommendations

Attachments

1. List of Leading Practices

2. List of Agencies and Components

3. IT Training Statistics and Data Issues

4. Case Studies
Appendix I

Information Technology: Federal IT Training

Page 9 GAO-04-791 Information Technology

3

Introduction

Our recent work,1 as well as studies by other organizations,2 has highlighted human 

capital shortfalls, including training-related issues, such as training management and 

employee satisfaction with training. Accordingly, we have identified strategic human 

capital management as a high-risk area for the federal government.

Concerning IT training issues, we issued a report, IT Training: Practices of Leading

Private-Sector Companies (GAO-03-390, January 2003), in which we described how

leaders in IT training from the private sector implemented five key training

management processes—aligning IT training with strategic goals; identifying and 

assessing IT training needs; allocating IT training resources; designing and

delivering IT training; and evaluating/demonstrating the value of IT training

(“evaluating IT training”). We identified 22 practices distributed across the key 

management processes. Experts in this field agreed that these practices could result

in more effective training management. See attachment 1 for a listing of these

practices.

1See, for example, the following: U.S. General Accounting Office, Military Transformation: Progress and Challenges for DOD’s Advanced

Distributed Learning Programs, GAO-03-393 (Washington, D.C.: Feb. 28, 2003); Export Promotion: Government Agencies Should 

Combine Small Business Export Training Programs, GAO-01-1023 (Washington, D.C.: Sept. 21, 2001); Veterans’ Benefits: Training for

Claims Processors Needs Evaluation, GAO-01-601 (Washington, D.C.: May 31, 2001); and Acquisition Reform: GSA and VA Efforts to

Improve Training of Their Acquisition Workforces, GAO/GGD-00-66 (Washington, D.C.: Feb. 18, 2000).

2See, for example, U.S. Office of Personnel Management (OPM), What Do Federal Employees Say? Results from the 2002 Federal

Human Capital Survey (Washington, D.C.: March 2003).
Appendix I

Information Technology: Federal IT Training

Page 10 GAO-04-791 Information Technology

4

Objectives, Scope, and Methodology

Objectives

• How are agencies providing information management and technology-related

training, and to what extent do they use leading practices?

• What are the major obstacles in providing effective IT training, and how are

agencies addressing them?

• What factors should agencies consider in preparing for future IT training?

• What progress is the Office of Personnel Management (OPM) making in

issuing policies and performing evaluations to encourage agencies to provide

effective IT training?
Appendix I

Information Technology: Federal IT Training

Page 11 GAO-04-791 Information Technology

5

Objectives, Scope, and Methodology

Scope and Methodology

To determine how federal agencies provide IT training and use leading practices,

identify obstacles and factors regarding IT training and provide additional

information on OPM’s implementation of E-Gov Act provisions, we performed the 

following activities: 

• We conducted about 100 structured interviews with executives, training 

managers, and training recipients at 26 agencies and departments. We

interviewed the departments and agencies covered by the Chief Financial

Officers Act, the Department of Homeland Security (DHS), and two

components of the Department of Defense—the Army and Navy. 

• We researched existing reports, including previous reports by us and other 

agencies, reviewed related training legislation and guidance, and conducted

interviews with experts on federal IT training.
Appendix I

Information Technology: Federal IT Training

Page 12 GAO-04-791 Information Technology

6

Objectives, Scope, and Methodology

• We administered a survey to agencies and asked them to self-assess the 

extent to which they were using the leading practices that we had previously

identified. We also collected basic statistics on IT training (see attachment 3). If 

survey responses raised concerns about consistency or accuracy, we followed

up with agencies to verify or correct data, but we did not independently verify

survey responses. In reporting the aggregated results, we do not include

information from DHS because the department did not exist in fiscal year 2002.

The Department of the Army did not provide statistics in its survey response.

We included information on Department of Energy practices, but not statistics,

because Energy was unable to separate statistics for federal employees from

those for contractors.

• We developed case studies of successful programs and practices implemented

by federal agencies (see attachment 4).

• We interviewed officials from various federal entities that provide 

governmentwide training programs or support initiatives related to workforce

development.

• The departments and agencies that we interviewed or that provided a survey

response are listed in attachment 2.
Appendix I

Information Technology: Federal IT Training

Page 13 GAO-04-791 Information Technology

7

Objectives, Scope, and Methodology

• To evaluate OPM’s progress in issuing IT training policies and evaluating

agency implementation of the E-Gov Act, we analyzed OPM’s response to 

written questions and conducted interviews with OPM officials.

We conducted our review from February 2003 to May 2004. We conducted our 

work in accordance with generally accepted government auditing standards.
Appendix I

Information Technology: Federal IT Training

Page 14 GAO-04-791 Information Technology

8

Results in Brief

Federal agencies provide IT training in a variety of ways. They use multiple delivery 

methods, acquire training from multiple sources, and differ widely in the amount of 

training they provide and the amount they spend. In addition, agencies reported

limited use of leading IT training practices. We also noted that several federal

agencies and other entities have programs or initiatives to provide IT training or 

training resources to employees governmentwide.

Agencies reported that the most common obstacles to effective training are funding

and the time training takes away from normal work hours. To address these

obstacles, agencies are looking at ways to reduce the associated time and costs 

associated with training—for example, by increasing the use of e-learning.

Regarding major factors that need to be considered in preparing for future IT

training, agencies cited rapidly changing technologies and the loss of skills and

expertise as the aging workforce retires; both factors could impact future training.

Further, federal officials emphasized the need for training to be focused on areas 

such as contract management, project management, and information security.
Appendix I

Information Technology: Federal IT Training

Page 15 GAO-04-791 Information Technology

9

Results in Brief

The E-Gov Act sets requirements for agency IT training programs. To provide

oversight, it requires OPM to issue policies to promote the development of 

performance standards for training, and to evaluate agency implementation of the 

act’s IT training provisions. These policies are particularly important in view of the 

lack of extensive agency use of the evaluation practices that we identified. OPM 

has begun drafting guidance, but it has not issued policies or evaluated agency

implementation of the act. Until policies are issued and progress is measured,

oversight of federal IT training will continue to fall short of what the act calls for.

We are making recommendations to OPM, including that it issue governmentwide

IT training policies and set milestones for evaluating agency implementation of the 

IT training provisions of the act.
Appendix I

Information Technology: Federal IT Training

Page 16 GAO-04-791 Information Technology

10

Background

As we reported in our strategic training guide,1 to effectively address the nation’s most

pressing priorities, federal agencies will need to invest resources, including time and money, 

to ensure that employees have the information, skills, and competencies that they need to 

work effectively in a rapidly changing and complex environment. The guide provides a

framework federal agencies can use to assure that training investments are targeted

strategically.

Our strategic training guide also highlighted human capital shortfalls, such as insufficient

training, duplicative and uncoordinated training efforts within and across agencies, and

incomplete information on the extent to which employees had received required training.

Further, the 2002 Federal Human Capital Survey,2 conducted by the Office of Personnel 

Management (OPM), showed that only about half of federal employees were satisfied with the 

training they receive.

Studies have shown that training may be a problem area for the federal government.

According to a study by the National Association of Public Administration,3 there is a lack of

investment in continuous learning within the federal government. The study further stated that

this lack of investment is especially problematic in the dynamic and rapidly changing world of 

IT.

1 U.S. General Accounting Office, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal

Government, GAO-03-893G (Washington, D.C.: July 2003). [The final version was published in March, 2004 as GAO-04-546G.] 2 OPM, What Do Federal Employees Say? Results from the 2002 Federal Human Capital Survey (Washington, D.C.: March 2003). 3 National Academy of Public Administration, The Transforming Power of Information Technology (August 2001).
Appendix I

Information Technology: Federal IT Training

Page 17 GAO-04-791 Information Technology

11

Background

As noted earlier, in January 2003, we issued a report on training practices of

leading private-sector companies.1 Our study discussed how leaders in IT training

from the private sector implemented five key training management processes

(aligning IT training with strategic goals; identifying and assessing IT training

needs; allocating IT training resources; designing and delivering IT training; and 

evaluating/demonstrating the value of IT training). These management processes

are consistent with those in the Strategic Training Guide. We identified 22 practices

distributed across the key management processes. Experts agreed that the

practices we identified could result in more effective training management.

1 U.S. General Accounting Office, Information Technology Training: Practices of Leading Private-Sector Companies, GAO-03-

390 (Washington, D.C.: Jan. 31, 2003).
Appendix I

Information Technology: Federal IT Training

Page 18 GAO-04-791 Information Technology

12

How Agencies Provide Training

Federal Agencies Provide IT Training in a Variety of Ways

Agencies use multiple delivery methods, acquire training from multiple sources,

and differ widely in the amount of training they provide and the amount they spend. 

Agencies frequently stated that they provide training through conferences, college

level courses, and certification programs. Some provide formal on-the-job training 

through techniques such as mentoring.

In our earlier report,1 we identified five key management processes:

• aligning IT training with strategic goals,

• identifying and assessing IT training needs,

• allocating IT training resources,

• designing and delivering IT training, and 

• evaluating IT training.

Within each process we identified practices of leading private-sector companies (22

in all) that could improve federal IT training. (Attachment 1 lists all the practices.)

1GAO-03-390.
Appendix I

Information Technology: Federal IT Training

Page 19 GAO-04-791 Information Technology

13

How Agencies Provide Training

Agencies report using these practices, but most were not used to a great or very 

great extent. Of the 22, only 5 were in use to a great or very great extent in a

majority of agencies: “Address future skill needs and new technologies,” “Consider

the benefits and costs associated with various training design and delivery

methods,” “Ensure that resources are allocated for management training,” “Provide 

IT trainees with the flexibility to choose among different IT training delivery

methods,” and “Consider outsourcing training solutions.” These five practices were 

associated with the following management processes:

• aligning IT training with strategic goals,

• allocating IT training resources, and 

• designing and delivering IT training.

None of the practices in the other two management processes—identifying and 

assessing IT training needs and evaluating IT training—were used to a great or 

very great extent in a majority of agencies.
Appendix I

Information Technology: Federal IT Training

Page 20 GAO-04-791 Information Technology

14

How Agencies Provide Training

Management Process: Aligning IT Training with Strategic Goals

As we reported in our earlier report,1 by linking IT training programs to their overall 

business strategy, companies can promote staff development that best achieve 

corporate objectives. Our strategic training guide2 further notes that it is essential 

for agencies to ensure that training and development efforts are undertaken as an 

integral part of their strategic and performance planning processes, and are driven 

by these. Strategic alignment establishes clear linkage between the agency’s

mission and goals and its training and development efforts. As part of alignment,

senior management should ensure that training goals and strategies are 

incorporated into organizational decision making and aligned with organizational

goals.

We also noted that a structured and documented process should be developed to 

ensure that strategic and tactical changes are promptly incorporated into training

and development efforts. This can help ensure that training and development

efforts are not initiated in an ad hoc, uncoordinated manner, but are focused on 

improving performance in support of the agency’s goals.

1 GAO-03-390.

2 GAO-03-893G.
Appendix I

Information Technology: Federal IT Training

Page 21 GAO-04-791 Information Technology

15

How Agencies Provide Training

Management Process: Aligning IT Training with Strategic Goals

During our discussions, agency officials typically described informal or no 

centralized methods of ensuring that IT training supported agencywide goals.

Training was usually managed at the business unit level.

• Three agencies provided documented plans or procedures for aligning

agencywide IT training with strategic plans.

• Five additional agencies reported that agencywide alignment takes place, but 

they did not provide plans or documented methods.

In our earlier report, we identified the following three practices used by leading

companies to align IT training with strategic goals.

xEnlist executive-level champions (sponsorship) to ensure that training strategies are incorporated

into corporate decisionmaking and aligned with business goals

xInvolve critical stakeholders, such as top management, business unit managers, subject matter

experts, human capital staff, and end users, in planning IT training

Align IT training

with business

goals

xAddress future skill needs and new technologies as part of the planning process
Appendix I

Information Technology: Federal IT Training

Page 22 GAO-04-791 Information Technology

16

How Agencies Provide Training

Management Process: Aligning IT Training with Strategic Goals

For the “aligning” process, only one of the three practices—“Address future skill

needs and new technologies”—was reported as being in wide use: that is, used to 

a great or very great extent by over half (15 of 26) of agencies.

Source: GAO compilation of agency provided data.
Appendix I

Information Technology: Federal IT Training

Page 23 GAO-04-791 Information Technology

17

How Agencies Provide Training 

Management Process: Identifying and Assessing IT Training Needs 

As we have reported in our earlier report1 and our strategic training guide,2 planning

and preparing training and development efforts are key to positioning federal

agencies to be able to address current problems and meet emerging demands. For 

example, skill needs may change because of new initiatives, new technology,

market forces, workforce attrition, or organizational restructuring. Part of this 

process must include a systematic method of determining what skills and

competencies currently exist and what is needed to address future demand and 

assessing the implications of any gaps. These activities are aligned with ClingerCohen Act3 requirements.

Our earlier study also noted that gathering information from various levels of the 

organization allowed managers to better assess training needs. We further noted 

that industry experts and practitioners were beginning to recognize that IT training

should focus on broader career development needs, as well as skill-specific

training needs.

1 GAO-03-390

2 GAO-03-893G

3 40 U.S.C. § 11315(c)(3).
Appendix I

Information Technology: Federal IT Training

Page 24 GAO-04-791 Information Technology

18

How Agencies Provide Training

Management Process: Identifying and Assessing IT Training Needs

During our discussions, agency officials frequently reported that the identification of 

training needs was employee-driven through the use of individual development

plans (IDPs) or discussions with supervisors. Training needs were typically

assessed at the component or program office level. Some IT training managers

reported that they assessed training needs based on immediate priorities, such as 

the need to perform a system update. Generally, agencies had not developed a 

comprehensive or systematic process for identifying and assessing IT training 

needs.

In our earlier report, we identified the following practices used by leading

companies to identify and assess IT training needs.

None of these six leading practices were reported as being in wide use: that is, 

used to a great or very great extent by over half the agencies. (See next slide.)

xIdentify and document competencies/skills required for each job description

xMaintain a current inventory of skills

xAddress overall career development issues as well as skill-specific training issues

xPerform a gap analysis to determine where training is needed

xUse self-directed tools, such as individual development plans, to give employees

responsibility in assessing their development needs

Identify and assess IT

training needs 

xUse a single portal to give staff and managers access to training and career development

information
Appendix I

Information Technology: Federal IT Training

Page 25 GAO-04-791 Information Technology

19

How Agencies Provide Training 

Management Process: Identifying and Assessing IT Training Needs

Source: GAO compilation of agency provided data.
Appendix I

Information Technology: Federal IT Training

Page 26 GAO-04-791 Information Technology

20

How Agencies Provide Training

Management Process: Allocating IT Training Resources 

As we reported in our earlier report,1 training must compete with other priorities for 

federal resources. It is important that federal agencies prioritize and manage

resources to ensure that training projects are effectively identified and

implemented.

During our discussions, agency officials reported use of a variety of different

processes for prioritizing and allocating resources for IT training. Among those 

mentioned were building training into plans for new projects so that training was 

considered part of the investment process; assigning priorities to certain subject

areas, such as project management or security; and considering the benefit and 

costs of various training delivery options.

1 GAO-03-390.
Appendix I

Information Technology: Federal IT Training

Page 27 GAO-04-791 Information Technology

21

How Agencies Provide Training 

Management Process: Allocating IT Training Resources

In our earlier report, we identified the following practices used by leading

companies to allocate IT training resources.

xEnsure that an investment process is in place to select and manage training projects

xConsider the benefits and costs associated with various training design and delivery

methods—e.g., Internet-based as opposed to classroom training

xIdentify people who have high potential and provide them specialized training opportunities

Allocate IT training

resources

xEnsure that resources are allocated for IT management training—e.g., leadership and

project management
Appendix I

Information Technology: Federal IT Training

Page 28 GAO-04-791 Information Technology

22

How Agencies Provide Training 

Management Process: Allocating IT Training Resources

Two of the four leading practices in the “allocating resources” process were

reported as being in wide use—“Consider the benefits and costs associated with 

various training design and delivery methods” (14 of 26) and “Ensure that 

resources are allocated for management training” (17 of 26).

Source: GAO compilation of agency provided data.
Appendix I

Information Technology: Federal IT Training

Page 29 GAO-04-791 Information Technology

23

How Agencies Provide Training

Management Process: Design and Delivery of IT Training

As noted in our earlier report1 and strategic training guide,2 after needs are 

determined and resources allocated, training should be designed, developed or 

purchased, and delivered to recipients using an appropriate mix of methods. It is 

important that the design and delivery process helps to ensure that employees

learn during the training and apply the new knowledge on the job. 

During our discussion, a majority of the agencies (15) stated that they develop little 

or no IT training. Most agencies reported that they either send staff to external

courses or have vendors deliver IT training in-house.

Only one agency did not permit staff to choose among a variety of delivery

methods. Classroom or hands-on training was generally preferred over e-learning,

especially for technical topics, but e-learning was often cited as cost effective. 

Advantages cited for classroom training include the ability to ask questions,

availability of hands-on practice, and ability to get away from workplace

interruptions and distractions. Advantages cited for e-learning include lower cost, 

avoidance of travel, and ability to work at one’s own pace. In general

classroom/hands-on training was seen as better for more technical topics, and elearning more appropriate for general topics and review.

1GAO-03-390. 2GAO-03-893G.
Appendix I

Information Technology: Federal IT Training

Page 30 GAO-04-791 Information Technology

24

How Agencies Provide Training

Management Process: Design and Delivery of IT Training

Agencies frequently stated that they provide training through conferences, college

level courses, and certification programs. Some provide formal on-the-job training 

through techniques such as mentoring.

The training areas most often reported as receiving emphasis are security and 

project management. Enterprise architecture and investment management were 

also frequently cited.

In our earlier report, we identified the following practices used by leading

companies to design and deliver IT training.

Two of these six leading practices were reported as being in wide use—“Provide IT 

trainees with the flexibility to choose among different IT training delivery methods” 

(14 of 26) and “Consider outsourcing training solutions” (17 of 26). (See next slide.)

xProvide IT trainees with the flexibility to choose among different IT training delivery methods

xEnsure that on-the-job training is planned and monitored as part of the training process

xConsider combining different teaching methods (for example, Web-based and instructor-led)

within the same course

xProvide just-in-time training

xConsider outsourcing training solutions—e.g., university partnerships and external IT training

and content providers

Design and deliver IT

training

xBuild courses using reusable modules
Appendix I

Information Technology: Federal IT Training

Page 31 GAO-04-791 Information Technology

25

How Agencies Provide Training

Management Process: Design and Delivery of IT Training

Source: GAO compilation of agency provided data.
Appendix I

Information Technology: Federal IT Training

Page 32 GAO-04-791 Information Technology

26

How Agencies Provide Training

Management Process: Evaluating IT Training

As noted in our earlier report1 and strategic training guide,2 evaluating IT training

involves assessing the extent to which training and development efforts contribute 

to improved performance and results. Agencies should systematically plan for and 

evaluate the effectiveness of their training and development efforts in order to 

manage scarce resources. Evaluation results should be incorporated into planning,

design, and implementation of training efforts. If training is not evaluated, agencies

risk misallocating resources and cannot ensure that they are meeting training

objectives.

1 GAO-03-390.

2 GAO-03-893G.
Appendix I

Information Technology: Federal IT Training

Page 33 GAO-04-791 Information Technology

27

How Agencies Provide Training

Management Process: Evaluating IT Training

Agencies should use systematic, multilevel approaches to collecting performance

data. We have reported on one commonly accepted model that identifies the 

following five levels of evaluation.1

Level One Participant reactions

Level Two Extent students have advanced in skills, knowledge, or attitude

Level Three Changes in students’ work behavior due to training

Level Four Effect on achieving organizational goals

Level Five Return on investment

1 GAO-03-893G.
Appendix I

Information Technology: Federal IT Training

Page 34 GAO-04-791 Information Technology

28

How Agencies Provide Training

Management Process: Evaluating IT Training

Based on our discussions with agency training officials, we found the following:

• Although all the leading companies in our earlier report were going beyond

obtaining participant reactions (level one), less than half of the federal agencies

(11) were performing evaluation activities above this level, even within agency

components.

• Some content testing is being done. For example, e-learning courses provided

by vendors may include tests. Certifications, such as Microsoft or Cisco, 

provide evidence that training content has been learned.
Appendix I

Information Technology: Federal IT Training

Page 35 GAO-04-791 Information Technology

29

How Agencies Provide Training

Management Process: Evaluating IT Training

In our earlier report, we identified the following practices used by leading

companies to evaluate IT training.

None of the three practices in the “evaluating IT training” process were reported as 

being in use to a great or very great extent by more than half the agencies.

xCollect information on how job performance is affected by training

xValidate IT content learning by testing and certification of specific skills—e.g., Java or C++

Evaluate/demonstrate

the value of IT 

training xAssess evaluation results in terms of business impact

Source: GAO compilation of agency provided data.
Appendix I

Information Technology: Federal IT Training

Page 36 GAO-04-791 Information Technology

30

How Agencies Provide Training

Training Programs and Initiatives Available to Employees

Governmentwide

In addition to agency-specific training programs, several federal agencies and other 

entities have programs or initiatives to provide IT training or training resources to 

employees governmentwide.

OPM’s Gov Online Learning Center (GoLearn). The e-training initiative is one of 

the e-government initiatives in the President’s Management Agenda. The Gov

Online Learning Center, part of the e-training initiative, will serve as a source for 

online training and strategic human capital development for all federal employees.

OPM also plans for the GoLearn Web site to provide tools that will allow human

capital specialists and employees to match an employee’s professional and

individual development to available courses and services. The site includes

competency-based workforce development roadmaps specifically for IT workforce 

occupations.
Appendix I

Information Technology: Federal IT Training

Page 37 GAO-04-791 Information Technology

31

How Agencies Provide Training

Governmentwide Programs and Initiatives

National Defense University’s Information Resources Management College 

(IRMC). IRMC offers graduate level education in information resources 

management to military and civilian personnel. Areas of concentration include

strategic planning, leadership/management, capital planning and investment,

performance and results-based management, architecture, information assurance

and security, and e-government. Students represent multiple communities including

Chief Information Officers (CIOs) and other agency officials. Through agreements

with various universities, IRMC students can earn 15 graduate credits towards

master’s and doctoral programs.

USDA Graduate School. The Graduate School provides IT training to 

approximately 10,000 federal IT professionals a year. Although associated with the 

U.S. Department of Agriculture, it has its own governing board and receives no 

federal funds. Its instructors are independent contractors. 
Appendix I

Information Technology: Federal IT Training

Page 38 GAO-04-791 Information Technology

32

How Agencies Provide Training

Governmentwide Programs and Initiatives

The CIO Council has efforts in IT workforce development.

The federal Chief Information Officers (CIO) Council’s Workforce and Human

Capital for Information Technology Committee (formerly called the Education and

Training Committee) is charged with examining workforce planning, recruitment

and retention, and career development; it serves as one of the government’s key 

advocates for strategies to recruit, develop, and maintain an effective federal IT 

workforce. The following are some of its workforce development initiatives:

Clinger-Cohen Core Competencies. The committee developed a listing of 

Clinger-Cohen core competencies to assist government agencies in complying with 

Clinger-Cohen Act requirements and to serve as a baseline to determine required

course and curriculum training development requirements in the information

technology field. The competencies are updated biennially; the most recent update 

included “enterprise architecture” as a new competency area.
Appendix I

Information Technology: Federal IT Training

Page 39 GAO-04-791 Information Technology

33

How Agencies Provide Training

Governmentwide Programs and Initiatives

IT Workforce Development Roadmap. An online career development tool

designed for current and prospective federal IT workers, the IT Roadmap was

developed by the CIO Council. The tool, currently hosted on GoLearn, assists IT 

professionals to identify competency recommendations, identify relevant training,

and build long-term career plans.

IT Workforce Skills Assessment Survey. Also known as the Clinger-Cohen

Assessment, this survey was conducted via the Internet in September 2003 to 

collect information regarding employee IT competencies, skills, certifications, and 

specialized job activities at federal agencies. This voluntary survey asked IT

employees to assess their own current proficiency in a set of general and technical

competencies and IT-related skills; it also asked employees to indicate broad

certification areas in which they currently possess a recent certification (past 3 

years). The survey also asked respondents to estimate the amount of time they 

spent each day on 10 different specialized job activities (e.g., IT project 

management, IT security or information assurance). Agencies can use the survey

results to satisfy the Clinger-Cohen requirement for a workforce assessment and to 

respond to Office of Management and Budget (OMB) requirements for budget 

submissions. OPM plans to use the survey results as part of its efforts to analyze

the personnel needs of the government relating to IT and information resources
Appendix I

Information Technology: Federal IT Training

Page 40 GAO-04-791 Information Technology

34

How Agencies Provide Training

Governmentwide Programs and Initiatives

management. The IT workforce population for this survey consists of both 

traditional and nontraditional IT-related job series.1 The voluntary survey achieved

a 26 percent response rate.

CIO University. A consortium of universities offering graduate level courses that 

directly address the Clinger-Cohen core competencies for IT professionals, the CIO 

University is sponsored by the CIO Council and administered by the General

Services Administration (GSA). As of November 2003, 289 individuals have

received CIO University Certificates, and an additional 150 individuals are expected

to complete the program in 2004.

STAR Program. The Strategic and Tactical Advocates for Results (STAR)

Program is a cross-discipline, strategic leadership course sponsored by GSA and 

the CIO Council. It is a one-week course offered to mid- and upper-level federal

managers in the information technology (CIO), finance (chief financial officer), 

program (chief executive officer), and procurement areas of an agency. The

program covers the following topics: leadership, program and project management,

capital planning/investment process, technology and e-government, security,

government and Congress. As of November 2003, approximately 225 mid- and

upper-level federal managers have completed the program.

1 E.g., GS-2210, GS-391, and GS-1550, as well as GS-301, GS-340, and GS-343.
Appendix I

Information Technology: Federal IT Training

Page 41 GAO-04-791 Information Technology

35

Obstacles to Effective IT Training and Strategies for 

Dealing with Obstacles

Almost all agencies cited funding and the amount of time that training requires

during normal work hours as common obstacles to providing and receiving effective 

training.

Agencies reported various strategies to address these obstacles. These strategies 

include

• building training requirements into project and program planning;

• increasing the use of e-learning and other delivery methods to reduce costs;

• encouraging training during nonwork hours (specifically, training recipients cited 

working with their supervisors to accommodate a study schedule and to leave

work early for training); and

• prioritizing individual training requests.
Appendix I

Information Technology: Federal IT Training

Page 42 GAO-04-791 Information Technology

36

Factors to Consider in Preparing for Future 

IT Training

Agencies reported various IT factors that need to be considered in preparing for the 

future:

• Rapidly changing technology: this drives changes in skills requirements and

curricula. For example, employees holding certifications may need frequent and 

costly recertifications.

• An aging workforce: retirements can lead to loss of skills and expertise.

• The need for more project management and contract management expertise.

• The need to address federal initiatives, such as e-government and enterprise

architecture.

• An increasing emphasis on cyber security.
Appendix I

Information Technology: Federal IT Training

Page 43 GAO-04-791 Information Technology

37

OPM Progress in Issuing Governmentwide Policies

The E-government Act of 2002 requires agencies to have IT training programs that 

• have curricula covering a broad range of IT disciplines;

• are developed and applied according to rigorous standards; and

• are designed to maximize efficiency, through the use of a variety of training

methods.

To provide oversight over these programs, the act requires OPM, in coordination

with OMB, to issue policies to promote the development of performance standards

for training, and to evaluate agency implementation of the provisions cited above.

OPM officials stated that they are working on draft guidance related to performance

standards, but the draft is currently on hold. OPM plans to coordinate with OMB, 

the CIO Council, and the Chief Human Capital Officers’ Council and to complete 

draft guidance in the fourth quarter of fiscal year 2004. They also told us that OPM 

is in the final stages of issuing a report on governmentwide IT training that sets out 

a framework for their implementation strategy. OPM announced milestones for 

issuing policies in its response to a draft of this report.
Appendix I

Information Technology: Federal IT Training

Page 44 GAO-04-791 Information Technology

38

OPM Progress in Issuing Governmentwide Policies

OPM has made little progress in evaluating agency implementation of the E-Gov 

Act provisions cited above. OPM cited evaluation activities that it plans to do. One 

of these—coordinating and reviewing the ongoing expansion of GoLearn, STAR, 

and CIO University course offerings—would address curricula. However, OPM did 

not identify planned evaluation activities that would address development

standards or efficiency. Further, OPM provided no milestones for evaluation

activities.

OPM said that it will consider incorporating the leading practices cited in our earlier 

report as appropriate in future guidance promoting uniform implementation of the 

E-Gov Act’s provisions by agencies.

Until policies are established that promote the development of performance

standards for training, and progress is measured against these standards, agency

and OPM oversight of federal IT training programs will continue to fall short of the 

expectations established in the E-Gov Act with respect to standards.
Appendix I

Information Technology: Federal IT Training

Page 45 GAO-04-791 Information Technology

39

Conclusions

• Federal agencies provide IT training in a variety of ways. 

• Governmentwide training programs and initiatives are available to the IT 

community.

• Agencies face the difficult obstacles of finding the funds and time to provide

effective IT training, and they continue to explore new ways of dealing with 

these obstacles. In addition, agencies must consider factors such as rapidly

changing technology and an aging workforce to effectively plan for the future.

• Agencies report using the leading practices that we identified. However, most

practices were not used to a great or very great extent, particularly in the area

of training evaluation. Increased use of these practices can result in more 

effective training management.

• Until OPM issues policies promoting the development of performance

standards for training and measures progress against these standards, its 

oversight of federal IT training programs will continue to fall short of the 

expectations established in the E-Gov Act with respect to standards.
Appendix I

Information Technology: Federal IT Training

Page 46 GAO-04-791 Information Technology

40

Recommendations

We recommend that, as part of OPM’s oversight responsibilities under the E-Gov 

Act, the Director of OPM 

• Issue IT training policies, consistent with our strategic training guide, that 

promote development of performance standards for training and encourage

agencies to use the leading practices that we identified, and

• set specific milestones for evaluating agency implementation of the E-Gov Act 

IT training provisions cited above.
Appendix I

Information Technology: Federal IT Training

Page 47 GAO-04-791 Information Technology

41

Attachment 1. Leading Practices 

xEnlist executive-level champions (sponsorship) to ensure that training strategies are incorporated into

corporate decisionmaking and aligned with business goals

x Involve critical stakeholders, such as top management, business unit managers, subject matter

experts, human capital staff, and end users, in planning IT training

Align IT training with

business goals

xAddress future skill needs and new technologies as part of the planning process

x Identify and document competencies/skills required for each job description

x Maintain a current inventory of skills

xAddress overall career development issues as well as skill-specific training issues

xPerform a gap analysis to determine where training is needed

Identify and assess IT

training needs

x Use self-directed tools, such as individual development plans, to give employees responsibility in

assessing their development needs

x Use a single portal to give staff and managers access to training and career development information

xEnsure that an investment process is in place to select and manage training projects

x Consider the benefits and costs associated with various training design and delivery methods—e.g.,

Internet-based as opposed to classroom training

x Identify people who have high potential and provide them specialized training opportunities

Allocate IT training

resources

xEnsure that resources are allocated for IT management training—e.g., leadership and project

management

xProvide IT trainees with the flexibility to choose among different IT training delivery methods

xEnsure that on-the-job training is planned and monitored as part of the training process

x Consider combining different teaching methods (for example, Web-based and instructor-led) within the

same course

xProvide just-in-time training

x Consider outsourcing training solutions—e.g., university partnerships and external IT training and

content providers

Design and deliver IT

training

xBuild courses using reusable modules

x Collect information on how job performance is affected by training

xValidate IT content learning by testing and certification of specific skills—e.g., Java or C++

Evaluate/demonstrate

the value of IT

training x Assess evaluation results in terms of business impact
Appendix I

Information Technology: Federal IT Training

Page 48 GAO-04-791 Information Technology

42

Attachment 2: Departments and Agencies Providing 

Information

Department of Justice

Department of Labor

Department of State

Department of the Treasury

Department of Transportation

Department of Veterans Affairs

Environmental Protection Agency

General Services Administration

National Aeronautics and Space 

Administration

National Science Foundation

Nuclear Regulatory Commission

Office of Personnel Management

Small Business Administration

Social Security Administration

Agency for International Development

Department of Agriculture

Department of Commerce

Department of Defense, Office of the

Secretary of Defense

Department of the Army

Department of the Navy

Department of the Air Force

Defense Information Systems Agency

Department of Education

Department of Energy

Department of Health and Human Services

Department of Homeland Security

Department of Housing and Urban

Development

Department of the Interior
Appendix I

Information Technology: Federal IT Training

Page 49 GAO-04-791 Information Technology

43

Attachment 3: IT Training Statistics

Statistics and data Issues

Time and money spent by agencies on IT training varied widely. Based on agency 

responses, for fiscal year 2002, a majority of agencies (14 out of 23) provided at 

least 5 to 6 days annually in IT training, on the average, for IT staff members. The 

average annual amount spent on IT training per individual IT staff member ranged 

from $301 to $2109.1 (See the next two slides for a distribution of all agency

responses for the amount of time and money spent on IT training.)

However, agencies had difficulty reporting such statistics, because they had

incomplete or limited management information on IT training. Specifically:

• Four agencies reported that they do not track the number of IT staff trained. In 

discussions, agency officials noted difficulties in identifying all their IT staff, 

because the various IT job series do not include all staff performing IT 

functions.

• Nine agencies reported that the data they provided were incomplete (i.e., did

not include all agency components).

• One agency reported that it was unable to separate contractors from federal

employees.

1Calculated on the basis of all training dollars against all IT staff, not just those that received training.
Appendix I

Information Technology: Federal IT Training

Page 50 GAO-04-791 Information Technology

44

Attachment 3: IT Training Statistics

Days Spent on IT Training

Average Number of Days Spent on IT Training during

Fiscal Year 2002

Source: GAO analysis of agency provided data.
Appendix I

Information Technology: Federal IT Training

Page 51 GAO-04-791 Information Technology

45

Attachment 3: IT Training Statistics

Dollars Spent on IT Training

Source: GAO analysis of agency provided data.

Average Number of Dollars Per Staff Member Obligated for IT 

Training during Fiscal Year 2002
Appendix I

Information Technology: Federal IT Training

Page 52 GAO-04-791 Information Technology

46

Attachment 4: Case Studies

Although agencies in general were not making wide use of the leading practices,

agencies identified examples in which effective use of these practices is resulting in

more effective training management. We provide five case studies of these

examples in this attachment.

Identifying Needs

• The Department of the Navy developed a career management tool for its 

workforce.

• The Department of State established a monetary incentive program to retain

employees with necessary skills and to encourage employees to acquire 

professional skills and certifications.

Allocating Resources

• The National Aeronautics and Space Administration (NASA) created a program

to develop project managers.
Appendix I

Information Technology: Federal IT Training

Page 53 GAO-04-791 Information Technology

47

Attachment 4: Case Studies 

Design and Delivery

• The Department of Homeland Security (DHS) established a reusable content

strategy that allows leveraging existing material through multiple courses and

various delivery methods. 

Evaluating IT Training

• Interior’s National Training Center has a method for estimating the value of IT 

training.

No examples were identified of the effective use of practices in the management

process “Aligning IT Training with Strategic Goals.”
Appendix I

Information Technology: Federal IT Training

Page 54 GAO-04-791 Information Technology

48

Attachment 4

Identifying Needs: Case Study 1 

Navy Five-Vector Model Helps Staff Manage and Advance Their Careers

Background

Because the Navy places a high priority on career development and training, it 

created a comprehensive career management program that its workforce can 

access worldwide to enable their success. 

Challenge

To provide training and career management tools to a workforce that is highly

decentralized, complex, and widely dispersed.

Practices Illustrated

• Identify and document skills

• Address career development & training

• Use self-directed tools
Appendix I

Information Technology: Federal IT Training

Page 55 GAO-04-791 Information Technology

49

Attachment 4

Identifying Needs: Case Study 1

Solution

The Navy developed the Five-Vector Model, which users access via its knowledge portal

(www.nko.navy.mil). The tool is used to track, guide, and evaluate career development efforts

for positions that include IT professions.

Source: Department of the Navy.

As shown, there are five 

proficiency areas (“vectors”)

in which staff skills and

abilities are evaluated:

(1) professional development,

(2) personal development,

(3) leadership,

(4) certifications and 

qualifications, and

(5) performance.

In each proficiency area, 

personnel may advance 

through four levels from 

recruit to master.
Appendix I

Information Technology: Federal IT Training

Page 56 GAO-04-791 Information Technology

50

Attachment 4 

Identifying Needs: Case Study 1

Solutions (cont’d)

Users can graphically view advancement opportunities from their current position to 

the next position, map their skill levels along vector points, track individual skills

inventory, and link their position in conjunction with training requirements. Users

may also conduct career advancement searches for planning purposes.

Reported Results

The Navy reported that it can more effectively manage its workforce planning and 

development efforts. Personnel and supervisors report satisfaction in better

managing training and career.
Appendix I

Information Technology: Federal IT Training

Page 57 GAO-04-791 Information Technology

51

Attachment 4 

Identifying Needs: Case Study 2

State Department Implements Skills Incentive Program for Critical IT Skills

Background

Because the department is dependent on a stable and highly trained IT workforce

to support an array of technically sophisticated functions, a high priority was placed 

on training and development.

Challenge

To increase the IT workforce’s expertise and skills base while maintaining a low 

turnover rate.

Solution

The department established a schedule of incentive allowances ranging from 5 to 

15 percent of base pay to retain employees with necessary skills and to encourage

employees to acquire professional skills and certifications. 

Practice Illustrated

• Address career development and training
Appendix I

Information Technology: Federal IT Training

Page 58 GAO-04-791 Information Technology

52

Attachment 4 

Identifying Needs: Case Study 2

Solution (cont’d)

• Bonus of up to 15 percent for highly valued qualifications, such as Cisco 

Certified Systems Engineer or an approved master’s degree.

• Bonus of up to 10 percent for such qualifications as Certified Information 

Systems Security Professional or approved bachelor’s degree.

• Bonus of up to 5 percent for such qualifications as USDA Graduate School

Webmaster Certification or Certified Lotus Notes Systems Administrator.

Reported Results

The department maintains a stable and highly trained workforce. Since inception of 

the program,

• 72 percent of eligible IT employees applied for program enrollment, and

• 61 percent of those who applied were approved.

Before the program started, less than 20 IT staff were certified as Microsoft System 

Engineers. As of October 2003, 345 have been certified.
Appendix I

Information Technology: Federal IT Training

Page 59 GAO-04-791 Information Technology

53

Attachment 4

Allocating IT Training Resources: Case Study

NASA targets resources to develop a cadre of skilled project managers. 

Background

NASA is a highly project-based organization with a workforce made up of nearly

twice as many contractors as civil service staff. In an environment where both the 

number of projects and their complexity is increasing, NASA requires skilled,

qualified project managers.

Challenge

To develop a cadre of employees with the proper management skills to deliver

projects within cost and on schedule.

Practices Illustrated

• Ensure that resources are allocated for management training—e.g.,

leadership and project management

• Combine different teaching methods

• Provide just-in-time training
Appendix I

Information Technology: Federal IT Training

Page 60 GAO-04-791 Information Technology

54

Attachment 4 

Allocating IT Training Resources: Case Study

Solution

NASA established the Academy of Program and Project Leadership (APPL)

program in 1997 to increase program and project management capabilities and 

competencies. APPL provides professional development services to individuals

and teams through a blended learning approach that integrates career 

development, performance enhancement, and knowledge sharing.

• Career development: Addresses the development of project management

competencies and skills. One resource, the Project Management Development

Process (PMDP), sets standards for achieving baseline competencies for 

program and project managers. PMDP also specifies a set of required

coursework at each level of development for participants.

• Performance enhancement: Provides support to the individual or project team 

when needed, extending the learning environment beyond the classroom to

deliver just-in-time support directly to new and existing project teams through 

activities such as mentoring and executive coaching.
Appendix I

Information Technology: Federal IT Training

Page 61 GAO-04-791 Information Technology

55

Attachment 4 

Allocating IT Training Resources: Case Study

Solution (cont’d)

• Knowledge sharing: Promotes leadership development through mentoring and

teaching.

Knowledge Knowledge

Sharing

Career

Development Development

Performance Performance

Enhancement Enhancement

Develop project

leadership

maturity

Create

opportunities for

project

management

collaboration

through research

and exchange

Serve as a

clearinghouse of

world-class

expertise

Communication

and transfer of

knowledge

Reported Benefits of APPL

Career development. Project

managers are better prepared in 

advance of project need.

Performance enhancement. Project

team probability of success is 

enhanced by the delivery of just-intime knowledge.

Knowledge sharing. A knowledge 

sharing community of project leaders

is cultivated within the agency.

Source: NASA. 
Appendix I

Information Technology: Federal IT Training

Page 62 GAO-04-791 Information Technology

56

Attachment 4 

Allocating IT Training Resources: Case Study

Reported Results

Overall, NASA reported the following benefits of this program:

• NASA project managers are better prepared to meet the challenges of today’s 

project environment through the three APPL business lines.

• APPL meets the needs of a diverse group of NASA project managers by

providing a full spectrum of products and services.

• Project managers can customize their professional development based on their 

skill level and amount of experience.

• APPL participants improved their scores on assessments that measure 

performance in project management areas such as leadership and teamwork

(see next slide).
Appendix I

Information Technology: Federal IT Training

Page 63 GAO-04-791 Information Technology

57

Attachment 4 

Allocating IT Training Resources: Case Study

Reported Results (cont’d)

Participants take baseline assessments before taking APPL classes and can 

choose to take a re-assessment to capture trend metrics on performance.

The median score on the APPL Leadership Assessment for participants who chose re-assessment after 3

months improved from 78 to 83 percent. APPL considers 80 percent to be the minimum acceptable

effectiveness rate and plans to revisit participants whose scores fall below the minimum and encourage

coaching and re-assessment. For additional information on NASA’s APPL program, see

www.appl.nasa.gov.

Leadership Scores Before Class

•-1

•0

•1

•2

•3

•4

•5

•6

•7

•50 •55 •60 •65 •70 •75 •80 •85 •90 •95

Overall Scores in 5% increments

•Number of Scores

Median = 78%

Leadership Scores 3 Months Later

•-1

•0

•1

•2

•3

•4

•5

•6

•7

•50 •55 •60 •65 •70 •75 •80 •85 •90 •95

Overall Scores in 5% increments

•Number of Scores

Median = 83%

Source: NASA.
Appendix I

Information Technology: Federal IT Training

Page 64 GAO-04-791 Information Technology

58

Attachment 4

Design and Delivery: Case Study

DHS’s Reusable Content Design and Development Strategy Offers Flexibility 

in Choosing among a Variety of Delivery Methods

Background

DHS’s Technology Training Services office in the Bureau of Immigration and

Customs Enforcement (formerly the Immigration and Naturalization Service)

provides training design, development, and delivery services to support IT 

professionals and technical support employees.

Challenge

To provide training through a variety of delivery methods in a cost-effective 

manner.

Practices Illustrated

• Provide IT trainees with the flexibility to choose among different IT 

training delivery methods

• Build courses using reusable components
Appendix I

Information Technology: Federal IT Training

Page 65 GAO-04-791 Information Technology

59

Attachment 4

Design and Delivery: Case Study

Solution

The agency established a standard template for course developers to use in

designing and delivering training courses. This approach allowed designers to reuse 

course content in various courses while using various delivery methods. For 

example, as the following slide shows, course developers were able to reuse

content from components of the LAN Academy instructor-led course to develop four 

additional courses—all delivered using different media modes.

Reported Results 

Course development time and cost were reduced to a fraction of the amount of time 

associated with designing/developing each course from scratch.
Appendix I

Information Technology: Federal IT Training

Page 66 GAO-04-791 Information Technology

60

Attachment 4

Design and Delivery: Case Study

Source: GAO, DHS, Art Explosion (images).

Note: GAO analysis of agency-provided data.
Appendix I

Information Technology: Federal IT Training

Page 67 GAO-04-791 Information Technology

61

Attachment 4

Evaluating Training: Case Study

Department of the Interior Uses Test Results to Estimate Training

Value

Background

The Bureau of Land Management (BLM) National Training Center (NTC) provides

technical training across all of BLM’s natural resource management program areas. 

Although calculating return on investment is complex and often costly, BLM wanted 

to answer the question “Are we getting our money’s worth?”

Challenge

Implement a process for estimating value of training investment.

Practices Illustrated

• Assess evaluation results in terms of business impact

• Validate content learning by testing
Appendix I

Information Technology: Federal IT Training

Page 68 GAO-04-791 Information Technology

62

Attachment 4 

Evaluating Training: Case Study

Solution

NTC estimates the value of training based on how much improvement is shown by 

pre- and post-testing.

• The “Value of Performance Before” is calculated by multiplying the average

percentage score on the pre-test by the average participant salary. (For 

example, if average salary is $50,000 and average pre-test score is 50 percent,

value before would be $25,000.) Similarly, “Value of Performance After” is 

calculated from the average post-test score and average salary. (If the post-test 

result is 70 percent, value after would be $35,000.)

• Value before is subtracted from value after to determine “Gain Per Participant” 

($10,000, in this case) and cost per participant is subtracted to determine net

value (if cost is $2,000, for example, net value would be $8,000).

• Net value is then divided by cost to facilitate comparisons among different

training courses.
Appendix I

Information Technology: Federal IT Training

Page 69 GAO-04-791 Information Technology

63

Attachment 4 

Evaluating Training: Case Study

Reported Results

NTC was able to document the value of its training.

Demonstrating the value of courses has helped justify the cost of technology, such 

as satellite broadcast, used to deliver training.

Courses with low calculated value were flagged for further discussion of their value

to the agency.

Note: This process is similar to one recommended by OPM in its publication, A

Guide to Strategically Planning Training and Measuring Results (July 2000), p. 44.
Page 70 GAO-04-791 Information Technology

Appendix II

Comments from the Office of Personnel 

Management ApendxiI
Appendix II

Comments from the Office of Personnel 

Management

Page 71 GAO-04-791 Information Technology
Page 72 GAO-04-791 Information Technology

Appendix III

Comments from the Department of Veterans 

Affairs ApendxiI
Appendix III

Comments from the Department of Veterans 

Affairs

Page 73 GAO-04-791 Information Technology
Page 74 GAO-04-791 Information Technology

Appendix IV

Comments from the Department of Health and 

Human Services ApendxiIV
Appendix IV

Comments from the Department of Health 

and Human Services

Page 75 GAO-04-791 Information Technology
Appendix IV

Comments from the Department of Health 

and Human Services

(310462) Page 76 GAO-04-791 Information Technology
GAO’s Mission The General Accounting Office, the audit, evaluation and investigative arm of 

Congress, exists to support Congress in meeting its constitutional responsibilities 

and to help improve the performance and accountability of the federal government 

for the American people. GAO examines the use of public funds; evaluates federal 

programs and policies; and provides analyses, recommendations, and other 

assistance to help Congress make informed oversight, policy, and funding 

decisions. GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

Obtaining Copies of 

GAO Reports and 

Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older 

products. The Web site features a search engine to help you locate documents 

using key words and phrases. You can print these documents in their entirety, 

including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site 

daily. The list contains links to the full-text document files. To have GAO e-mail this 

list to you every afternoon, go to www.gao.gov and select “Subscribe to 

e-mail alerts” under the “Order GAO Products” heading.

Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. A check 

or money order should be made out to the Superintendent of Documents. GAO 

also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single 

address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Public Affairs Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Service Requested

Presorted Standard

Postage & Fees Paid

GAO

Permit No. GI00