INFORMATION 

TECHNOLOGY 

DASHBOARD

Opportunities Exist to 

Improve Transparency 

and Oversight of 

Investment Risk at 

Select Agencies

Report to Congressional Requesters

October 2012

GAO-13-98

United States Government Accountability Office

GAO
United States Government Accountability Office

Highlights of GAO-13-98, a report to 

congressional requesters

October 2012

INFORMATION TECHNOLOGY DASHBOARD

Opportunities Exist to Improve Transparency and 

Oversight of Investment Risk at Select Agencies

Why GAO Did This Study

In June 2009, OMB launched the 

federal IT Dashboard, a public website 

that reports performance data for over

700 major IT investments that 

represent about $40 billion of the 

estimated $80 billion budgeted for IT in 

fiscal year 2012. The Dashboard is to 

provide transparency for these 

investments to aid public monitoring of 

government operations. It does so by 

reporting, among other things, how 

agency CIOs rate investment risk.

GAO was asked to (1) characterize the 

CIO ratings for selected federal 

agencies’ IT investments as reported 

over time on the Dashboard, (2) 

determine how agencies' approaches 

for assigning and updating CIO ratings 

vary, and (3) describe the benefits and 

challenges associated with agencies’ 

approaches to the CIO rating.

To do so, GAO selected six agencies 

spanning a range of 2011 IT spending 

levels and analyzed data reported for 

each of their investments on the 

Dashboard. GAO also interviewed 

agency officials and analyzed related 

documentation and written responses 

to questions about ratings and 

evaluation approaches, as well as 

agency views on the benefits and 

challenges related to the CIO rating.

What GAO Recommends

GAO is recommending that OMB 

analyze agencies’ investment risk over 

time as reflected in the Dashboard’s 

CIO ratings and present its analysis 

with the President’s annual budget

submission, and that DOD ensure that 

its CIO ratings reflect available 

investment performance assessments

and its risk management guidance. 

Both OMB and DOD concurred with 

our recommendations. 

What GAO Found

Chief Information Officers (CIO) at six federal agencies rated the majority of their 

information technology (IT) investments as low risk, and many ratings remained 

constant over time. Specifically, CIOs at the selected agencies rated a majority of 

investments listed on the federal IT Dashboard as low risk or moderately low risk

from June 2009 through March 2012; at five of these agencies, these risk levels 

accounted for at least 66 percent of investments. These agencies also rated no 

more than 12 percent of their investments as high or moderately high risk, and 

two agencies (Department of Defense (DOD) and the National Science 

Foundation (NSF)) rated no investments at these risk levels (see table). Over 

time, about 47 percent of the agencies’ Dashboard investments received the 

same rating in every rating period. For ratings that changed, the Department of 

Homeland Security (DHS) and Office of Personnel Management (OPM) reported

more investments with reduced risk when initial ratings were compared with 

those in March 2012; the other four agencies reported more investments with

increased risk. In the past, the Office of Management and Budget (OMB) 

reported trends for risky IT investments needing management attention as part of 

its annual budget submission, but discontinued this reporting in fiscal year 2010. 

Average Composition of CIO Ratings for Agencies’ Major IT Investments, June 2009 

through March 2012

Agency

DOD DHS HHS DOI OPM NSF

High risk and moderately high risk investments 0% 11% 5% 4% 12% 0%

Medium risk investments 15% 38% 21% 13% 22% 0%

Low risk and moderately low risk investments 85% 51% 74% 83% 66% 100%

Range in the number of investments during 

the period 49-87 63-83 55-81 33-48 6-9 3-5

Source: GAO analysis of data downloaded from OMB’s IT Dashboard.

Note: Table does not include downgraded or eliminated investments. HHS is the Department of 

Health and Human Services. DOI is the Department of the Interior.

Agencies generally followed OMB’s instructions for assigning CIO ratings, which 

included considering stakeholder input, updating ratings when new data become 

available, and applying OMB’s six evaluation factors. DOD’s ratings were unique 

in reflecting additional considerations, such as the likelihood of OMB review, and 

consequently DOD did not rate any of its investments as high risk. However, in 

selected cases, these ratings did not appropriately reflect significant cost, 

schedule, and performance issues reported by GAO and others. Moreover, DOD 

did not apply its own risk management guidance to the ratings, which reduces

their value for investment management and oversight. 

Various benefits were associated with producing and reporting CIO ratings. Most

agencies reported (1) increased quality of their performance data, (2) greater 

transparency and visibility of investments, and (3) increased focus on project 

management practices. Agencies also noted challenges, such as (1) the effort 

required to gather, validate, and gain internal approval for CIO ratings; and (2) 

obtaining information from OMB to execute required changes to the Dashboard.

OMB has taken steps to improve its communications with agencies.

View GAO-13-98. For more information, 

contact David A.Powner at (202) 512-9286 or 

pownerd@gao.gov.
Page i GAO-13-98 Information Technology Dashboard

Letter 1 

Background 3

CIOs Rated Most IT Investments as Low Risk or Moderately Low 

Risk; Agencies Had Mixed Results in Reducing Higher Risk 

Levels 14

Most Agencies Established Approaches for CIO Ratings That 

Follow OMB’s Instructions 20

CIO Ratings Present Benefits and Challenges for Agencies’ 

Investment Management 25

Conclusions 27

Recommendations for Executive Action 28

Agency Comments and Our Evaluation 28

Appendix I Objectives, Scope, and Methodology 30

Appendix II Risk Levels for Investments at Selected Agencies, as of March 

2012 33

Appendix III CIO Ratings and Net Changes for Major IT Investments at Selected 

Agencies, June 2009 through March 2012 34

Appendix IV Comments from the Department of Defense 47

Appendix V GAO Contacts and Staff Acknowledgments 49

Tables

Table 1: Dashboard Variance and Rating Colors 8

Table 2: Investment Evaluation Factors Identified by OMB for 

Assigning CIO Ratings 10

Table 3: IT Dashboard CIO Rating Colors, Based on a Five-Point 

Scale for CIO Ratings 11

Table 4: Average Composition of CIO Ratings for Agencies’ Major 

IT Investments from June 2009 through March 2012 15

Contents
Page ii GAO-13-98 Information Technology Dashboard

Table 5: Number of Major IT Investments with CIO Ratings That 

Changed Compared to Those That Remained Constant, 

Initial Rating through March 2012 16

Table 6: Net Changes in Investment Risk, Based on Comparison of 

Initial Rating to March 2012 18

Table 7: Data Sources and Derivation of CIO Ratings by Selected 

Agencies 21

Table 8: CIO Ratings and Budget Totals for Selected Agencies’ 

Major IT Investments as of March 2012 33

Figures

Figure 1: Example of an Agency Portfolio Page as Reported on 

OMB’s IT Dashboard Website, July 2012 6

Figure 2: Example of an Agency’s IT Investment Page from the IT 

Dashboard Website, July 2012 13

Figure 3: CIO Ratings and Associated IT Budgets for Fiscal Year 

2012 Dashboard Investments from Selected Agencies, as 

of March 2012 14

Figure 4: Percentages of Major IT Investments at Selected Agencies 

with CIO Ratings That Changed Compared to Ratings That 

Remained Constant, Initial Rating through March 2012 16

Figure 5: Percentages of Major IT Investments at Selected Agencies 

with Changes in CIO Ratings, Initial Rating Compared to 

March 2012 18

Figure 6: CIO Ratings for Major IT Investments at the Department 

of Defense, as Reported on the IT Dashboard from July 

2009 through March 2012 35

Figure 7: Changes to Risk Levels for Major IT Investments at the 

Department of Defense, Initial CIO Rating through March 

2012 36

Figure 8: CIO Ratings for Major IT Investments at the Department 

of Homeland Security, as Reported on the IT Dashboard 

from July 2009 through March 2012 37

Figure 9: Changes to Risk Levels for Major IT Investments at the 

Department of Homeland Security, Initial CIO Rating 

through March 2012 38

Figure 10: CIO Ratings for Major IT Investments at the Department 

of Health and Human Services, as Reported on the IT 

Dashboard from July 2009 through March 2012 39
Page iii GAO-13-98 Information Technology Dashboard

Figure 11: Changes to Risk Levels for Major IT Investments at the 

Department of Health and Human Services, Initial CIO 

Rating through March 2012 40

Figure 12: CIO Ratings for Major IT Investments at the Department 

of the Interior, as Reported on the IT Dashboard from July 

2009 through March 2012 41

Figure 13: Changes to Risk Levels for Major IT Investments at the 

Department of the Interior, Initial CIO Rating through 

March 2012 42

Figure 14: CIO Ratings for Major IT Investments at the National 

Science Foundation, as Reported on the IT Dashboard 

from June 2009 through March 2012 43

Figure 15: Changes to Risk Levels for Major IT Investments at the 

National Science Foundation, Initial CIO Rating through 

March 2012 44

Figure 16: CIO Ratings for Major IT Investments at the Office of 

Personnel Management, as Reported on the IT Dashboard 

from July 2009 through March 2012 45

Figure 17: Changes to Risk Levels for Major IT Investments at the 

Office of Personnel Management, Initial CIO Rating 

through March 2012 46
Page iv GAO-13-98 Information Technology Dashboard

Abbreviations

CIO chief information officer

DEAMS Defense Enterprise Accounting and Management 

 System

DOD Department of Defense 

DOI Department of the Interior

DHS Department of Homeland Security

GCSS-Army Global Combat Support System-Army

GFEBS General Fund Enterprise Business System

HHS Department of Health and Human Services

IT information technology

NSF National Science Foundation 

OMB Office of Management and Budget

OPM Office of Personnel Management

TechStat TechStat Accountability Sessions

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-13-98 Information Technology Dashboard

United States Government Accountability Office

Washington, DC 20548

October 16, 2012

The Honorable Thomas R. Carper

Chairman

The Honorable Scott P. Brown

Ranking Member

Subcommittee on Federal Financial Management, 

Government Information, Federal Services, 

 and International Security

Committee on Homeland Security 

 and Governmental Affairs

United States Senate

Spending on information technology (IT) represents a significant portion 

of the federal budget—estimated at $80 billion for fiscal year 2012.1 More 

than 700 major investments account for approximately $40 billion of this 

IT spending. The Clinger-Cohen Act of 1996 charges the Director of the 

Office of Management and Budget (OMB) with responsibility for 

analyzing, tracking, and evaluating the risks and results of all major IT 

investments as part of the federal budget process, and reporting to 

Congress on the performance benefits achieved by these investments.2

OMB launched the Federal IT Dashboard in June 2009 as a public 

website that reports performance and supporting data for the major IT 

investments. The Dashboard is to provide transparency for these 

investments in order to facilitate public monitoring of government 

operations and accountability for investment performance by the federal 

CIOs who oversee them. In January 2010, OMB began using the 

Dashboard as one of several tools to identify troubled investments. These 

investments became the focus of joint OMB-agency TechStat 

Accountability Sessions (TechStats)—evidence-based reviews intended 

to improve investment performance through concrete actions. In 

December 2010, OMB reported that these sessions resulted in $3 billion 

The act also places responsibility for managing investments with the 

heads of agencies and establishes chief information officers (CIOs) to 

advise and assist agency heads in carrying out this responsibility.

 

1

As reported by agencies to the Office of Management and Budget.

2

40 U.S.C. § 11302(c). 

 
 

Page 2 GAO-13-98 Information Technology Dashboard

in reduced life-cycle costs and subsequently incorporated the TechStat 

model into its 25-point plan for reforming federal IT management.3 With 

this plan, agency CIOs became responsible for leading TechStat sessions 

at the department level, analyzing investments using data from the 

Dashboard, and terminating or turning around at least one-third of 

underperforming IT projects within 18 months. OMB reported its progress 

on the plan, improvements to the Dashboard, and results of TechStat 

sessions in the analytical perspectives it provided for the President’s 2012 

and 2013 budget submissions.4

In response to your request, our objectives for this review were to (1) 

characterize the CIO ratings for selected federal agencies’ IT investments 

as reported over time on the Dashboard, (2) determine how agencies’ 

approaches for assigning and updating CIO ratings vary, and (3) describe 

the benefits and challenges associated with agencies’ approaches to the 

CIO rating.

To establish the scope of our review, we selected six agencies that 

spanned a range of IT spending for fiscal year 2011, including the three 

highest spending agencies, two of the lowest, and an agency in the 

middle. Collectively, these agencies accounted for approximately $51 

billion, or 65 percent, of 2011 spending on IT investments. The six 

agencies are the Department of Defense (DOD), Department of 

Homeland Security (DHS), Department of Health and Human Services 

(HHS), Department of the Interior (DOI), National Science Foundation 

(NSF), and Office of Personnel Management (OPM).

To address our objectives, we downloaded CIO ratings and related data 

reported for investments on the Dashboard and analyzed these data for 

the period June 2009 to March 2012.5

 

3

U.S. Chief Information Officer, 25 Point Implementation Plan to Reform Federal 

Information Technology Management, The White House (Washington, D.C.: Dec. 9, 

2010).

We did not independently evaluate 

the ratings as reported by the agencies, but determined that they were 

sufficiently complete and accurate for our analyses. We interviewed 

agency officials, including CIOs where possible, and obtained written 

4

Office of Management and Budget, Budget of the United States Government, Analytical 

Perspectives. Fiscal Years 2012, 2013.

5

http://www.itdashboard.gov.
 

Page 3 GAO-13-98 Information Technology Dashboard

responses and supporting documents, related agency policies, 

procedures, reported data, artifacts, as well as agency views on the 

benefits and challenges associated with performing these ratings and 

reporting them to the Dashboard. We also utilized recent GAO and DOD 

Inspector General reviews of DOD’s major IT investments and compared 

findings in these reports to the CIO ratings that the department submitted 

to the Dashboard. In addition, we analyzed OMB documentation and 

interviewed OMB staff to update our information on how the Dashboard 

has evolved, identify the guidance agencies received about CIO ratings, 

determine the efforts OMB has under way to improve the Dashboard, and 

describe the ways in which OMB is using the data to improve IT 

management.

We conducted this performance audit from January 2012 through 

September 2012 in accordance with generally accepted government 

auditing standards. Those standards require that we plan and perform the 

audit to obtain sufficient, appropriate evidence to provide a reasonable 

basis for our findings and conclusions based on our audit objectives. We 

believe that the evidence obtained provides a reasonable basis for our 

findings and conclusions based on our audit objectives. Further details of 

our objectives, scope, and methodology are provided in appendix I.

The Clinger-Cohen Act of 1996 requires OMB to establish processes to 

analyze, track, and evaluate the risks and results of major capital 

investments in information systems made by federal agencies and report 

to Congress on the net program performance benefits achieved as a 

result of these IT investments.6

OMB established the Management Watch List in 2003 to help carry out its 

oversight role. The Management Watch List included mission-critical 

projects that needed to improve performance measures, project 

management, IT security, or overall justification for inclusion in the 

President’s budget submission. Further, in August 2005, OMB 

established a High-Risk List, which consisted of projects identified by 

federal agencies, with OMB’s input, as requiring special attention from 

Further, the act places responsibility for 

managing investments with the heads of agencies and establishes CIOs 

to advise and assist agency heads in carrying out this responsibility.

 

6

40 U.S.C. § 11302(c).

Background 
 

Page 4 GAO-13-98 Information Technology Dashboard

oversight authorities and the highest levels of agency management. 

Between 2005 and 2009, OMB described its efforts to monitor and 

manage risky federal IT investments in the annual budget submission.

Over the past several years, we have reported and testified on OMB’s 

initiatives to highlight troubled IT projects, justify investments, and use 

project management tools.7 For instance, in 2006 we recommended that 

OMB develop a single aggregated list of high-risk projects and their 

deficiencies and use that list to report to Congress on progress made in 

correcting high-risk problems.8 As a result, OMB started publicly releasing 

aggregate data on its Management Watch List and disclosing the 

projects’ deficiencies. Moreover, between 2007 and 2009, the President’s 

budget submission included an overview of investment performance over 

several budget years, including the number of federal IT projects in need 

of management attention. Such information helped Congress stay better 

informed of high-risk projects and make related funding decisions.9 With 

the advent of its IT Dashboard in 2009, OMB discontinued this type of 

reporting in the fiscal year 2010 budget submission.

In June 2009, OMB deployed a public website to further improve the 

transparency and oversight of agencies’ IT investments, replacing the 

Management Watch List and High-Risk List. Known as the IT 

Dashboard,10

 

7

this site displays federal agencies’ cost, schedule, and 

performance data for over 700 major federal IT investments at 27 federal 

agencies that are responsible for about $40 billion of the federal budget. 

According to OMB, these data are intended to provide a near-real-time 

perspective on the performance of these investments, as well as a 

historical perspective. Further, the public display of these data is intended 

GAO-09-624T; GAO, Information Technology: Treasury Needs to Better Define and 

Implement Its Earned Value Management Policy, GAO-08-951 (Washington, D.C.: Sept. 

22, 2008); and Air Traffic Control: FAA Uses Earned Value Techniques to Help Manage 

Information Technology Acquisitions, but Needs to Clarify Policy and Strengthen 

Oversight, GAO-08-756 (Washington, D.C.: July 18, 2008).

8

GAO, Information Technology: Agencies and OMB Should Strengthen Processes for 

Identifying and Overseeing High Risk Projects, GAO-06-647 (Washington, D.C.: June 15, 

2006).

9

Office of Management and Budget, Budget of the United States Government, Analytical 

Perspectives. Fiscal Years 2005, 2006, 2007, 2008, 2009.

10http://www.itdashboard.gov. 

OMB’s Dashboard 

Provides Visibility into the 

Performance of Federal IT 

Investments 
 

Page 5 GAO-13-98 Information Technology Dashboard

to allow OMB; other oversight bodies, including Congress; and the 

general public to hold the government agencies accountable for progress 

and results.

OMB reported on plans and implementation progress for this 

management tool in the “Analytical Perspectives” section of the 

President’s budget submissions for fiscal years 2012 and 2013,11

The Dashboard visually presents performance ratings for agencies overall 

and for individual investments using metrics that OMB has defined—cost, 

schedule, and CIO evaluation. The website also provides the capability to 

download certain data. Figure 1 is an example of an agency’s (OPM) 

portfolio page as recently depicted on the Dashboard.

including planned updates to the Dashboard during 2012 to support 

closer executive oversight and intervention to prevent schedule delays, 

cost overruns, and failures in delivering key functionality needed by 

federal programs. For example, it reported using the Dashboard to 

identify investments for TechStat reviews.

 

11Office of Management and Budget, Budget of the United States Government, Analytical 

Perspectives. Fiscal Years 2012, 2013.
 

Page 6 GAO-13-98 Information Technology Dashboard

Figure 1: Example of an Agency Portfolio Page as Reported on OMB’s IT Dashboard Website, July 2012

The Dashboard’s data spans the period from June 2009 to the present, 

and is based, in part, on each agency’s exhibit 53 and exhibit 300 
 

Page 7 GAO-13-98 Information Technology Dashboard

submissions12

• improvements to Dashboard calculations to incorporate the variance 

of “in progress” milestones rather than just “completed” milestones;

to OMB, as well as on agency assessments and 

supporting information on each investment. Over the life of the 

Dashboard, OMB has issued guidance to agencies on, among other 

things, what data to report, how those data need to be structured and 

formatted for upload to the Dashboard, and procedures for using the 

Dashboard’s submission tools. For instance, OMB instructed agencies to 

update and submit investment cost and schedule data monthly. OMB has

made various changes to the organization, available data, and features of 

the Dashboard over time, including

• web pages containing data on historical ratings and rebaselines of 

eliminated and downgraded investments;

• added data on awarded contracts, with links to USAspending.gov;

• release of IT Dashboard source code and documentation to an open 

source hosting provider;

• enhancements to baseline history, which give users the ability to see 

field-by-field changes for each rebaseline;

• a mechanism for OMB analysts to provide feedback to agencies on 

investment submissions; and

• mobile-friendly formatting of Dashboard displays.

Once OMB has received agency-reported investment data, it converts 

these into investment performance ratings for display on the dashboard 

according to calculations and protocols described on its website. OMB 

assigns cost and schedule performance ratings by using data submitted 

by agencies to calculate variances between the planned cost or schedule 

targets and the actual or projected cost or schedule values. OMB 

converts these variances to percentages, and assigns the ratings to be 

 

12Exhibit 53s list all of the IT projects and their associated costs within a federal 

organization. An exhibit 300 is also called the Capital Asset Plan and Business Case. It is 

used to justify resource requests for major IT investments and is intended to enable an 

agency to demonstrate to its own management, as well as to OMB, that a major project is 

well planned.
 

Page 8 GAO-13-98 Information Technology Dashboard

presented on the Dashboard within three ranges, red, yellow, and green, 

as shown in table 1.

Table 1: Dashboard Variance and Rating Colors

Project level cost and schedule variance rating Rating color

≥ 30% Red

≥ 10% and < 30% Yellow

< 10% Green

Source: OMB’s IT Dashboard.

Although the thresholds for assigning cost and schedule variance ratings 

has remained constant over the life of the Dashboard, the cost and 

schedule data agencies are required to submit have changed in several 

ways, as have the variance calculations. For example, in response to our 

recommendations (further discussed in the next section), OMB changed 

how the Dashboard calculates the cost and schedule ratings in July 2010, 

to include “in progress” milestones rather than just “completed” ones for 

more accurate reflection of current investment status.

We have previously reported that OMB has taken significant steps to 

enhance the oversight, transparency, and accountability of federal IT 

investments by creating its IT Dashboard, and by improving the accuracy 

of investment ratings. We also found issues with the accuracy and data 

reliability of cost and schedule data, and recommended steps that OMB 

should take to improve these data.

In July 2010, we reported13

 

13

that the cost and schedule ratings on OMB’s 

Dashboard were not always accurate for the investments we reviewed, 

because these ratings did not take into consideration current 

performance. As a result, the ratings were based on outdated information. 

We recommended that OMB report on its planned changes to the 

Dashboard to improve the accuracy of performance information and 

provide guidance to agencies to standardize milestone reporting. OMB 

agreed with our recommendations and, as a result, updated the 

Dashboard’s cost and schedule calculations to include both ongoing and 

GAO-10-701. 

GAO Has Previously 

Reported on the 

Dashboard’s Value, Data 

Quality, and Recent 

Improvements
 

Page 9 GAO-13-98 Information Technology Dashboard

completed activities. Similarly, in March 2011, we reported14

More recently, in November 2011, we reported

that OMB 

had initiated several efforts to increase the Dashboard’s value as an 

oversight tool, and had used its data to improve federal IT management. 

We also reported, however, that agency practices and the Dashboard’s 

calculations contributed to inaccuracies in the reported investment 

performance data. For instance, we found missing data submissions or 

erroneous data at each of the five agencies we reviewed, along with 

instances of inconsistent program baselines and unreliable source data. 

As a result, we recommended that the agencies take steps to improve the 

accuracy and reliability of their Dashboard information, and that OMB 

improve how it rates investments relative to current performance and 

schedule variance. Most agencies generally concurred with our 

recommendations; OMB agreed with our recommendation for improving 

ratings for schedule variance. It disagreed with our recommendation to 

improve how it reflects current performance in cost and schedule ratings, 

but more recently made changes to Dashboard calculations to address 

this while also noting challenges in comprehensively evaluating cost and 

schedule data for these investments.

15 that the accuracy of 

investment cost and schedule ratings had improved since our July 2010 

report because OMB had refined the Dashboard’s cost and schedule 

calculations. Most of the ratings for the eight investments we reviewed 

were accurate, although we noted that more could be done to inform 

oversight and decision making by emphasizing recent performance in the 

ratings. We recommended that the General Services Administration 

comply with OMB’s guidance for updating its ratings when new 

information becomes available (including when investments are 

rebaselined) and the agency concurred. Since we previously 

recommended that OMB improve how it rates investments, we did not 

make any further recommendations.

Unlike the Dashboard’s cost and schedule ratings, which are derived by 

OMB based on agency-submitted data, the “Investment Evaluation by 

Agency CIO” (also called the CIO rating) is determined by agency 

officials; OMB translates the agency’s numerical assignment for an 

 

14GAO-11-262. 

15GAO-12-210.

CIO Ratings Are Important 

for OMB’s IT Reform and 

Management Initiatives
 

Page 10 GAO-13-98 Information Technology Dashboard

investment into a color for depiction on the Dashboard. An OMB staff 

member from the Office of E-Government and Information Technology 

noted that the CIO rating should be a current assessment of future 

performance based on historical results and is the only Dashboard 

performance indicator that has been defined and produced the same way 

since the Dashboard’s inception. According to OMB’s instructions, a CIO 

rating should reflect the level of risk facing an investment on a scale from 

1 (high risk) to 5 (low risk) relative to that investment’s ability to 

accomplish its goals. Each agency CIO is to assess their IT investments 

against a set of six preestablished evaluation factors identified by OMB 

(shown in table 2) and then assign a rating of 1 to 5 based on his or her 

best judgment of the level of risk facing the investment. According to an 

OMB staff member, agency CIOs are responsible for determining 

appropriate thresholds for the risk levels and for applying them to 

investments when assigning CIO ratings.

Table 2: Investment Evaluation Factors Identified by OMB for Assigning CIO 

Ratings 

Evaluation factor Supporting examples

Risk management Risk management strategy exists

Risks are well understood by senior leadership

Risk log is current and complete

Risks are clearly prioritized

Mitigation plans are in place to address risks 

Requirements 

management 

Investment objectives are clear and scope is controlled

Requirements are complete, clear and validated

Appropriate stakeholders are involved in requirements definition 

Contractor 

oversight 

Acquisition strategy is defined and managed via an Integrated 

Program Team

Agency receives key reports, such as earned value reports, current 

status, and risk logs

Agency is providing appropriate management of contractors such 

that the government is monitoring, controlling, and mitigating the 

impact of any adverse contract performance 

Historical 

performance 

No significant deviations from planned cost and schedule

Lessons learned and best practices are incorporated and adopted 

Human capital Qualified management and execution team for the IT investments 

and/or contracts supporting the investment

Low turnover rate 

Other Other factors that the CIO deems important to forecasting future 

success 

Source: OMB’s IT Dashboard.
 

Page 11 GAO-13-98 Information Technology Dashboard

OMB recommends that CIOs consult with appropriate stakeholders in 

making their evaluation, including Chief Acquisition Officers, program 

managers, and other interested parties. Ultimately, CIO ratings are 

assigned colors for presentation on the Dashboard, according to the fivepoint rating scale, as illustrated in table 3.

Table 3: IT Dashboard CIO Rating Colors, Based on a Five-Point Scale for CIO 

Ratings

Rating (by agency CIO) Color 

5-Low risk Green 

4-Moderately low risk Green 

3-Medium risk Yellow 

2-Moderately high risk Red 

1-High risk Red 

Source: OMB’s IT Dashboard.

OMB has made the CIO’s evaluation and rating a key component of its 

larger IT Reform Initiative and 25 Point Plan. In its plan, OMB reported 

that it used agencies’ CIO ratings to select investments for the TechStat 

review sessions it conducted between 2010 and 2011. These sessions 

are data-driven assessments of IT investments by agency leaders that 

are intended to result in concrete action to improve performance. OMB 

reported that the TechStats it conducted on selected investments resulted 

in approximately $3 billion in reduced costs. Building on the results of 

those sessions, the plan articulates a strategy for strengthening IT 

governance, in part, through the adoption of the TechStat model by 

federal agencies. In conducting TechStats, agencies are to rely, in part, 

on CIO ratings from the IT Dashboard. The TechStat Toolkit, developed 

by OMB and a task force of agency leads, provides sample questions 

regarding an investment’s CIO rating and associated risks for use in 

TechStat sessions.

Furthermore, OMB issued guidance in August 201116

 

16Chief Information Officer Authorities, Memorandum for Heads of Executive Departments 

and Agencies, M-11-29 (Washington, D.C.: Aug. 8, 2011).

that stated, among 

other things, that agency CIOs shall be held accountable for the 

performance of IT program managers based on their governance process 

and the data reported on the IT Dashboard, which includes the CIO 
 

Page 12 GAO-13-98 Information Technology Dashboard

rating. According to OMB, the addition of CIO names and photos on 

Dashboard investments is intended to highlight this accountability and link 

it to the Dashboard’s reporting on investment performance. Figure 2 

illustrates the CIO rating information presented on the Dashboard for an 

example IT investment.
 

Page 13 GAO-13-98 Information Technology Dashboard

Figure 2: Example of an Agency’s IT Investment Page from the IT Dashboard Website, July 2012
 

Page 14 GAO-13-98 Information Technology Dashboard

As of March 2012, CIO ratings for most investments listed on the 

Dashboard for the six agencies we reviewed indicated either low risk or 

moderately low risk (223 out of 313 investments across all the selected 

agencies). High risk or moderately high risk ratings were assigned to 

fewer investments (12 out of 313 investments across all the selected 

agencies). Figure 3 presents the total number of IT investments rated on 

the Dashboard for each of the selected agencies according to their risk 

levels, as of March 2012, and illustrates the predominance of low risk 

investments for the agencies in our review. The figure also reports 

agencies’ budgets for their major IT investments for fiscal year 2012, as 

presented on the Dashboard.

Figure 3: CIO Ratings and Associated IT Budgets for Fiscal Year 2012 Dashboard 

Investments from Selected Agencies, as of March 2012

Note: Figure does not include downgraded or eliminated investments. Budget figures represent 

spending on major IT investments for fiscal year 2012.

Historically, over the life of the Dashboard from June 2009 to March 2012, 

low or moderately low risk ratings accounted for at least 66 percent of all 

ratings at five of the six agencies (the exception is DHS with 51 percent). 

CIOs Rated Most IT 

Investments as Low 

Risk or Moderately 

Low Risk; Agencies 

Had Mixed Results in 

Reducing Higher Risk 

Levels 
 

Page 15 GAO-13-98 Information Technology Dashboard

Medium risk ratings accounted for between 0 to 38 percent of all reported 

ratings across agencies during this period. The maximum percentage of 

ratings in the high risk or moderately high risk categories for any agency 

during this 34-month period was 12 percent, with two agencies—DOD 

and NSF—reporting no high risk investments.17

Table 4: Average Composition of CIO Ratings for Agencies’ Major IT Investments from June 2009 through March 2012

DOD stated in written 

comments that this was because they did not deem any of their 

investments to be high risk. (DOD’s investment risks are further 

discussed in the next section.) An NSF official from the Division of 

Information Systems stated that there were no high risk investments 

because most of their investments were in the operations and 

maintenance phase. Table 4 presents the average composition of ratings 

for each agency during the reporting period of June 2009 to March 2012. 

Appendix III depicts each agency’s CIO ratings by risk level on a monthly 

basis during the reporting period.

DOD DHS HHS DOI OPM NSF

High risk and moderately high risk investments 0% 11% 5% 4% 12% 0%

Medium risk investments 15% 38% 21% 13% 22% 0%

Low risk and moderately low risk investments 85% 51% 74% 83% 66% 100%

Range in the number of major IT investments rated on the 

Dashboard during this period 49-87 63-83 55-81 33-48 6-9 3-5 

Source: GAO analysis of data from OMB’s IT Dashboard.

Note: Table does not include downgraded or eliminated investments.

Overall, the CIO rating remained constant for 147 of 313 investments that 

were active as of March 2012 (about 47 percent of the investments we 

reviewed). These investments were rated at the same risk level during 

every rating period (see fig. 4).

 

17CIO ratings for investments that were downgraded (no longer defined as a major 

investment) or eliminated during this period are not included in these percentages.
 

Page 16 GAO-13-98 Information Technology Dashboard

Figure 4: Percentages of Major IT Investments at Selected Agencies with CIO 

Ratings That Changed Compared to Ratings That Remained Constant, Initial Rating 

through March 2012

Note: Figure does not include downgraded or eliminated investments.

Four of the six agencies did not change the CIO rating for a majority of 

their investments (excluding any investments that were downgraded or 

eliminated) during the time frame we examined. In contrast, the other two 

agencies—OPM and HHS—changed the CIO rating for more than 70 

percent of their investments at least once between the investment’s initial 

rating and the rating reported as of March 2012. Table 5 lists the number 

of each agency’s investments whose ratings were constant and changed 

over time.

Table 5: Number of Major IT Investments with CIO Ratings That Changed Compared 

to Those That Remained Constant, Initial Rating through March 2012

DOD DHS HHS DOI OPM NSF

Remained constant 51 42 17 31 2 4 

Changed 36 41 64 17 7 1 

Total 87 83 81 48 9 5 

Source: GAO analysis of data from OMB’s IT Dashboard.

Note: Table does not include downgraded or eliminated investments.
 

Page 17 GAO-13-98 Information Technology Dashboard

Agencies offered several reasons for why many investments had no 

changes in their CIO ratings during their entire time on the Dashboard. 

Five of the six selected agencies indicated that many investments were in 

a steady-state or operations and maintenance phase with no new 

development. One agency reported that their investments’ CIO ratings 

remained constant because the investments consistently met all 

requirements and deadlines and were using project management best 

practices.

The agencies we reviewed showed mixed results in reducing the number 

of higher risk investments during the rating period.18

 

18A reduction in risk level is indicated by a higher CIO rating. An increase in risk level is 

indicated by a lower CIO rating.

For investments 

whose rating changed at least once during the period, 40 percent (67 

investments) received a lower risk rating in March 2012 than they 

received initially, 41 percent of investments (68 investments) received a 

higher risk rating, and the remaining 19 percent (31 investments) received 

the same rating in March 2012 as they had initially received, despite 

whatever interim changes may have occurred (i.e., there was no “net” 

change to their reported risk levels). (See fig. 5.)
 

Page 18 GAO-13-98 Information Technology Dashboard

Figure 5: Percentages of Major IT Investments at Selected Agencies with Changes 

in CIO Ratings, Initial Rating Compared to March 2012

Note: Figure does not include downgraded or eliminated investments. Percentage represents initial 

CIO rating compared to the CIO rating as of March 2012 and does not represent any fluctuations that 

may have occurred in between that time frame.

Two agencies—DHS and OPM—reported more investments with reduced 

risk in March 2012, as compared with initial ratings. The other four 

agencies reported more investments with increased risk. Table 6 presents 

net changes in risk levels at each of the selected agencies (among 

investments that were not downgraded or eliminated). Appendix III 

graphically summarizes these data for all six agencies.

Table 6: Net Changes in Investment Risk, Based on Comparison of Initial Rating to 

March 2012 

DOD DHS HHS DOI OPM NSF

Risk level reduced 15 26 17 4 5 0 

Risk level increased 18 12 25 11 1 1 

No net change 3 3 22 2 1 0 

Total number of changed investments 36 41 64 17 7 1

Source: GAO analysis of data downloaded from OMB’s IT Dashboard.

Note: Table does not include downgraded or eliminated investments. Table represents investments’ 

initial CIO rating compared to the CIO rating as of March 2012 and does not represent any 

fluctuations that may have occurred in between that time frame.
 

Page 19 GAO-13-98 Information Technology Dashboard

Agencies most commonly cited additional oversight or program reviews 

as factors that contributed to decreased risk levels. Specifically, agencies 

commented that the CIO ratings and Dashboard reporting had spurred 

improved program management and risk mitigation. For example, one 

agency’s officials commented that the CIO now closely monitors the 

monthly performance and risk data generated by their investments, and 

that the additional oversight has brought about strengthened processes 

and more focused attention to issues.

In contrast, several agencies cited generally poor risk management at the 

investment level, the introduction of new investment/programs risks, as 

well as instances of poor project management as factors contributing to 

increased risk for investments. For example, one agency responded that 

internal review findings revealed new risks that caused an investment’s 

risk level to increase. Another agency’s officials reported that various 

technical issues caused one of their investments to fall behind schedule, 

thus increasing risk. 

Both OMB and several agencies suggested caution in interpreting 

changing risk levels for investments. They noted that an increase in an 

investment’s risk level can sometimes indicate better management by the 

program or CIO because previously unidentified risks have been 

assessed and included in the CIO evaluation. Conversely, a decrease in 

an investment’s risk level may not indicate improved management if the 

data and analysis on which the CIO rating are based is incomplete, 

inconsistent, or outdated.

Further analysis of the characteristics and causes of Dashboard’s CIO 

ratings, and reporting on the patterns of risk within and among agencies, 

could provide Congress and the public with additional perspectives on 

federal IT investment risk over time. However, for the past four budget 

submissions, OMB has not summarized the extent of risk represented by 

major federal IT investments in the analysis it prepares annually for the 

President’s budget submission, as it did prior to the fiscal year 2010 

submission. As a result, OMB is missing an opportunity to integrate such 

risk assessments into its evaluation of major capital investments in 

reporting to Congress.
 

Page 20 GAO-13-98 Information Technology Dashboard 

OMB has provided agencies with instructions for assigning CIO ratings for 

the major IT investments reported on the Dashboard. Specifically, OMB’s 

instructions state that agency CIOs should rate each investment based on 

his/her best judgment and should 

 include input from stakeholders, such as Chief Acquisition Officers, 

program managers, and others; 

 update the rating as soon as new information becomes available that 

might affect the assessment of a given investment; and 

 utilize OMB’s investment rating factors, including: risk management, 

requirements management, contractor oversight, historical 

performance, and human capital, as well as any other factors deemed 

relevant by the CIO. 

Despite differences in the specific inputs and processes used, agencies 

generally followed OMB’s instructions for assigning CIO ratings. However, 

DOD’s ratings reflected additional considerations beyond OMB’s 

instructions and did not reflect available information about significant risks 

for certain investments. The sections that follow describe how each 

agency addressed OMB’s instructions. 

Include input from stakeholders. Each of the six agencies we reviewed 

relied on stakeholder input, at least in part, when assigning CIO ratings. 

Agencies also cited a variety of review boards, data from program and 

financial systems, and other investment assessments as inputs to the 

rating. Table 7 describes the data and processes that agencies reported 

using when they derived their CIO ratings. 

Most Agencies 

Established 

Approaches for CIO 

Ratings That Follow 

OMB’s Instructions 
 

Page 21 GAO-13-98 Information Technology Dashboard

Table 7: Data Sources and Derivation of CIO Ratings by Selected Agencies

Agency Data sources used in CIO rating How rating is derived

DOD Input and recommendations from a variety of 

stakeholders, including functional leads, acquisition 

community, component CIOs, and cost and schedule 

experts 

The DOD CIO reviews investment performance information and 

recommendations from stakeholders and makes a subjective 

evaluation of the investment based upon those inputs. 

DHS Data from internal investment management systems 

and the department’s periodic reporting system, 

program management reviews, CIO portfolio reviews, 

and input from the executive steering committees

The DHS CIO determines the rating based upon information 

and recommendation gathered by an analysis team from the 

department’s Enterprise Business Management Office. 

HHS Cost and schedule variance data, exhibit 300 Quality 

Review, number of rebaselines, and investment 

manager self assessment

The HHS CIO determines the CIO ratings using a score 

calculated from the data.

DOI Bureau-proposed rating of investment, Electronic 

Capital Planning Investment Control Technologya data, 

iStatb

One of the processes the agency follows begins with the bureau 

responsible for the investment completing a template to 

document its proposed bureau rating of the investment and 

submits the proposed rating to the department CIO through the 

Capital Planning and Investment Control office. The office 

reviews and analyzes the data, compares the data against 

known information, such as iStat performance reviews or 

applicable GAO or Office of Inspector General findings 

regarding the current state of the investment, and provides a 

recommendation to the CIO, who determines the rating. 

Additionally, the CIO can make the decision to change the 

rating outside of this process.

performance reviews, applicable GAO or Office of 

Inspector General findings

NSF IT portfolio management tool; program manager and 

stakeholder input from monthly reviews where budget, 

schedule, and risks are reviewed

NSF staff meets monthly with the CIO to discuss investment 

data and review evaluations using OMB’s guidance. The staff 

assesses and ranks the investments. The CIO issues the rating 

based upon this information.

OPM IT security rating; enterprise architecture rating; earned 

value management program manager rating; risk 

management rating; Electronic Capital Planning 

Investment Control Technology; cost, schedule, and 

overall variance rating, IT Dashboard cost and schedule 

variance; and program managers’ self-assessments 

The IT Investment Management group reports to the CIO on 

collected information regarding the investments. The group 

produces a chart that tracks 20 different metrics; each metric is 

presented as either a red, yellow, or green rating. The CIO 

issues the rating based upon this information. 

Source: GAO analysis of agency-reported data.

a

Electronic Capital Planning Investment Control Technology is a government-owned technology 

system that is designed to help federal agencies in the management and control of their initiatives, 

portfolios, and investment priorities. This web-based application assists managers and staff involved 

in IT planning in assessing IT investments in terms of their costs, risks, and expected returns.

b

Update CIO ratings. All six agencies established guidelines for 

periodically reviewing and updating their CIO ratings. Specifically, HHS, 

NSF, DOI, and OPM reported that they update CIO ratings on a monthly 

basis. DOD has adopted a quarterly update cycle, although an official 

noted that the actual process of collecting information and evaluating 

investments for the ratings takes slightly longer than 3 months. DHS 

iStat is DOI’s internal investment review process, which was modeled after OMB’s TechStat process.
 

Page 22 GAO-13-98 Information Technology Dashboard

officials with the Office of the CIO stated that the frequency of its updates 

varies based on the risk level of an investment’s previous rating: 

investments with a previous CIO rating of green are to be reviewed 

semiannually; yellow investments are to be reviewed quarterly; and red 

investments are to be reviewed monthly.

Utilize OMB’s investment rating factors. Most of the selected agencies 

use OMB’s investment rating factors when evaluating their investments. 

Only one agency (HHS) does not use all of them. Specifically, an HHS 

official from the Office of the CIO told us that human capital issues are not 

explicitly covered in their CIO rating criteria because investment owners 

are to provide adequate IT human capital, and that these owners will 

reflect any issues that arise when providing input for the CIO rating.

Among the agencies we reviewed, DOD was unique in that its ratings 

reflected additional considerations beyond OMB’s instructions. For 

example, briefing slides prepared for DOD’s 2011 CIO rating exercise 

identified the need to “balance” CIO ratings, and advised that yellow or 

red ratings could lead to an OMB review.19

Although the CIO ratings submitted by DOD to the Dashboard are 

consistent with their ratings approach, they do not reflect other available 

information about the risk of these investments. As we previously noted, 

none of DOD’s investments that were active in March 2012 were rated as 

high risk, and approximately 85 percent were rated as either low risk or 

moderately low risk throughout their time on the Dashboard. However, 

these ratings did not always reflect significant schedule delays, cost 

increases, and other weaknesses identified for certain investments in our 

In addition, DOD officials 

explained that the department rated investments green (or low risk) if the 

risk of the investment not meeting its performance goals is low; yellow (or 

medium risk) if the investment is facing difficulty; and red (high risk) only if 

the department planned to restructure or cancel the investment, or had 

already done so. DOD officials further stated that their CIO ratings 

provide a measured assessment of how DOD believes an investment will 

perform in the future.

 

19Office of the DOD CIO, “Summary of June 2011 Recommended Rating Changes” 

(including Backup Slides), June 2011 Update. 
 

Page 23 GAO-13-98 Information Technology Dashboard

recent reviews, or problems with those investments identified in a recent 

report by the DOD Inspector General.20

Based on the department’s long-standing difficulties with such programs, 

we designated DOD business systems modernization as a high-risk area 

in 1995 and it remains a high-risk area today. More recently, we reported 

weaknesses in several of the department’s business system 

investments.21

Although the following selected examples of DOD investments 

experienced significant performance problems and were included with 

those considered to be high-risk business system investments in our 

recent reviews of those systems, they were all rated low risk or 

moderately low risk by the DOD CIO.

Specifically, we reported that the department had not 

effectively ensured that these systems would deliver capabilities on time 

and within budget; that acquisition delays required extended funding for 

duplicative legacy systems; that delays and cost overruns were likely to 

erode the cost savings these systems were to provide; and that, 

ultimately, DOD’s management of these investments was putting the 

department’s transformation of business operations at risk.

• Air Force’s Defense Enterprise Accounting and Management 

System (DEAMS): DEAMS is the Air Force’s target accounting 

system designed to provide accurate, reliable, and timely financial 

information. In early 2012, GAO reported that DEAMS faced a 2-year 

deployment delay, an estimated cost increase of about $500 million 

for an original life-cycle cost estimate of $1.1 billion (an increase of 

approximately 45 percent), and that assessments by DOD users had 

identified operational problems with the system, such as data 

accuracy issues, an inability to generate auditable financial reports, 

 

20Department of Defense Office of Inspector General, Enterprise Resource Planning 

Systems Schedule Delays and Reengineering Weaknesses Increase Risks to DOD’s 

Auditability Goals. DODIG-2012-111 (Alexandria, Va.: July 13, 2012).

21GAO, DOD Business Transformation: Improved Management Oversight of Business 

System Modernization Efforts Needed, GAO-11-53 (Washington, D.C.: Oct. 7, 2010) and 

DOD Financial Management: Reported Status of Department of Defense’s Enterprise 

Resource Planning Systems, GAO-12-565R (Washington, D.C: Mar. 30, 2012).
 

Page 24 GAO-13-98 Information Technology Dashboard

and the need for manual workarounds.22

• Army’s General Fund Enterprise Business System (GFEBS):

GFEBS is an Army financial management system intended to improve 

the timeliness and reliability of financial information and to support the 

department’s auditability goals. In early 2012, we reported that 

GFEBS faced a 10-month implementation delay, and that DOD users 

reported operational problems, including deficiencies in data accuracy 

and an inability to generate auditable financial reports.

In July 2012, the DOD 

Inspector General reported that the DEAMS’ schedule delays were 

likely to diminish the cost savings it was to provide, and would 

jeopardize the department’s goals for attaining an auditable financial 

statement. DOD’s CIO rated DEAMS low risk or moderately low risk 

from July 2009 through March 2012.

23

• Army’s Global Combat Support System-Army (GCSS-Army):

GCSS-Army is intended to improve the Army’s supply chain 

management capabilities and provide accurate equipment readiness 

status reports, among other things. In March 2012, we reported that 

GCSS-Army was experiencing a cost overrun of approximately $300 

million on an original life-cycle cost estimate of $3.9 billion (an 

increase of approximately 8 percent) and a deployment delay of 

approximately 2 years.

These 

concerns were reiterated by the DOD Inspector General in July 2012. 

DOD’s CIO rated GFEBS as moderately low risk from July 2009 

through March 2012.

24

Explanations submitted by DOD with the CIO ratings for these 

investments did not provide meaningful insight for why they were rated at 

DOD rated GCSS-Army as low or 

moderately low risk from July 2009 through March 2012.

 

22GAO, DOD Financial Management: Reported Status of Department of Defense’s 

Enterprise Resource Planning Systems, GAO-12-565R (Washington, D.C.: Mar. 30, 2012) 

and DOD Financial Management: Implementation Weaknesses in Army and Air Force 

Business Systems Could Jeopardize DOD’s Auditability Goals, GAO-12-134 (Washington, 

D.C.: Feb. 28, 2012).

23GAO-12-565R and GAO-12-134. 

24GAO-12-565R. 
 

Page 25 GAO-13-98 Information Technology Dashboard

the lowest risk levels in the face of known issues.25 DOD officials told us 

that they rated these investments as low risk because, in their view, the 

cost and schedule variances listed above did not constitute significant 

risks. Officials explained that: (1) the cost variances were not that large 

compared to DOD’s overall size and large amount of IT spending; (2) the 

schedule variance needed to be understood in the context that the 

average DOD large-scale IT program takes 7 years (or 84 months) to 

implement; and (3) that each of those programs had risk mitigation plans 

in place. However, the first two reasons are inconsistent with DOD’s own 

risk management guidance,26 which recommends that risks be assessed 

against the program’s own cost and schedule estimates, not other 

department investments. In addition, completing risk mitigation plans 

does not necessarily lower investment risk. DOD’s guidance calls for 

implementing the mitigation plan and then reassessing resulting changes 

to the risk. Even if the department adopts these elements of its own 

guidance, the CIO’s evaluation will be incomplete unless it also reflects 

the assessments of investment performance and risks identified by us 

and others. Until the department does so, CIO ratings for DOD’s 

Dashboard investments may not be sufficiently accurate or useful for its 

TechStat sessions or OMB’s management and oversight.

Selected agencies identified various benefits associated with performing 

CIO ratings and Dashboard reporting in general. Almost all of the 

agencies (five of six) reported the following three benefits.

• Increased quality of investment performance data. For example, one 

agency also reported that the Dashboard has made information about 

investments more understandable.

• Greater transparency and visibility for CIOs and their staff into 

investment- and program-level performance data. One agency 

reported that its CIO was better able to conduct reviews with actual 

investment numbers, as opposed to self-reported data presented by 

 

25Explanations included “program is being closely monitored” (Defense Enterprise 

Accounting and Management System), “program schedule being closely monitored” 

(General Fund Enterprise Business System), and “no outstanding issues or concerns” 

(Global Combat Support System-Army). 

26Department of Defense, Risk Management Guide for DOD Acquisition, Sixth Edition 

(Version 1.0), August 2006. 

CIO Ratings Present 

Benefits and 

Challenges for 

Agencies’ Investment 

Management 
 

Page 26 GAO-13-98 Information Technology Dashboard

the investment’s program managers. Agencies could also compare 

their investments’ ratings to those of other agencies and departments.

• Increased focus on project management practices. Two agencies 

reported improved investment performance as a direct result of their 

Dashboard rating and reporting activities; another stated that 

Dashboard reporting supported and reinforced their existing IT 

governance, capital planning, and program management processes.

Some of these benefits were interrelated. Several agencies viewed the 

improved data quality as a by-product of greater scrutiny brought about 

by having to report such data to the Dashboard on a regular basis. One 

agency response noted that their program managers were surprised to 

see the extent to which investment data were visible to the public, and 

that this visibility motivated their staff to provide accurate and timely data 

(which has improved data quality). Another agency noted that the visibility 

of the IT Dashboard has increased awareness among investment and 

project managers about the need to improve the planning of project 

activities and the definition of operational performance metrics (which 

support program management).

Nevertheless, agencies also identified challenges associated with 

producing and reporting CIO ratings. First, three agencies reported a 

challenge associated with the time and effort required to gather, validate, 

and gain internal approval for CIO ratings and other data reported to the 

Dashboard. For example, one agency reported that, due to the number of 

organizations involved and the number of investments being evaluated, it 

generally takes 90 to 120 days to develop and update its CIO ratings. The 

agency further reported that this effort was separate from (and in addition 

to) time it already spends on its own internal processes for managing and 

overseeing acquisition programs.

Second, four of the six agencies identified challenges with the number of 

changes OMB has made to the Dashboard, as well as with the timeliness 

and clarity of OMB’s communication regarding those changes. For 

example, officials at one agency commented that the frequency of 

changes has actually hindered their efforts to improve data quality, since 

errors sometimes resulted when it adapted to changes required by OMB. 

Officials at another agency stated that OMB allowed insufficient time for 

agencies to test their systems’ interfaces with the Dashboard when 

changes were made, which they said resulted in data errors and 

challenges for staff. These officials also noted that OMB’s guidance for 

agency submissions has, at times, not matched the technical data 
 

Page 27 GAO-13-98 Information Technology Dashboard

schemas implemented by OMB, impeding agencies’ efforts to 

successfully upload their data. An OMB staff member commented that 

their office releases changes to the Dashboard as early in the fiscal year 

as possible to give agencies time to adjust and that OMB announces 

planned changes to agencies before they are implemented via the 

Dashboard’s interagency web portal. OMB has recently held meetings 

with agency officials to discuss these issues and determine ways to better 

communicate going forward.

Finally, one agency responded that while monthly updates to the 

Dashboard have increased investment and project managers’ attention to 

the performance of their investments and projects, this regular scrutiny 

could encourage investment and project managers to “perform to the test” 

rather than concentrate on effective investment and project management. 

However, based on the interrelationships of the benefits of CIO ratings 

identified by some agencies, the process of generating and reporting CIO 

ratings does not have to be just a grading exercise. As previously noted, 

the benefit of improved investment performance data for the CIO’s 

investment evaluation can lead to more effective management, which 

could, in turn, improve investment performance. Executives and staff who 

can envision these results from the Dashboard’s CIO evaluations may be 

less likely to view the additional time and effort required to generate the 

CIO ratings as a challenge, but as an opportunity for more efficient and 

effective management.

Since its inception in 2009, the Federal IT Dashboard has increased the 

transparency of the performance of major federal IT investments. Its CIO 

ratings, in particular, have improved visibility into changes in the risk 

levels of agencies’ investments over time. Determining whether such 

changes represent improvements or deficiencies in management and 

oversight can be difficult without additional information on investment 

performance and the rating process, but analyzing and reporting the 

ratings for investments and agencies over time for the President’s budget 

submission could help OMB ensure that risk is accurately assessed and 

that patterns of risk deserving of special management attention are 

identified.

DOD demonstrated one such pattern of interest in its CIO ratings. During 

the 34-month life of the Dashboard, none of the 87 investments that were 

active as of March 2012 were rated high risk or moderately high risk, and 

approximately 85 percent of ratings were low risk or moderately low risk. 

Although DOD implemented OMB’s broad instructions for producing CIO 

Conclusions 
 

Page 28 GAO-13-98 Information Technology Dashboard

ratings, it also considered how the ratings might increase the likelihood of 

an OMB review of an investment and minimized the effects of significant 

schedule delays and cost increases, which were identified in our reviews 

and those of DOD’s Inspector General. As a result, DOD is masking 

significant investment risks, has not employed its own risk management 

guidance, and has not delivered the transparency intended by the 

Dashboard. By incorporating the results of external reviews into its 

evaluations, DOD can further improve the quality of the information on 

which investment risk ratings are based.

Beyond the transparency they promote, CIO ratings present an 

opportunity to improve the data and processes agencies use to assess 

investment risk. Some agencies have already experienced collateral 

benefits and management results from their risk evaluations. Continuing 

focus from OMB and agencies on how to accurately portray and derive 

value from the ratings and the associated processes could enable 

agencies to experience such benefits.

To ensure that OMB’s preparation of the President’s budget submission 

accurately reflects the risks associated with all major IT investments, we 

are recommending that the Federal CIO analyze agency trends reflected 

in Dashboard CIO ratings, and present the results of this analysis with the 

President’s annual budget submission.

To ensure that DOD’s CIO evaluations of investment risk for its major IT 

Dashboard investments reflect all available performance assessments 

and are consistent with the department’s own guidance for managing risk, 

we are recommending that the Secretary of Defense direct the 

department’s CIO to reassess the department’s considerations for 

assigning CIO risk levels for Dashboard investments, including 

assessments of investment performance and risk from outside the 

programs, and apply the appropriate elements of the department’s risk 

management guidance to OMB’s evaluation factors in determining CIO 

ratings.

We provided a draft of our report to the six agencies selected for our 

review and to OMB. In oral comments, staff from OMB’s Office of EGovernment & Information Technology stated that OMB concurred with 

our recommendation that the Federal CIO analyze agency trends 

reflected in Dashboard CIO ratings and present the results of this analysis 

with the President’s annual budget submission. OMB staff also provided 

Recommendations for 

Executive Action

Agency Comments 

and Our Evaluation
 

Page 29 GAO-13-98 Information Technology Dashboard

technical comments, which we incorporated as appropriate. In a written 

response, DOD’s Deputy Chief Information Officer for Information 

Enterprise agreed with our recommendation that the department’s CIO 

reassess considerations for assigning CIO risk levels for Dashboard

investments, and committed to updating the department’s CIO ratings 

process to better report risk and improve the timeliness and transparency 

of reporting. DOD’s written response is reprinted in Appendix IV. Officials 

at DOI provided technical comments, which we incorporated as 

appropriate. The remaining agencies had no comment on the draft report. 

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies to interested congressional 

committees; the Secretaries of Defense, Interior, Homeland Security, 

Health and Human Services, the Director of the National Science 

Foundation, the Director of the Office of Personnel Management, the 

Director of the Office of Management and Budget; and other interested 

parties. In addition, the report will be available at no charge on the GAO 

website at http://www.gao.gov. 

If you or your staffs have any questions on the matters discussed in this 

report, please contact David A. Powner at (202) 512-9286 or by e-mail at 

pownerd@gao.gov. Contact points for our Offices of Congressional 

Relations and Public Affairs may be found on the last page of this report.

GAO staff who made major contributions to this report are listed in 

appendix IV.

David A. Powner

Director, Information Technology

 Management Issues
Appendix I: Objectives, Scope, and 

Methodology

Page 30 GAO-13-98 Information Technology Dashboard

Our objectives were to (1) characterize the Chief Information Officer (CIO) 

ratings for selected federal agencies’ information technology (IT) 

investments as reported over time on the federal IT Dashboard; (2) 

determine how agencies’ approaches for assigning and updating CIO 

ratings vary; and (3) describe the benefits and challenges associated with 

agencies’ approaches to the CIO rating.

To establish the scope of our review, we downloaded and examined data 

on total IT spending for fiscal year 2011 for the 27 agencies reported on 

the IT Dashboard. (The Office of Management and Budget (OMB) 

extracts these data based on exhibit 300 forms submitted by each 

agency.) We then selected six agencies that spanned a range of IT 

spending for fiscal year 2011, including the three highest spending 

agencies, two of the lowest, and an agency in the middle. Collectively, 

these agencies accounted for approximately $51 billion, or 65 percent, of 

2011 spending on IT investments. The six agencies are the Department 

of Defense, Department of Homeland Security, Department of Health and 

Human Services, Department of the Interior, National Science 

Foundation, and Office of Personnel Management. The results in this 

report represent only these agencies.

To address the first objective, we downloaded and examined the 

Dashboard’s CIO ratings for all investments at the six agencies we 

selected (a total of approximately 308 investments reported by these 

agencies).1

 

1

We did not independently evaluate the ratings reported by agencies. However, we 

determined that they were sufficiently reliable for the purposes of our objectives by 

confirming with each agency that the ratings that we downloaded from the IT Dashboard 

were complete, accurate, and reflected the data they had reported to OMB. 

To characterize the numbers and percentages of major IT

investments at each risk level at each of our subject agencies, we 

analyzed, summarized, and—where appropriate—graphically depicted 

average CIO ratings for investments by agencies over time during the 

period from June 2009 to March 2012. Specifically, we compared the CIO 

ratings in June 2009 (or whenever an individual investment was first 

rated) up through and including each investment’s rating as of March 

2012 and summarized the data by agency. To describe whether CIO 

ratings indicated higher or lower investment risk over time, we calculated 

the numbers and percentages of investments (by agency and collectively 

for all the agencies) that maintained a constant rating over the entire 

performance period, and those that experienced a change to their CIO 

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology

Page 31 GAO-13-98 Information Technology Dashboard

rating in at least one rating period. Then we analyzed the subset of 

investments that experienced at least one changed rating and compared 

the first CIO rating with the latest CIO rating (no later than March 2012) to 

determine the numbers and percentages of investments (by agency and 

collectively for all the agencies) that experienced a net rating increase, a 

net rating decrease, or no net change. We also examined the comments 

provided with the ratings to determine whether such comments were 

useful in understanding the ratings. We presented our results to each 

agency and OMB and solicited their input, explanations for the results, 

and additional corroborating documentation, where appropriate.

To address our second objective, we reviewed available documentation, 

obtained written responses to questions we posed to all agencies, and 

interviewed OMB and agency officials to determine their policies and 

practices related to assigning and updating the CIO ratings and related 

data for the Dashboard. Specifically, we gathered descriptions about the 

data, participants, and processes used to generate CIO ratings for 

investments; when and under what circumstances each agency updates 

its ratings; the specific factors agencies used in assigning their ratings; 

and the reason(s) for their approaches to assigning and reporting the 

ratings. We reviewed our results with agency officials to ensure that our 

presentation of their approach was accurate. In addition, we utilized our 

prior work and a report by the Department of Defense’s Office of the 

Inspector General related to the department’s major IT investments. We 

compared the findings in these reports to the CIO ratings the department 

submitted to the Dashboard for investments that had been rated 

consistently low or moderately low risk, and discussed our results with 

department officials.

To address our third objective, we reviewed written and oral descriptions 

of the benefits and challenges that agencies and OMB have experienced 

in developing, submitting, updating, and utilizing CIO ratings. We sought 

specific examples, corroborating documentation, and causal factors, 

where available. After obtaining this information from individual agencies, 

we compared their responses to identify benefits and challenges common 

to multiple agencies and applied our judgment in determining whether any 

additional benefits or challenges were present.

We conducted this performance audit from January 2012 to September

2012 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 
Appendix I: Objectives, Scope, and 

Methodology

Page 32 GAO-13-98 Information Technology Dashboard

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.
Appendix II: Risk Levels for Investments at 

Selected Agencies, as of March 2012

Page 33 GAO-13-98 Information Technology Dashboard

The table below lists the total number of major information technology (IT) 

investments rated on the federal IT Dashboard as of March 2012 for each 

agency selected for this review, with the numbers of investments rated at 

each of the risk levels specified by the Office of Management and Budget 

(OMB) for the chief information officer (CIO) rating. The last line in the 

table reports each agency’s total budget for fiscal year 2012 for their 

major IT investments, as also reported on the Dashboard in March 2012.

Table 8: CIO Ratings and Budget Totals for Selected Agencies’ Major IT Investments as of March 2012 

DOD DHS HHS DOI OPM NSF

High risk and moderately high risk 0 3 3 6 0 0 

Medium risk 13 35 19 8 3 0 

Low risk and moderately low risk 74 45 59 34 6 5 

Total number of investments rated on the Dashboard 87 83 81 48 9 5 

Budget for major IT investments (FY 2012) $13.3B $4.4B $2.8B $865.6M $46.3M $85.5M

Source: GAO analysis of data downloaded from OMB’s IT Dashboard.

Note: Table does not include downgraded or eliminated investments.

Appendix II: Risk Levels for Investments at 

Selected Agencies, as of March 2012
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 34 GAO-13-98 Information Technology Dashboard

This appendix provides additional information about chief information 

officer (CIO) ratings for major IT information technology (IT) investments 

at each of the agencies selected for this review. The first figure for each 

agency depicts the number of investments at each rating level for the end 

of each month, as reported on the federal IT Dashboard.1 The second 

figure depicts the number of investments whose risk level demonstrated a 

net increase, net decrease, no net change, or remained constant during 

the investment’s entire time on the Dashboard.2

 

1

The Office of Management and Budget (OMB) directs agency CIOs to evaluate 

investments and assign ratings according to a five-point scale. The risk levels are: 5-low 

risk (green), 4-moderately low risk (green), 3-medium risk (yellow), 2-moderately high risk 

(red), and 1-high risk (red). 

2

An investment’s risk level increased when it received a lower CIO rating in March 2012 

compared with its initial Dashboard rating. An investment’s risk level was reduced when it 

received a higher CIO rating in March 2012 compared with its initial Dashboard rating. An 

investment’s CIO rating exhibited no net change when it received the same rating in 

March 2012 as its initial rating, despite any interim changes that may have occurred. An 

investment’s CIO rating was constant when the rating remained unchanged from the initial 

CIO rating through March 2012. 

Appendix III: CIO Ratings and Net Changes 

for Major IT Investments at Selected 

Agencies, June 2009 through March 2012
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 35 GAO-13-98 Information Technology Dashboard

Figure 6: CIO Ratings for Major IT Investments at the Department of Defense, as Reported on the IT Dashboard from July 2009 

through March 2012

Note: Figure does not include downgraded or eliminated investments.

Department of Defense
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 36 GAO-13-98 Information Technology Dashboard

Figure 7: Changes to Risk Levels for Major IT Investments at the Department of 

Defense, Initial CIO Rating through March 2012

Note: Figure does not include downgraded or eliminated investments. An investment’s CIO rating is 

constant when the rating remained unchanged from the initial rating through March 2012. All other 

categories of risk in the chart compare the initial CIO rating to the CIO rating as of March 2012, 

ignoring any interim changes that may have occurred in between those dates.
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 37 GAO-13-98 Information Technology Dashboard

Figure 8: CIO Ratings for Major IT Investments at the Department of Homeland Security, as Reported on the IT Dashboard 

from July 2009 through March 2012

Note: Figure does not include downgraded or eliminated investments.

Department of Homeland 

Security 
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 38 GAO-13-98 Information Technology Dashboard

Figure 9: Changes to Risk Levels for Major IT Investments at the Department of 

Homeland Security, Initial CIO Rating through March 2012

Note: Figure does not include downgraded or eliminated investments. An investment’s CIO rating is 

constant when the rating remained unchanged from the initial rating through March 2012. All other 

categories of risk in the chart compare the initial CIO rating to the CIO rating as of March 2012, 

ignoring any interim changes that may have occurred in between those dates.
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 39 GAO-13-98 Information Technology Dashboard

Figure 10: CIO Ratings for Major IT Investments at the Department of Health and Human Services, as Reported on the IT 

Dashboard from July 2009 through March 2012

Note: Figure does not include downgraded or eliminated investments.

Department of Health and 

Human Services
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 40 GAO-13-98 Information Technology Dashboard

Figure 11: Changes to Risk Levels for Major IT Investments at the Department of 

Health and Human Services, Initial CIO Rating through March 2012

Note: Figure does not include downgraded or eliminated investments. An investment’s CIO rating is 

constant when the rating remained unchanged from the initial rating through March 2012. All other 

categories of risk in the chart compare the initial CIO rating to the CIO rating as of March 2012, 

ignoring any interim changes that may have occurred in between those dates.
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 41 GAO-13-98 Information Technology Dashboard

Figure 12: CIO Ratings for Major IT Investments at the Department of the Interior, as Reported on the IT Dashboard from July 

2009 through March 2012

Note: Figure does not include downgraded or eliminated investments.

Department of the Interior
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 42 GAO-13-98 Information Technology Dashboard

Figure 13: Changes to Risk Levels for Major IT Investments at the Department of the 

Interior, Initial CIO Rating through March 2012

Note: Figure does not include downgraded or eliminated investments. An investment’s CIO rating is 

constant when the rating remained unchanged from the initial rating through March 2012. All other 

categories of risk in the chart compare the initial CIO rating to the CIO rating as of March 2012, 

ignoring any interim changes that may have occurred in between those dates.
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 43 GAO-13-98 Information Technology Dashboard

Figure 14: CIO Ratings for Major IT Investments at the National Science Foundation, as Reported on the IT Dashboard from 

June 2009 through March 2012

Note: Figure does not include downgraded or eliminated investments.

National Science 

Foundation 
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 44 GAO-13-98 Information Technology Dashboard

Figure 15: Changes to Risk Levels for Major IT Investments at the National Science 

Foundation, Initial CIO Rating through March 2012

Note: Figure does not include downgraded or eliminated investments. An investment’s CIO rating is 

constant when the rating remained unchanged from the initial rating through March 2012. All other 

categories of risk in the chart compare the initial CIO rating to the CIO rating as of March 2012, 

ignoring any interim changes that may have occurred in between those dates.
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 45 GAO-13-98 Information Technology Dashboard

Figure 16: CIO Ratings for Major IT Investments at the Office of Personnel Management, as Reported on the IT Dashboard 

from July 2009 through March 2012

Note: Figure does not include downgraded or eliminated investments.

Office of Personnel 

Management
Appendix III: CIO Ratings and Net Changes for 

Major IT Investments at Selected Agencies, 

June 2009 through March 2012

Page 46 GAO-13-98 Information Technology Dashboard

Figure 17: Changes to Risk Levels for Major IT Investments at the Office of 

Personnel Management, Initial CIO Rating through March 2012

Note: Figure does not include downgraded or eliminated investments. An investment’s CIO rating is 

constant when the rating remained unchanged from the initial rating through March 2012. All other 

categories of risk in the chart compare the initial CIO rating to the CIO rating as of March 2012, 

ignoring any interim changes that may have occurred in between those dates.
Appendix IV: Comments from the Department 

of Defense

Page 47 GAO-13-98 Information Technology Dashboard

Appendix IV: Comments from the 

Department of Defense
Appendix IV: Comments from the Department 

of Defense

Page 48 GAO-13-98 Information Technology Dashboard
Appendix V: GAO Contacts and Staff 

Acknowledgments

Page 49 GAO-13-98 Information Technology Dashboard

David A. Powner, (202) 512-9286 or pownerd@gao.gov

In addition to the contact name above, the following staff also made key 

contributions to this report: Paula Moore (Assistant Director), Neil 

Doherty, Lynn Espedido, Rebecca Eyler, Kate Feild, Dan Gordon, 

Andrew Stavisky, Sonya Vartivarian, Shawn Ward, Kevin Walsh, Jessica 

Waselkow, and Monique Williams.

Appendix V: GAO Contacts and Staff 

Acknowledgments 

GAO Contacts

Staff 

Acknowledgments 

(311264)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (www.gao.gov). Each weekday afternoon, 

GAO posts on its website newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations 

Public Affairs

Please Print on Recycled Paper.