IT DASHBOARD 

Agencies Are 

Managing Investment 

Risk, but Related 

Ratings Need to Be 

More Accurate and 

Available

Report to Congressional Requesters

December 2013

GAO-14-64

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-14-64, a report to 

congressional requesters

December 2013

IT DASHBOARD

Agencies Are Managing Investment Risk, but Related 

Ratings Need to Be More Accurate and Available

Why GAO Did This Study

OMB launched the Dashboard in June 

2009 as a public Web site that reports 

performance for major IT 

investments—on which the federal 

government plans to invest over $38

billion in fiscal year 2014. The 

Dashboard is to provide transparency 

for these investments and to facilitate 

public monitoring of them. After its 

launch, OMB began using it to identify 

at-risk investments. 

This report (1) characterizes the CIO 

ratings for selected federal agencies’ 

IT investments as reported on the 

Dashboard over time, (2) determines

the extent to which selected agencies’ 

CIO ratings are consistent with 

investment risk, and (3) determines the 

extent to which selected agencies are 

addressing at-risk investments. GAO

selected the eight agencies with the 

most reported major IT spending in 

fiscal year 2012 (excluding those GAO 

recently reviewed) and selected 10

investments at each agency. GAO 

reviewed the investments’ 

documentation, compared it to the CIO 

ratings, and reviewed processes used 

for the highest-risk investments. GAO

also interviewed appropriate officials.

What GAO Recommends

GAO recommends that OMB make 

Dashboard information available 

independent of the budget process,

and that agencies appropriately 

categorize IT investments and address 

identified weaknesses. OMB neither 

agreed nor disagreed. Six agencies 

generally agreed with the report or had 

no comments and two others did not 

agree, believing their categorizations 

were appropriate. GAO continues to 

believe its recommendations remain 

valid, as discussed.

What GAO Found

As of August 2013, the Chief Information Officers (CIO) at the eight selected 

agencies rated 198 of their 244 major information technology (IT) investments 

listed on the Federal IT Dashboard (Dashboard) as low risk or moderately low 

risk, 41 as medium risk, and 5 as high risk or moderately high risk. However, the 

total number of investments reported by these agencies has varied over time, 

which impacts the number of investments receiving CIO ratings. For example, 

Energy reclassified several of its supercomputer investments from IT to facilities 

and Commerce decided to reclassify its satellite ground system investments. 

Both decisions resulted in the removal of the investments from the Dashboard, 

even though the investments were clearly IT. In addition, the Office of 

Management and Budget (OMB) does not update the public version of the 

Dashboard as the President’s budget request is being created. As a result, the

public version of the Dashboard was not updated for 15 of the past 24 months, 

and so was not available as a tool for investment oversight and decision making. 

Of the 80 investments reviewed, 53 of the CIO ratings were consistent with the 

investment risk, 20 were partially consistent, and 7 were inconsistent (see table).

Table: Consistency of Selected Agencies’ CIO Ratings with Supporting Documentation

Agency Rating consistency of each agency’s 10 selected investments

Department of Agriculture ● ● ◐ ◐ ◐ ◐ ◐ ◐ ◐ ◐

Department of Commerce ● ● ● ● ● ● ● ● ◐ ◐

Department of Energy ● ● ● ● ● ◐ ◐ ◐ ◐ ◐

Department of Justice ● ● ● ● ● ● ● ● ● ◐

Department of Transportation ● ● ● ● ● ● ● ● ● ●

Department of the Treasury ● ● ● ● ● ● ● ● ● ●

Department of Veterans Affairs ◐ ◐ ◐ ◌ ◌ ◌ ◌ ◌ ◌ ◌

Social Security Administration ● ● ● ● ● ● ● ● ● ◐

Key: ●: The investment’s ratings were consistent with supporting documents. ◐: The investment’s 

ratings were partially consistent with supporting documents. ◌: The investment’s ratings were 

inconsistent with supporting documents.

Source: GAO analysis of OMB’s Dashboard and agency data.

While two agencies’ CIO ratings were entirely consistent, other agencies’ ratings 

were inconsistent for a variety of reasons, including delays in updating the 

Dashboard and how investment performance was tracked. For example, the 

Department of Justice downgraded an investment in July 2012, but the 

Dashboard was not updated to reflect this until April 2013. Further, the Social 

Security Administration resets investment cost and schedule performance 

baselines annually, an approach that increases the risk of undetected cost or 

schedule variances that will impact investment success.

Of the eight investments that were at highest risk in 2012, seven were reviewed 

by their agencies using tools such as TechStat sessions—evidence-based 

reviews intended to improve investment performance and other high-level 

reviews. Each of these resulted in action items intended to improve performance. 

The final investment was scheduled to have a TechStat, but instead, according to 

department officials, a decision was made to modify its program cost and 

schedule commitments to better reflect the investment’s actual performance.

View GAO-14-64. For more information, 

contact David A. Powner at (202) 512-9286 or 

pownerd@gao.gov.
Page i GAO-14-64 IT Dashboard

Letter 1

Background 3

CIOs at Selected Agencies’ Rated Majority of IT Investments as 

Low or Moderately Low Risk 12

Most CIO Ratings Are Consistent with Reported Investment Risk 20

Highest-Risk Investments Were Appropriately Addressed by 

Selected Agencies 25

Conclusions 27

Recommendations for Executive Action 28

Agency Comments and Our Evaluation 29

Appendix I Objectives, Scope, and Methodology 33

Appendix II Selected Investments 36

Appendix III Comments from the Office of Management and Budget 39

Appendix IV Comments from the Department of Agriculture 43

Appendix V Comments from the Department of Commerce 44

Appendix VI Comments from the Department of Energy 47

Appendix VII Comments from the Department of the Treasury 48

Appendix VIII Comments from the Department of Veterans Affairs 49

Contents
Page ii GAO-14-64 IT Dashboard

Appendix IX Comments from the Social Security Administration 52

Appendix X GAO Contact and Staff Acknowledgments 54

Tables

Table 1: Dashboard Variance and Rating Colors 6

Table 2: Investment Evaluation Factors Identified by OMB for 

Assigning CIO Ratings 7

Table 3: IT Dashboard CIO Rating Colors, Based on a Five-Point 

Scale for CIO Ratings 8

Table 4: Assessment of the Consistency of Selected Agencies’ CIO 

Ratings with Supporting Documentation from January 

2012 through December 2012 20

Table 5: Investments Selected for Review and Associated Fiscal 

Year 2012 Spending 36

Figures

Figure 1: Example of an Agency Portfolio Page as Reported on 

OMB’s IT Dashboard, May 2013 5

Figure 2: Dashboard CIO Ratings over Time from the Eight 

Selected Agencies 13

Figure 3: Dashboard Rating Changes over Time and Associated 

Number of Investments from the Eight Selected Agencies 

between June 2009 and August 2013 15

Figure 4: Dashboard Rating Changes over Time and Associated 

Number of Investments by Agency between June 2009 and 

August 2013 16

Figure 5: Number of Investments the Eight Selected Agencies 

Reported on the Dashboard over Time 17

Figure 6: Dashboard Ratings in 2012 for the Eight At-Risk 

Investments 26
Page iii GAO-14-64 IT Dashboard

Abbreviations

Agriculture Department of Agriculture

CIO Chief Information Officer

Commerce Department of Commerce

Energy Department of Energy

FAA Federal Aviation Administration

FBI Federal Bureau of Investigation

Justice Department of Justice

IT Information technology

OMB Office of Management and Budget

SSA Social Security Administration

TechStats TechStat Accountability Sessions

Transportation Department of Transportation

Treasury Department of the Treasury

VA Department of Veterans Affairs

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-14-64 IT Dashboard

441 G St. N.W.

Washington, DC 20548

December 12, 2013

The Honorable Thomas R. Carper

Chairman

The Honorable Tom Coburn, M.D.

Ranking Member

Committee on Homeland Security and Governmental Affairs

United States Senate

The Honorable Fred Upton

Chairman

Committee on Energy and Commerce

House of Representatives

In accordance with its responsibilities set forth in law,1 the Office of 

Management and Budget (OMB) launched the Federal Information 

Technology (IT) Dashboard in June 2009 as a public Web site that 

reports performance and supporting data for major IT investments2

This report responds to your request that we review the consistency of 

investment risk information submitted to the Dashboard. Our objectives 

—on 

which the federal government plans to invest over $38 billion in fiscal year 

2014. The Dashboard is to provide transparency for these investments in 

order to facilitate public monitoring of government operations and 

accountability for investment performance by the federal Chief 

Information Officers (CIO) who oversee them. According to OMB, it began 

using the Dashboard to identify at-risk investments with its launch in June 

2009. These investments became the focus of joint OMB-agency 

TechStat Accountability Sessions (TechStats)—evidence-based reviews 

intended to improve investment performance through concrete actions.

 1

For example, the Paperwork Reduction Act (44 U.S.C. § 3501 et seq.) specifies OMB 

and agency responsibilities for managing IT and the Clinger-Cohen Act of 1996 (40 U.S.C. 

§ 11302(c)) charges OMB with responsibility for analyzing, tracking, and evaluating the 

risks and results of all major IT investments as part of the federal budget process, and 

reporting to Congress on the performance benefits achieved by these investments. 

2

“Major IT investment” means a system or an acquisition requiring special management 

attention because it has significant importance to the mission or function of the 

government; significant program or policy implications; high executive visibility; high 

development, operating, or maintenance costs; an unusual funding mechanism; or is 

defined as major by the agency’s capital planning and investment control process.
Page 2 GAO-14-64 IT Dashboard

for this engagement were to (1) characterize the CIO ratings for selected 

federal agencies’ IT investments as reported on the Dashboard over time, 

(2) determine the extent to which selected agencies’ CIO ratings are 

consistent with reported investment risk, and (3) determine the extent to 

which selected agencies are addressing at-risk investments.

To address our first objective, we selected the eight agencies3 with the 

most reported major IT spending in fiscal year 2012, after excluding 

agencies reviewed in our most recent report on the Dashboard.4

To accomplish the second objective, we selected the top 10 major 

investments with the highest reported IT spending at each of the 8 

selected agencies (for a total of 80 investments). We reviewed investment 

documentation (such as executive-level briefings, operational analyses, 

and the results of performance reviews) and interviewed officials from 

each of the agencies to understand how they rated and monitored 

investments. We compared these documents to the CIO ratings 

submitted to the Dashboard in calendar year 2012 to determine whether 

the ratings on the Dashboard were accurately portraying the risk of these 

investments. We elected to review the investments’ ratings during 

calendar year 2012 so that our assessment would span multiple fiscal 

years.

We then 

reviewed publicly available data from the Dashboard from its inception in 

June 2009 through August 2013. We also interviewed officials from OMB 

and the selected agencies.

Finally, for our third objective, we reviewed and described each of the 

processes used by the eight selected agencies to address the highestrisk major investments (such as TechStat guides and investment review 

board results). We also identified investments within the 80 investments 

selected for the second objective which had received high or moderately 

high ratings of risk on the Dashboard during 2012. We analyzed both the 

processes used to address these investments and the results of 

 3

The eight selected agencies are the Departments of Agriculture (Agriculture), Commerce 

(Commerce), Energy (Energy), Justice (Justice), Transportation (Transportation), the 

Treasury (Treasury), and Veterans Affairs (VA), as well as the Social Security 

Administration (SSA). See app. II for a list of the selected investments.

4

GAO, Information Technology Dashboard: Opportunities Exist to Improve Transparency 

and Oversight of Investment Risk at Select Agencies, GAO-13-98 (Washington, D.C.: Oct. 

16, 2012).
Page 3 GAO-14-64 IT Dashboard

performance reviews. We also interviewed officials to determine whether 

the agencies implemented the appropriate risk management processes to 

oversee and review the selected investments.

We conducted this performance audit from February to December 2013 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives. Further details of our 

objectives, scope, and methodology are provided in appendix I.

While IT can enrich people’s lives and improve organizational 

performance, we have previously reported that federal IT projects too 

frequently incur cost overruns and schedule slippages while contributing 

little to mission-related outcomes. For example, in July 2013, we testified 

that the federal government continued to spend billions of dollars on 

troubled IT investments and we identified billions of dollars worth of failed 

and challenged IT projects.5

OMB plays a key role in overseeing how federal agencies manage their 

IT investments by working with them to better plan, justify, and determine 

how to manage them. Each year, OMB and federal agencies work 

together to determine how much the government plans to spend on IT 

projects and how these funds are to be allocated. OMB also guides 

agencies in developing sound business cases for IT investments and 

establishing management processes for overseeing these investments 

throughout their life cycles. The scope of this undertaking is quite large: in 

planning for fiscal year 2014, 27 federal agencies reported plans to spend 

 5

GAO, Information Technology: OMB and Agencies Need to More Effectively Implement 

Major Initiatives to Save Billions of Dollars, GAO-13-796T (Washington, D.C.: July 25, 

2013).

Background
Page 4 GAO-14-64 IT Dashboard

about $82 billion on IT.6

Within OMB, the Office of E-Government and Information Technology, 

headed by the Federal CIO, directs the policy and strategic planning of 

federal IT investments and is responsible for oversight of federal 

information technology spending.

Of that, $38.7 billion is to be spent on major IT 

investments, $37.6 billion is to be spent on non-major IT investments, and 

$5.5 billion is to be spent on classified Department of Defense IT 

investments.

In June 2009, OMB deployed a public Web site to further improve the 

transparency and oversight of agencies’ IT investments. Known as the IT 

Dashboard,7

The Dashboard visually presents performance ratings for agencies and 

for individual investments using metrics that OMB has defined—cost, 

schedule, and CIO evaluation. The Web site also provides the capability 

to download certain data. Figure 1 is an example of VA’s portfolio page as 

recently depicted on the Dashboard. (The ratings are explained in the 

narrative following the figure.)

this site displays federal agencies’ cost, schedule, and 

performance data for over 700 major federal IT investments at 27 federal 

agencies, accounting for $38.7 billion of those agencies’ planned $82 

billion budget for fiscal year 2014. According to OMB, these data are 

intended to provide a near-real-time perspective on the performance of 

these investments, as well as a historical perspective. Further, the public 

display of these data is intended to allow OMB; other oversight bodies, 

including Congress; and the general public to hold the government 

agencies accountable for progress and results.

 6

The 27 federal agencies are the Departments of Agriculture, Commerce, Defense, 

Education, Energy, Health and Human Services, Homeland Security, Housing and Urban 

Development, Justice, Labor, State, the Interior, the Treasury, Transportation, Veterans 

Affairs; the Environmental Protection Agency; the General Services Administration; the 

National Aeronautics and Space Administration; National Archives and Records 

Administration; National Science Foundation; Nuclear Regulatory Commission; Office of 

Personnel Management; Small Business Administration; Smithsonian Institution; Social 

Security Administration; U.S. Agency for International Development; and the U.S. Army 

Corps of Engineers. 

7

http://www.itdashboard.gov.

OMB’s Dashboard Is 

Intended to Provide 

Visibility into the 

Performance of Federal 

IT Investments
Page 5 GAO-14-64 IT Dashboard

Figure 1: Example of an Agency Portfolio Page as Reported on OMB’s IT Dashboard, May 2013 
Page 6 GAO-14-64 IT Dashboard

The Dashboard’s data spans the period from its June 2009 inception to 

the present, and is based, in part, on agency assessments of individual 

investment performance and each agency’s budget submissions to 

OMB.8

The cost and schedule data agencies are required to submit to the 

Dashboard have changed over time, as have the related calculations. For 

example, in response to our recommendations (further discussed in the 

following section), OMB changed how the Dashboard calculates the cost 

and schedule ratings in July 2010 to include “in progress” milestones 

rather than just “completed” ones for a more accurate reflection of current 

investment status. While the required cost and schedule data have 

changed, the thresholds for assigning cost and schedule ratings have 

remained constant over the life of the Dashboard. Specifically, the 

Dashboard assigns rating colors (red, yellow, green) based on agencysubmitted cost and schedule variances, as shown in table 1.

Table 1: Dashboard Variance and Rating Colors

Project level cost and schedule variance rating Rating color 

≥ 30% Red 

≥ 10% and < 30% Yellow 

< 10% Green 

Source: OMB’s IT Dashboard.

Unlike the variance-based cost and schedule ratings, the Dashboard’s 

“Investment Evaluation by Agency CIO” (also called the CIO rating) is 

determined by agency officials. According to OMB’s instructions, each 

agency CIO is to assess his or her IT investments against a set of six preestablished evaluation factors identified by OMB (shown in table 2) and 

then assign a rating of 1 (high risk) to 5 (low risk) based on the CIO’s best 

judgment of the level of risk facing the investment.

 8

Each agency’s budget submissions include an exhibit 53 and an exhibit 300 for each 

major IT investment. The exhibit 53 lists all of an agency’s IT projects and associated 

costs. Exhibit 300s (also called the Capital Asset Plan and Business Case) are prepared 

for every major IT investment, are used to justify resource requests, and are intended to 

enable an agency to demonstrate to its own management, as well as to OMB, that a major 

project is well-planned.

Cost and Schedule Ratings

CIO Ratings
Page 7 GAO-14-64 IT Dashboard

Table 2: Investment Evaluation Factors Identified by OMB for Assigning CIO 

Ratings

Evaluation factor Supporting examples

Risk management Risk management strategy exists

Risks are well understood by senior leadership

Risk log is current and complete

Risks are clearly prioritized

Mitigation plans are in place to address risks 

Requirements 

management 

Investment objectives are clear and scope is controlled

Requirements are complete, clear, and validated

Appropriate stakeholders are involved in requirements definition 

Contractor 

oversight 

Acquisition strategy is defined and managed via an Integrated 

Program Team

Agency receives key reports, such as earned value reports, current 

status, and risk logs

Agency is providing appropriate management of contractors such 

that the government is monitoring, controlling, and mitigating the 

impact of any adverse contract performance 

Historical 

performance 

No significant deviations from planned cost and schedule

Lessons learned and best practices are incorporated and adopted 

Human capital Qualified management and execution team for the IT investments 

and/or contracts supporting the investment

Low turnover rate 

Other Other factors that the CIO deems important to forecasting future 

success 

Source: OMB’s IT Dashboard.

OMB recommends that CIOs consult with appropriate stakeholders in 

making their evaluation, including Chief Acquisition Officers, program 

managers, and other interested parties. According to an OMB staff 

member, agency CIOs are responsible for determining appropriate 

thresholds for the risk levels and for applying them to investments when 

assigning CIO ratings. OMB requires agencies to update these ratings as 

soon as new information becomes available which will affect an 

investment’s assessment. After agencies assign a level of risk to each 

investment, the Dashboard assigns colors to CIO ratings according to a 

five-point scale, as illustrated in table 3.
Page 8 GAO-14-64 IT Dashboard

Table 3: IT Dashboard CIO Rating Colors, Based on a Five-Point Scale for CIO 

Ratings

Rating (by agency CIO) Color 

1-High risk Red 

2-Moderately high risk Red 

3-Medium risk Yellow 

4-Moderately low risk Green 

5-Low risk Green 

Source: OMB’s IT Dashboard.

An OMB staff member from the Office of E-Government and Information 

Technology noted that the CIO rating should be a current assessment of 

future performance based on historical results and is the only Dashboard 

performance indicator that has been defined and produced the same way 

since the Dashboard’s inception.

Furthermore, OMB issued guidance in August 20119 that stated, among 

other things, that agency CIOs shall be held accountable for the 

performance of IT program managers based on their governance process 

and the data reported on the IT Dashboard, which includes the CIO 

rating. According to OMB, the addition of CIO names and photos on 

Dashboard investments is intended to highlight this accountability and link 

it to the Dashboard’s reporting on investment performance.

We have previously reported that OMB has taken significant steps to 

enhance the oversight, transparency, and accountability of federal IT 

investments by creating its IT Dashboard, and by improving the accuracy 

of investment ratings. We also found issues with the accuracy and data 

reliability of cost and schedule data, and recommended steps that OMB 

should take to improve these data.

• In July 2010, we reported10

 9

Chief Information Officer Authorities, Memorandum for Heads of Executive Departments 

and Agencies, M-11-29 (Washington, D.C.: Aug. 8, 2011).

that the cost and schedule ratings on 

OMB’s Dashboard were not always accurate for the investments we 

10GAO, Information Technology: OMB’s Dashboard Has Increased Transparency and 

Oversight, but Improvements Needed, GAO-10-701 (Washington, D.C.: July 16, 2010).

GAO Has Previously 

Reported on the 

Dashboard’s Value, 

Data Quality, and 

Improvements
Page 9 GAO-14-64 IT Dashboard

reviewed, because these ratings did not take into consideration 

current performance. As a result, the ratings were based on outdated 

information. We recommended that OMB report on its planned 

changes to the Dashboard to improve the accuracy of performance 

information and provide guidance to agencies to standardize 

milestone reporting. OMB agreed with our recommendations and, as 

a result, updated the Dashboard’s cost and schedule calculations to 

include both ongoing and completed activities.

• Similarly, our report in March 201111

• We also reported on OMB’s guidance to agencies for reporting their IT 

investments in September 2011 and found that it did not ensure 

complete reporting.

noted that OMB had initiated 

several efforts to increase the Dashboard’s value as an oversight tool, 

and had used its data to improve federal IT management. We also 

reported, however, that agency practices and the Dashboard’s 

calculations contributed to inaccuracies in the reported investment 

performance data. For instance, we found missing data submissions 

or erroneous data at each of the five agencies we reviewed, along 

with instances of inconsistent program baselines and unreliable 

source data. As a result, we recommended that the agencies take 

steps to improve the accuracy and reliability of their Dashboard 

information, and that OMB improve how it rates investments relative 

to current performance and schedule variance. Most agencies 

generally concurred with our recommendations and three have taken 

steps to address our recommendation. OMB agreed with our 

recommendation for improving ratings for schedule variance. It 

disagreed with our recommendation to improve how it reflects current 

performance in cost and schedule ratings, but more recently made 

changes to Dashboard calculations to address this while also noting 

challenges in comprehensively evaluating cost and schedule data for 

these investments.

12

 11GAO, Information Technology: OMB Has Made Improvements to Its Dashboard, but 

Further Work Is Needed by Agencies and OMB to Ensure Data Accuracy, 

Specifically, agencies differed on what 

investments they included as an IT investment. Among other things, 

we recommended that OMB clarify its guidance on reporting IT 

investments to specify whether certain types of systems—such as 

GAO-11-262

(Washington, D.C.: Mar. 15, 2011).

12GAO, Information Technology: OMB Needs to Improve Its Guidance on IT Investments, 

GAO-11-826 (Washington, D.C.: Sept. 29, 2011).
Page 10 GAO-14-64 IT Dashboard

space systems—are to be included. OMB did not agree that further 

efforts were needed to clarify reporting in regard to the types of 

systems.

• Subsequently, in November 2011, we noted13

• More recently, in October 2012 we reported that CIOs at six agencies 

rated a majority of investments listed on the federal IT Dashboard as 

low or moderately low risk from June 2009 through March 2012.

that the accuracy of 

investment cost and schedule ratings had improved since our July 

2010 report because OMB had refined the Dashboard’s cost and 

schedule calculations. Most of the ratings for the eight investments we 

reviewed as part of our November 2011 report were accurate, 

although we noted that more could be done to inform oversight and 

decision making by emphasizing recent performance in the ratings. 

We recommended that the General Services Administration comply 

with OMB’s guidance for updating its ratings when new information 

becomes available (including when investments are rebaselined). The 

agency concurred and has since taken actions to address this 

recommendation. Since we previously recommended that OMB 

improve how it rates investments, we did not make any further 

recommendations.

14

 13GAO, IT Dashboard: Accuracy Has Improved, and Additional Efforts Are Under Way to 

Better Inform Decision Making, 

Additionally, two agencies, the Department of Defense and the 

National Science Foundation, rated no investments as high or 

moderately high risk during this time period. Agencies generally 

followed OMB’s instructions for assigning CIO ratings, although the 

Department of Defense’s ratings were unique in reflecting additional 

considerations, such as the likelihood of OMB review. Most of the 

selected agencies reported various benefits associated with producing 

and reporting CIO ratings, such as increased quality of their 

performance data and greater transparency and visibility of 

investments. We recommended that OMB analyze agencies’

investment risk over time as reflected in the Dashboard’s CIO ratings 

and present its analysis with the President’s annual budget 

submission, and that the Department of Defense ensure that its CIO 

ratings reflect available investment performance assessments and its 

risk management guidance. Both agencies concurred with our 

recommendations, and OMB reported on CIO rating trends in the 

GAO-12-210 (Washington, D.C.: Nov. 7, 2011).

14GAO-13-98. 
Page 11 GAO-14-64 IT Dashboard

fiscal year 2014 budget submission; however, the Department of 

Defense has not updated any of its ratings since September 2012.

• Further, we studied OMB’s efforts to help agencies address IT 

projects with cost overruns, schedule delays, and performance 

shortfalls in June 2013.15 In particular, we reported that OMB used 

CIO ratings from the Dashboard, among other sources, to select atrisk investments for reviews known as TechStats.16 OMB initiated 

these reviews in January 2010 to further improve investment 

performance, and subsequently incorporated the TechStat model into 

its 25-point plan for reforming federal IT management.17 We reported 

that OMB and selected agencies had held multiple TechStat sessions 

but additional OMB oversight was needed to ensure that these 

meetings were having the appropriate impact on underperforming 

projects and that resulting cost savings were valid. Among other 

things, we recommended that OMB require agencies to address their 

highest-risk investments and to report on how they validated the 

outcomes. OMB generally agreed with our recommendations, and 

stated that it and the agencies were taking appropriate steps to 

address them.

 15GAO, Information Technology: Additional Executive Review Sessions Needed to 

Address Troubled Projects, GAO-13-524 (Washington, D.C.: June 13, 2013).

16TechStat sessions are face-to-face meetings to terminate, halt, or turnaround IT 

investments that are failing or are not producing results. 

17U.S. Chief Information Officer, 25 Point Implementation Plan to Reform Federal 

Information Technology Management, The White House (Washington, D.C.: Dec. 9, 

2010).
Page 12 GAO-14-64 IT Dashboard

As of August 2013, CIOs at the eight selected agencies rated 198 of their 

244 major investments listed on the Dashboard as low risk or moderately 

low risk. Of the remaining 46 investments, 41 were rated medium risk, 

and 5 were rated high risk or moderately high risk. Historically, over the 

life of the Dashboard from June 2009 to August 2013, low or moderately 

low risk investments accounted for an average of 72 percent of all ratings 

at the eight agencies, medium risk ratings an average of 22 percent, and 

high risk or moderately high risk ratings an average of 6 percent.18

 18As mentioned earlier, CIOs are responsible for updating these rating as soon as new 

information becomes available which will affect an investment’s assessment.

The 

CIO ratings and associated number of investments at each the eight 

agencies as reported on the Dashboard over the past 4 years are 

presented in figure 2.

CIOs at Selected 

Agencies’ Rated 

Majority of IT 

Investments as 

Low or Moderately 

Low Risk
Page 13 GAO-14-64 IT Dashboard

Figure 2: Dashboard CIO Ratings over Time from the Eight Selected Agencies
Page 14 GAO-14-64 IT Dashboard
Page 15 GAO-14-64 IT Dashboard

When comparing the investments’ first and most recent ratings, the 

agencies we reviewed showed generally positive results. Specifically, of 

the 383 total investments listed on the Dashboard from June 2009 to

August 2013 for the selected agencies,19 121 increased (meaning risk is 

lower) and 81 decreased (meaning risk is higher).20

Figure 3: Dashboard Rating Changes over Time and Associated Number of 

Investments from the Eight Selected Agencies between June 2009 and August 2013

Additionally, a 

significant portion of investments had returned to their original rating (74) 

and about one third of investments’ ratings had never changed (107). 

(See fig. 3.)

When considered individually, five of the eight selected agencies—

Agriculture, Commerce, Energy, Treasury, and VA—had more 

investments’ ratings increase than decrease, whereas the other three—

Justice, Transportation, and SSA—had more investments decrease. 

 19This number varies from the 244 described earlier because it takes into account 

investments that the selected agencies are no longer reporting on the Dashboard. The 

following section provides further details.

20CIO’s assign a rating of 1 (high risk) to 5 (low risk) based on the level of risk facing the 

investment. As such, a reduction in risk level is indicated by a higher CIO rating and an 

increase in risk level is indicated by a lower CIO rating. 
Page 16 GAO-14-64 IT Dashboard

Figure 4 presents the net changes in investments’ ratings for each 

agency during the reporting period of June 2009 to August 2013.

Figure 4: Dashboard Rating Changes over Time and Associated Number of Investments by Agency between June 2009 and 

August 2013

The agencies cited additional oversight or program reviews as factors that 

contributed to improved ratings. Furthermore, Agriculture and Commerce 

both attributed improved ratings to enhanced timeliness and sufficiency of 

investment reporting. Both of these agencies factor investment reporting 

into CIO ratings and increased ratings as better performance and risk 

information was provided by investment management. This suggests that 

caution should be exercised when interpreting changing risk levels for 

investments, as rating changes by agencies may not be entirely due to 
Page 17 GAO-14-64 IT Dashboard

investment performance. Agencies also commented that the CIO ratings 

and Dashboard reporting had spurred improved investment management 

and risk mitigation.

Additionally, the total number of investments that the eight agencies 

reported on the Dashboard has varied over time, which impacts the 

number of investments receiving CIO ratings (see fig. 5).

Figure 5: Number of Investments the Eight Selected Agencies Reported on the Dashboard over Time

The variation in the number of investments reported on the Dashboard 

can be attributed, in part, to agencies’ ability to add, downgrade, and 

remove investments throughout the annual federal budget process. 

However, as we concluded in September 2011, OMB’s guidance also did 

not ensure complete reporting of IT investments by agencies.21

Now, 2 years later, and given the continuing lack of clarity in investment 

reporting guidance, agencies have elected to withdraw investments from 

Specifically, we found that OMB’s definition of an IT investment is broad, 

and the 10 agencies we evaluated differed on what systems they included 

as IT investments. For example, 5 of the 10 agencies we reviewed 

consistently considered investments in research and development 

systems as IT, and 5 did not. Consequently, we recommended that OMB 

clarify its guidance on reporting IT investments to specify whether certain 

types of systems—such as space systems—were to be included. OMB 

did not agree that further efforts were needed to clarify reporting in regard 

to the types of systems.

 21GAO-11-826.
Page 18 GAO-14-64 IT Dashboard

the Dashboard that are clearly IT. Because we have continued to identify 

inappropriate reclassifications of IT investments, we continue to believe 

this recommendation has merit. For example, as part of the most recent 

budget process, Energy officials reported several of Energy’s 

supercomputer investments as facilities rather than IT, thus removing 

those investments from the Dashboard and accounting for a portion of the 

recent decrease in investments. Energy officials also stated that OMB 

approved this change. These investments account for $368 million, or 

almost 25 percent, of Energy’s fiscal year 2012 IT spending, and include 

the National Nuclear Security Administration’s Sequoia system, which 

Energy reported as the most powerful computing system in the world as 

of June 2012. According to Energy officials, these investments were 

recategorized because they include both supercomputers and laboratory 

facilities (which are not IT).

As another example, the Deputy Secretary of Commerce issued a 

directive in September 2012 which resulted in the removal of Commerce’s 

satellite investments from the Dashboard. As we found in 2011, 

Commerce only reported the ground systems of a spacecraft as IT 

investments, and not the technology components on the spacecraft 

itself.22

However, these recategorizations run contrary to the Clinger-Cohen Act

of 1996, which specifies requirements for the management of IT and 

defines IT as “any equipment or interconnected system or subsystem of 

equipment, used in the automatic acquisition, storage, analysis, 

evaluation, manipulation, management, movement, control, display, 

switching, interchange, transmission, or reception of data or information 

by the executive agency.”

With this directive, Commerce removed the ground-based 

portions from IT investment reporting as well, accounting for $447 million, 

or 26 percent, of the department’s fiscal year 2012 IT spending. In making 

this decision, Commerce determined that it needed to refocus oversight 

efforts to a more appropriate level and consequently minimized the role of 

the CIO and others in the oversight of satellites. A Commerce official 

stated that Commerce plans to exclude all such investments from the 

department’s fiscal year 2015 IT budget submission.

23

 22

Supercomputers are indisputably IT and 

satellite ground systems also meet the above definition as they receive 

GAO-11-826. 

23Clinger-Cohen Act of 1996 (40 U.S.C. § 11101(6)).
Page 19 GAO-14-64 IT Dashboard

and transmit satellite data. A staff member from the Office of EGovernment stated that OMB could not stop agencies from making such 

recategorizations and that OMB had no control over such agency 

decisions. By gathering incomplete information on IT investments, OMB 

increases the risk of not fulfilling its oversight responsibilities, of agencies 

making inefficient and ineffective investment decisions, and of Congress 

and the public being misinformed as to the performance of federal IT 

investments.

As part of the budget process, OMB is required to analyze, track, and 

evaluate the risks and results of all major IT investments.24

Although intended to be a mechanism to enhance transparency, OMB 

does not update the public version of the Dashboard as it and the 

agencies work to assist in the formulation of the President’s budget 

request. Nevertheless, agencies continue to provide updates during this 

time. For the fiscal year 2013 presidential budget request, the static 

period during which the public saw no updates lasted for 7 months 

(September 2011 to March 2012) and for fiscal year 2014 the period 

lasted for 8 months (September 2012 to April 2013). In total, OMB did not 

update the public Dashboard for 15 of the past 24 months. An Office of EGovernment staff member stated that OMB considers the Dashboard 

information submitted during the budget process to be predecisional and 

will only release it to the public in tandem with the President’s budget. 

However, the CIO rating does not contain any of the information that the 

Dashboard’s Web page for frequently asked questions

The 

Dashboard gives the public access to the same tools and analysis that 

the government uses in conducting this performance oversight. According 

to OMB, Dashboard data are intended to provide a near-real-time 

perspective on the performance of major investments.

25

 24Clinger-Cohen Act of 1996 (40 U.S.C. § 11302(c)).

identifies as 

predecisional, namely: “new” investments, planned IT spending levels in 

the budget year, costs of projects and activities that were completed 

during or after the budget year, and eliminated or downgraded 

investments. Given that the CIO rating does not include such data, the 

rating could be updated independent of the budget process without 

disclosing information that OMB views as predecisional. However, by 

withholding current investment performance information for much of the 

25 http://www.itdashboard.gov/faq.
Page 20 GAO-14-64 IT Dashboard

year, OMB decreases the utility of the Dashboard as a tool for greater IT 

investment oversight and transparency.

Of the 80 selected investments at the eight agencies we reviewed, 53 of 

the CIO ratings were consistent with the risks portrayed in the supporting 

investment performance documents, 20 were partially consistent, and 7 

VA investments were inconsistent. Those that were partially consistent

had few instances of discrepancies during the 12-month period. For 

example, both of Commerce’s inconsistent investments had a 

discrepancy during 1 of the 12 months which we reviewed. As previously 

mentioned, a CIO rating should reflect the level of risk facing an 

investment on a scale from 1 (high risk) to 5 (low risk) relative to that 

investment’s ability to accomplish its goals. However, seven of the eight 

agencies we reviewed had at least one instance wherein the Dashboard’s 

CIO ratings did not consistently reflect the risks identified in agency 

documents. Table 4 summarizes our assessment of the 10 investments at 

each of the selected agencies during the 12-month period from January 

2012 through December 2012 and a discussion of the analysis of each 

agency’s documentation follows the table.

Table 4: Assessment of the Consistency of Selected Agencies’ CIO Ratings with 

Supporting Documentation from January 2012 through December 2012

Agency Rating consistency of each agency’s 10 selected investments 

Agriculture ● ● ◐ ◐ ◐ ◐ ◐ ◐ ◐ ◐

Commerce ● ● ● ● ● ● ● ● ◐ ◐

Energy ● a ● ● ● ● ◐ ◐ ◐ ◐ ◐

Justice ● a ● ● ● ● ● ● ● ● ◐

Transportation ● ● ● ● ● ● ● ● ● ●

Treasury ● ● ● ● ● ● ● ● ● ●

VA ◐ ◐ ◐ ◌ ◌ ◌ ◌ ◌ ◌ ◌

SSA ● ● ● ● ● ● ● ● ● ◐

Key:

●: The investment’s ratings were consistent with supporting documents.

◐: The investment’s ratings were partially consistent with supporting documents.

◌: The investment’s ratings were inconsistent with supporting documents.

Source: GAO analysis of OMB’s Dashboard and agency data.

a

• Agriculture: Eight of the 10 investments at Agriculture had some 

inconsistencies with reported risks. In particular, Agriculture 

experienced technical issues uploading its ratings to the Dashboard in 

These inconsistencies were the result of OMB’s budget process, as discussed below.

Most CIO Ratings

Are Consistent

with Reported

Investment Risk
Page 21 GAO-14-64 IT Dashboard

early 2012, which impacted the ratings of six investments. Agriculture 

officials identified an issue with these investments after observing that 

submitted changes were not reflected on the Dashboard. They 

addressed this issue by working with their contractor to determine the 

cause of the issue, modifying their process to prevent the issue from 

recurring, and establishing procedures to identify future technical 

issues. Additionally, Agriculture’s Financial Management 

Modernization Initiative—one of the six investments that experienced 

the technical issue described above—and Human Resources Line of 

Business’s October ratings should have been lower (higher risk) in 

October 2012. Officials stated that these CIO ratings were updated at 

the beginning of November 2012 and attributed the delays in the 

rating submissions to the annual budget process.

Additionally, Agriculture’s CIO rated the Farm Program Modernization 

investment as moderately low risk until November 2012, despite the 

investment showing indications of significantly increased risk as early 

as May 2012. For example, the number of high and very high risks 

increased every month in a series of briefings from May to July 2012, 

at which point there was 1 high-impact risk and 2 very high-impact 

risks each with an estimated 70 percent probability of occurring. 

Further, in September 2012 a senior management oversight 

committee determined the investment to be off-track and unable to 

resolve the issues. However, the investment remained rated on the 

Dashboard as moderately low risk until November 2012, when it was 

updated to medium risk. As a result, the CIO rating did not reflect the 

results of the oversight committee’s review for 3 months. The 

investment was ultimately given a high-risk rating in December 2012.

• Commerce: Two of the 10 selected Commerce investments differed 

from risk levels identified in underlying documentation for 1 month 

each. In particular, Commerce’s Census - Economic Census and 

Surveys investment should have been green rather than yellow in 

March 2012, and its National Oceanic and Atmospheric Administration 

IT Infrastructure should have been yellow rather than green in June 

2012. In both cases Commerce officials recognized that there were 

problems and took steps to correct them the following month.

• Energy: Five of the 10 selected Energy investments remained listed 

on the Dashboard after the department had recategorized them as 

investment types that would have no longer been displayed. 

Specifically, in October 2012, as part of its annual budget process, 

Energy downgraded its SR Mission Support Systems from major to

non-major and, as discussed previously, changed four of the selected 

supercomputer investments from IT to non-IT. An SR Mission Support 
Page 22 GAO-14-64 IT Dashboard

Systems official explained that Energy downgraded the investment 

after realizing that significant portions of it could be interpreted as 

non-IT under the Federal Acquisition Regulation, and the remainder 

did not meet the department’s threshold for being a major IT 

investment. Further, although Energy made these decisions in 

October 2012, OMB officials explained that they do not publish 

budgetary data such as these until the release of the President’s 

budget, which did not happen until April 2013 at which time the 

Dashboard reflected Energy’s changes.

• Justice: One of the 10 selected Justice investments remained on the 

Dashboard after the department recategorized it as an investment 

type that would have no longer been displayed, similar to Energy’s 

situation. Specifically, Justice downgraded its Law Enforcement 

Wireless Communications investment from major to non-major as part

of its annual budget process in July 2012, but the Dashboard did not 

reflect the change until April 2013. As noted above, OMB officials 

explained that they do not publish budgetary data until the release of 

the President’s budget, which occurred in April 2013.

• Transportation: There were no inconsistencies between CIO ratings 

and reported investments’ risks. Continuing to report consistent and 

timely data should help ensure that Transportation’s Dashboard’s CIO 

ratings are accurate.

• Treasury: We did not identify inconsistencies between CIO ratings 

and reported investments’ risks. Such attention to reporting consistent 

and timely data should help ensure the accuracy of the Dashboard’s 

CIO ratings.

• VA: Seven of the 10 selected investments were never updated during 

our evaluation period, and 3 were updated once. VA did not update its 

ratings because it did not have the ability to automatically submit 

ratings from September 2011 to March 2013, a period of 19 months. 

Instead, VA elected to build the capability to submit ratings to the 

Dashboard, rather than purchase this capability. VA completed 

development of this tool in March 2013, at which point it resumed 

making the required updates to CIO ratings on the Dashboard. For the 

3 that VA updated during this period, VA used a manual budget 

process to update their ratings in September 2012.

There were also several discrepancies within the ratings depicted on 

VA’s internal Dashboard, known as the Project Management and 

Accountability System. Specifically, we found 19 instances where 

negative performance management review results did not change the 

system’s ratings and 5 instances where missed deliverables were not 
Page 23 GAO-14-64 IT Dashboard

recorded.26 For example, we identified 8 separate VA performance 

management reviews of the Medical Legacy investment in 2012 which 

did not impact the associated ratings, and 2 missed deliverables for 

the same investment which were not recorded. Neglecting to include 

such performance deviations in the system upon which VA bases its 

CIO ratings could have resulted in a lower (increased) risk rating. 

Further, VA’s Office of the Inspector General also identified issues 

with this system in August 2011,27

• SSA: The CIO rating on the Dashboard differed from the risks 

portrayed in underlying documentation for 1 of the 10 selected SSA 

investments. In particular, SSA’s Citizen Access Routing Enterprise 

through 2020 investment was described as both having zero percent 

schedule variance in project management reports, but also being 

more than a year behind schedule and unable to determine a 

milestone date in a CIO review. According to SSA officials from the 

Office of Systems, this is because of the new milestones and 

corresponding baseline that all SSA investments create at the start of 

every fiscal year based on annual funding amounts received from 

SSA’s Strategic Information Technology Assessment and Review 

process. SSA describes this process further in its Information 

Resources Management Strategic Plan,

including a lack of well-established 

key management controls to ensure data reliability, verify project 

compliance, and track project costs. VA agreed with those findings 

and provided steps the department would take to address the 

identified weaknesses. To that end, VA’s February 2013 guidance for 

this system requires the inclusion of negative performance

management reviews and missed deliverables when considering 

investment risk.

28 which states that SSA 

makes adjustments each year based on current goals, objectives, and 

measures. Such changes, known as rebaselines,29

 26VA tracks missed increment deliverable dates by recording a “strike” against a project. 

VA may pause or close projects that receive three strikes.

erase past cost 

27VA, Audit of the Project Management Accountability System Implementation, 10-03162-

262 (Aug. 29, 2011).

28SSA, Information Resources Management Strategic Plan: FY 2012-2016 (May 2012).

29At times, a project’s cost, schedule, and performance goals—known as its baseline—are 

modified to reflect changed development circumstances. These changes—called a 

rebaseline—can be done for valid reasons, but can also be used to mask cost overruns 

and schedule delays. 
Page 24 GAO-14-64 IT Dashboard

and schedule variances, prevent the agency from monitoring year-toyear investment performance, and make it difficult to estimate and 

understand life-cycle costs. While this only affected one investment 

we reviewed, it has the potential to impact SSA’s entire IT investment 

portfolio. As described in our Cost Estimating and Assessment Guide, 

the purpose of such baseline changes should be to restore 

management’s control of the remaining effort by providing a 

meaningful basis for performance management. Further, these 

changes should be infrequent, and recurrent changes may indicate 

that the scope is not well understood or simply that program 

management is unable to develop realistic estimates.30

While agencies experienced several issues with reporting the risk of their 

investments, such as technical problems and delayed updates to the 

Dashboard, the CIO ratings were mostly or completely consistent with 

investment risk at seven of the eight selected agencies. Additionally, the 

agencies had already addressed several of the discrepancies that we 

identified. The final agency, VA, did not update 7 of its 10 selected 

investments because it elected to build, rather than buy, the ability to 

automatically update the Dashboard, and has now resumed updating all 

investments. To their credit, agencies’ continued attention to reporting the 

risk of their major investments supports the Dashboard’s goal of providing 

transparency and oversight of federal IT investments. Nevertheless, the 

rating issues that we identified with performance reporting and annual 

baselining, some of which are now corrected, serve to highlight the need 

for agencies’ continued attention to the timeliness and accuracy of 

submitted information, in order to allow the Dashboard to continue to fulfill 

its stated purpose.

Regarding 

SSA, frequent rebaselining increases the risk that its investments 

have undetected cost or schedule variances that will impact the 

success of the associated investment.

 30GAO, Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: Mar. 2, 2009).
Page 25 GAO-14-64 IT Dashboard

We have previously concluded that, consistent with government and 

industry best practices, including our own guidance on IT investment 

management, agencies’ highest-risk investments should receive 

additional management attention.31 In particular, agency leaders should 

use data-driven reviews as a leadership strategy to drive performance 

improvement.32

 31GAO, Information Technology Investment Management: A Framework for Assessing 

and Improving Process Maturity, 

Correspondingly, OMB requires reviews, known as 

TechStat sessions, to enable the federal government to intervene to 

turnaround, halt, or terminate IT projects that are failing or are not 

producing results. Of the 80 investments we reviewed, there were 8 

investments at four of the eight selected agencies that received a red 

(high or moderately high risk) rating in 2012. See figure 6 for a list of 

those at-risk investments and the associated CIO ratings.

GAO-04-394G (Washington, D.C.: March 2004). 

32GAO, Managing for Results: Data-Driven Performance Reviews Show Promise But 

Agencies Should Explore How to Involve Other Relevant Agencies, GAO-13-228

(Washington, D.C.: Feb. 27, 2013). 

Highest-Risk 

Investments Were 

Appropriately 

Addressed by 

Selected Agencies
Page 26 GAO-14-64 IT Dashboard

Figure 6: Dashboard Ratings in 2012 for the Eight At-Risk Investments

Of the eight investments receiving a red rating in 2012, the agencies 

reviewed seven using tools such as TechStat sessions,33

 33A TechStat is a face-to-face, evidence-based accountability review of an IT investment 

that enables the federal government to intervene to turnaround, halt, or terminate projects 

that are failing or are not producing results. OMB began leading TechStat sessions on 

agency IT projects in 2010, and subsequently required federal agencies and bureaus to 

hold them.

department 

investment review boards, and other high-level reviews. Each of these 

resulted in action items intended to improve performance. The final 

investment, Agriculture’s Human Resources Line of Business: Service 
Page 27 GAO-14-64 IT Dashboard

Center was scheduled to be reviewed using a TechStat session, but in 

August 2013 officials from Agriculture’s Office of the CIO stated that this 

investment was in the process of going through a rebaseline. As noted 

earlier, such changes should be infrequent and reinforce the need for 

agencies to review the highest-risk investments to ensure that root 

causes of baseline changes are effectively addressed.

Each of these agencies we reviewed had similar approaches to 

identifying and conducting these reviews. The agencies with red-rated 

investments in 2012—Agriculture, Commerce, Justice, and 

Transportation—monitored investment performance, identified troubled 

investments using a variety of criteria, reviewed investments in need of 

attention, and assigned and tracked action items. For example, both 

Agriculture and Commerce identified investments for review using the 

Dashboard’s CIO ratings. All four agencies further ensured that 

investments implemented corrective actions resulting from management 

reviews. For example, as a result of its November 2012 TechStat, 

Agriculture leaders assigned 17 action items to the Farm Program 

Modernization investment, which were to be completed by January 2013. 

Additionally, the four remaining agencies without red-rated investments—

Energy, Treasury, VA, and SSA—have documented processes which 

provide comparable monitoring and oversight capabilities. For example, 

VA’s process relies on missed deliverable dates as the key requirement 

to hold a review, after which senior management makes a determination 

as to the future of the investment. Alternatively, Energy reviews all major 

IT investments on a quarterly basis and requires those which fall outside 

expected thresholds to complete corrective actions. As such, all of the 

agencies have focused management attention on troubled investments 

and once identified, have established and followed through on clear 

action items to turn around or terminate such investments.

Since its inception in 2009, the IT Dashboard has increased the 

transparency of the performance of major federal IT investments. Its CIO 

ratings, in particular, have improved visibility into changes in the risk level 

of investments over time, as well as agencies’ ability to accomplish their 

goals. To that end, over the past 4 years, the agencies we reviewed have 

reported lower risk for more than one quarter of all their investments. 

However, the effectiveness of the Dashboard depends on the quality of 

the information that agencies submit. We previously recommended that 

OMB clarify guidance on whether certain types of systems should be 

included in IT reporting, but OMB did not agree. Agencies’ subsequent 

decisions to remove investments from the Dashboard by changing 

Conclusions
Page 28 GAO-14-64 IT Dashboard

investments’ categorizations represent a troubling trend toward 

decreased transparency and accountability. Thus, we continue to believe 

that our prior recommendation remains valid and should be implemented. 

Additionally, OMB’s annual freeze of the Dashboard for as long as 8 

months negates one of the primary purposes of this valuable tool—to 

facilitate transparency and oversight of the government’s billions of 

dollars in IT investments.

Beyond the transparency they promote, CIO ratings present an 

opportunity to improve the data and processes agencies use to assess 

investment risk. Each of the agencies we examined had established such 

processes, and most of the resulting Dashboard ratings were consistent 

with underlying documentation. While two agencies, Transportation and 

Treasury, had ratings that accurately reflected their investments’

supporting documentation, other agencies’ Dashboard ratings were 

inconsistent with the actual risk of the associated investment. These 

inconsistencies—such as inaccurate reflection of negative performance, 

questionable decisions about the recategorization of investments, and 

too-frequent changes to performance baselines—highlight the continued 

need for agencies to populate the Dashboard with an accurate 

representation of the full breadth and health of their investments. In 

providing this information, agencies will help the Dashboard accomplish 

its goal of providing transparency and oversight of millions of dollars in 

federal IT investments.

To better ensure that the Dashboard provides meaningful ratings and 

reliable investment data, we are recommending that the Director of OMB 

direct the Federal CIO to make accessible regularly updated portions of 

the public version of the Dashboard (such as CIO ratings) independent of 

the annual budget process.

In addition, to better ensure that the Dashboard provides accurate ratings, 

we are making three recommendations to the heads of three of the 

selected agencies. Specifically, we are recommending that:

• The Secretary of Commerce direct the department CIO to ensure that 

the department’s investments are appropriately categorized in 

accordance with existing statutes and that major IT investments are 

included on the Dashboard.

Recommendations for 

Executive Action
Page 29 GAO-14-64 IT Dashboard

• The Secretary of Energy direct the department CIO to ensure that the 

department’s investments are appropriately categorized in 

accordance with existing statutes and that major IT investments are 

included on the Dashboard.

• The Commissioner of the SSA direct the CIO to revise the agency’s 

investment management processes to ensure that they are consistent 

with the baselining best practices identified in our published guidance 

on cost estimating and assessment.

We received comments on a draft of this report from OMB and all eight 

departments and agencies in our review. OMB neither agreed nor 

disagreed with our recommendation; Agriculture agreed with the report’s

findings; Commerce disagreed with our recommendation; Energy 

concurred with our recommendation with one exception related to 

supercomputers; Justice, Treasury, and Transportation neither agreed 

nor disagreed with the report’s findings; VA agreed with the report’s 

findings; and SSA agreed with our recommendation. Each agency’s 

comments are discussed in more detail below.

• In written comments, the Federal CIO neither agreed nor disagreed 

with our recommendation for OMB to make accessible regularly 

updated portions of the public version of the Dashboard (such as CIO 

ratings) independent of the annual budget process. However, OMB 

also noted that the manner in which agencies submit Dashboard 

information to OMB makes it difficult to separate materials it believes 

are predecisional or deliberative. Despite this, OMB agreed to explore 

whether it would be practical to make the Dashboard more accessible, 

and to consider how it could separate predecisional or deliberative 

materials. We support OMB in its efforts to increase public 

transparency and accountability.

OMB continued to disagree with the 2011 recommendation that we 

believe continues to have merit, related to the clarity of its guidance 

on whether certain types of systems should be included in IT 

reporting. OMB believes that the existing guidance is appropriate and 

does not have plans to review it. However, as noted earlier in this 

report, we believe that the recommendation is appropriate because 

the existing guidance does not address key categories of IT 

investments where we continued to find inconsistencies.

OMB also disputed our characterization of two issues. First, OMB 

noted that up-to-date Dashboard information is available to the 

agencies and OMB during the budget development period, who use it 

as a tool for investment oversight and decision making. While we 

Agency Comments 

and Our Evaluation
Page 30 GAO-14-64 IT Dashboard

acknowledge that OMB and federal agencies continue to have access 

to the Dashboard during the budget process, the public display of 

these data is intended to allow other oversight bodies, including 

Congress and the general public, to hold government agencies 

accountable for progress and results. Secondly, OMB stated that it 

began using the IT Dashboard to help identify at-risk investments 

starting with its launch in June 2009, rather than 2010. We modified 

the report to reflect OMB’s statement. OMB also provided technical 

comments, which we have incorporated in the report as appropriate.

OMB’s written comments are provided in appendix III.

• In written comments, Agriculture’s CIO stated that the department 

concurred with the content of the report and had no comments. 

Agriculture’s written comments are provided in appendix IV.

• In written comments, Commerce agreed with most of the report’s 

findings but disagreed with our recommendation to ensure that the 

department’s investments are appropriately categorized in 

accordance with existing statutes and that major IT investments are 

included on the Dashboard. Specifically, Commerce stated that 

although it is no longer reporting publicly on its satellite ground system

investments through the Dashboard, neither the department CIO nor 

OMB has relinquished its oversight role. Moreover, Commerce stated 

that it is reviewing its satellites more frequently and in more detail; as 

an example, the department noted that Commerce’s CIO conducts 

monthly Dashboard-like assessments that cover the status of the 

satellite investments. However, regardless of these additional efforts, 

the removal of the satellite investments from the Dashboard prevents 

the public display of these data intended to allow OMB and other 

oversight bodies, including Congress, to hold the government 

agencies accountable for progress and results. Additionally, as 

discussed in this report, Commerce’s recategorization of its satellite 

ground system investments is contrary to the definition of IT as set 

forth in the Clinger-Cohen Act. Based on these facts, we continue to 

believe that our recommendation to Commerce that its CIO ensure 

that the department’s investments are appropriately categorized in 

accordance with existing statutes and that major IT investments are 

included on the Dashboard has merit and should be implemented. 

Commerce’s written comments are provided in appendix V. The 

department also provided technical comments related to our 

characterization of the department’s rating process, which we have 

incorporated in the report as appropriate.

• In written comments, Energy concurred, but with a comment 

regarding our recommendation to ensure that the department’s 
Page 31 GAO-14-64 IT Dashboard

investments are appropriately categorized in accordance with existing 

statutes and that major IT investments are included on the 

Dashboard. Specifically, while agreeing that Energy should ensure 

that its IT investments are appropriately categorized, the department 

stated that it would continue to report investments that it believes 

should be categorized and managed on the Dashboard, noting that 

supercomputers would be an exception to this policy. However, this 

exception runs contrary to the finding on which this recommendation 

is based, namely that Energy removed $368 million in investments 

from the Dashboard, including a supercomputer reported in 2012 as

the most powerful in the world. While Energy agreed to partner with 

OMB to develop an alternate mechanism to track and report 

supercomputer investments to OMB, it is not clear whether this 

information will be publicly available. Additionally, Energy’s 

recategorization of its supercomputer investments is contrary to the 

definition of IT as set forth in the Clinger-Cohen Act, as discussed in 

this report. Therefore, we disagree that the department’s planned 

actions adequately address existing requirements for open and 

transparent reporting of major IT investments, and we stand by our 

assessment that Energy’s CIO should ensure that the department’s 

investments are appropriately categorized in accordance with existing 

statutes and that major IT investments are included on the 

Dashboard. Energy’s written comments are provided in appendix VI.

• In comments provided via e-mail on November 6, 2013, a 

representative of Justice’s Justice Management Division stated that 

the department had no comments.

• In comments provided via e-mail on November 1, 2013, 

Transportation’s Deputy Director of Audit Relations stated that the 

department had no comments. The department also provided 

technical comments, which we have incorporated in the report as 

appropriate.

• In written comments, the Treasury’s CIO stated that the department 

had no comments. Treasury’s written comments are provided in 

appendix VII.

• In written comments, VA’s Chief of Staff stated that the department 

generally agreed with the findings of the report and provided general 

comments in which it confirmed that issues identified in the report had 

been addressed. Specifically, VA confirmed that the department has 

implemented the capability to automatically submit CIO ratings to the 

Dashboard and now uses more specific data to generate the CIO 

rating, such as the number of “red flags” associated with an 

investment or the number of TechStat reviews held. Finally, while 
Page 32 GAO-14-64 IT Dashboard

stating that the department will continue its efforts to improve timely 

and accurate reporting to the Dashboard, VA also noted that it is 

important to recognize that the department manages investment 

delivery at the project level in comparison to the investment level 

found in the Dashboard. VA’s written comments are provided in 

appendix VIII.

• In written comments, SSA’s Deputy Chief of Staff agreed with our 

recommendation to revise the agency’s investment management 

processes to ensure that they are consistent with the baselining 

practices, and discussed planned actions to address it. SSA’s written 

comments are provided in appendix IX.

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies to interested congressional 

committees; the Secretaries of Agriculture, Commerce, Energy, 

Transportation, Treasury, Veterans Affairs, Attorney General of the United 

States, Acting Commissioner of Social Security, the Director of the Office 

of Management and Budget; and other interested parties. In addition, the 

report will be available at no charge on the GAO website at 

http://www.gao.gov.

If you or your staffs have any questions on matters discussed in this 

report, please contact me at (202) 512-9286 or pownerd@gao.gov. 

Contact points for our Offices of Congressional Relations and Public 

Affairs may be found on the last page of this report. GAO staff who made 

major contributions to this report are listed in appendix X.

David A. Powner

Director, Information Technology

Management Issues
Appendix I: Objectives, Scope, and 

Methodology

Page 33 GAO-14-64 IT Dashboard

Our objectives for this engagement were to (1) characterize the Chief 

Information Officer (CIO) ratings for selected federal agencies’

information technology (IT) investments as reported on the Dashboard 

over time, (2) determine the extent to which selected agencies’ CIO 

ratings are consistent with reported investment risk, and (3) determine the 

extent to which selected agencies are addressing at-risk investments.

To address our first objective, we selected the eight agencies with the 

most reported major IT spending in fiscal year 2012, after excluding 

agencies reviewed in our most recent Dashboard report.1 The eight 

agencies are the Departments of Agriculture (Agriculture), Commerce 

(Commerce), Energy (Energy), Justice (Justice), Transportation 

(Transportation), the Treasury (Treasury), and Veterans Affairs (VA), and 

the Social Security Administration (SSA). The results in this report 

represent only these agencies. We downloaded and examined the 

Dashboard’s CIO ratings for all major investments at the eight agencies 

(a total of 383 investments reported by these agencies).2

 1

GAO, Information Technology Dashboard: Opportunities Exist to Improve Transparency 

and Oversight of Investment Risk at Select Agencies, 

To characterize 

the numbers and percentages of major IT investments at each risk level 

at each of our subject agencies, we analyzed, summarized, and—where 

appropriate—graphically depicted average CIO ratings for investments by 

agencies over time during the period from June 2009 to August 2013. 

Specifically, we compared each investment’s first and last CIO ratings 

(including those investments that were not on the Dashboard at its 

inception, and those that were downgraded or eliminated) and 

summarized the data by agency. To describe whether CIO ratings 

indicated higher or lower investment risk over time, we calculated the 

numbers and percentages of investments (by agency and collectively for 

all the agencies) that maintained a constant rating over the entire 

performance period, and those that experienced a change to their CIO

rating in at least one rating period. Then we analyzed the subset of 

investments that experienced at least one changed rating and compared 

the first CIO rating with the latest CIO rating (no later than August 2013) 

to determine the numbers and percentages of investments (by agency 

GAO-13-98 (Washington, D.C.: Oct. 

16, 2012).

2

We did not independently evaluate the ratings of every investment reported by agencies. 

However, we determined that they were sufficiently reliable for the purposes of our 

objectives by confirming with each agency that the ratings that we downloaded from the IT 

Dashboard were complete, accurate, and reflected the data they had reported to OMB. 

Appendix I: Objectives, Scope, and 

Methodology
Appendix I: Objectives, Scope, and 

Methodology

Page 34 GAO-14-64 IT Dashboard

and collectively for all the agencies) that experienced a net rating 

increase, a net rating decrease, or no net change. We also reviewed 

investments which had been removed from the Dashboard due to 

recategorization and compared their definitions to the Clinger-Cohen 

Act’s3

To accomplish the second objective, we selected the top 10 major 

investments at each of the eight selected agencies (for a total of 80 

investments), with the highest reported IT spending in fiscal year 2012. 

We reviewed investment documentation (such as executive-level 

briefings, operational analyses, and TechStat results) and interviewed 

officials from each of the agencies to understand how they rate and 

monitor investments.

definition of IT. We presented our results to each agency and the 

Office of Management and Budget (OMB) and solicited their input, 

explanations for the results, and additional corroborating documentation, 

where appropriate.

4 Within these artifacts, we identified representations 

of investment risk and performance (such as cost and schedule 

variances), which could impact the success of the associated investment. 

We then organized the results by month, and compared these results to 

the relevant investment Dashboard CIO ratings for calendar year 2012.5

Finally, for our third objective, we reviewed and described each of the 

eight selected agencies’ processes used to address the highest-risk 

major investments (such as TechStat guides and investment review 

board results). Among the 80 investments selected for the second 

objective, we identified investments which had received a “red” (high or 

We then evaluated whether each investment’s monthly CIO rating was 

consistent with reported investment risks, and categorized each 

investment by the number of months which were inconsistent. 

Investments which were consistent with underlying documentation for 

every month in 2012 were categorized as “consistent,” those which were 

inconsistent for 1 or more months were categorized as “partially 

inconsistent,” and those with inconsistencies in each month were 

“inconsistent.”

 3

Clinger-Cohen Act of 1996 (40 U.S.C. § 11101(6)).

4

Because we were only evaluating the consistency of Dashboard ratings with reported 

risk, we did not evaluate the accuracy of the investment documentation.

5

We elected to review the investments’ ratings during calendar year 2012 so that our 

assessment would span multiple fiscal years.
Appendix I: Objectives, Scope, and 

Methodology

Page 35 GAO-14-64 IT Dashboard

moderately high) rating on the Dashboard during 2012. We reviewed the 

processes used to address these investments and interviewed relevant 

officials. We also examined the results of performance reviews and 

interviewed officials to determine whether the agencies implemented the 

appropriate risk management processes to oversee and review the 

selected investments.

We conducted this performance audit from February to December 2013 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.
Appendix II: Selected Investments

Page 36 GAO-14-64 IT Dashboard

Below is the list of the investments that are included in this review, as well 

as the reported fiscal year 2012 IT spending.

Table 5: Investments Selected for Review and Associated Fiscal Year 2012 Spending

Agency Investment title IT spending ($M)

Agriculture Forest Service Computer Base 256.5 

Human Resources Line of Business: Service Center 138.3 

Farm Program Modernization 112.4 

Comprehensive Loan Program 64.0 

Financial Management Modernization Initiative 63.1 

Public Health Data Communications Infrastructure System 53.3 

Public Safety Land Mobile Radio System 43.8 

Animal and Plant Health Inspection Service Enterprise Infrastructure 34.0 

Optimized Computing Environment 30.6 

Natural Resource Manager 30.4 

Commerce Geostationary Operational Environmental Satellite R Series Ground Segment 257.0 

United States Patent and Trademark Office Network and Security Infrastructure 221.4 

Joint Polar Satellite System Ground System 153.6 

National Oceanic and Atmospheric Administration IT Infrastructure 132.6 

Census IT Infrastructure 130.0 

Census Economic Census and Surveys 74.9 

Advanced Weather Interactive Processing System 63.3 

Consolidated Herbert C. Hoover Building IT Infrastructure 56.5 

United States Patent and Trademark Office Patents End-to-End: Software Engineering 46.6 

National Climatic Data Center 42.6 

Energy Consolidated Infrastructure, Office Automation, and Telecommunications Program 967.9 

Sequoia Platform 101.9 

Oak Ridge National Laboratory, Leadership Computing Facility - Direct Mission 94.0 

Argonne National Lab, Leadership Computing Facility - Direct mission 62.2 

Lawrence Berkeley National Lab, National Energy Research Scientific Computing Center -

Direct mission

57.8 

SR Mission Support Systems 44.6 

Lawrence Berkeley National Lab, Energy Sciences Network - Direct Mission 34.5 

Identity, Credential, and Access Management 33.3 

Office of the Chief Financial Officer iManage 29.1 

Simulation-Based Engineering User Center 17.5 

Appendix II: Selected Investments
Appendix II: Selected Investments

Page 37 GAO-14-64 IT Dashboard

Agency Investment title IT spending ($M)

Justice Federal Bureau of Investigation (FBI) Next Generation Identification 220.5 

Justice Management Division Law Enforcement Wireless Communications 87.0 

Justice Management Division Unified Financial Management System 83.8 

Drug Enforcement Administration Firebird 82.9 

FBI Data Integration and Visualization System 59.0 

FBI Data Centers 57.1 

FBI Sentinel 46.1 

FBI National Crime Information Center 45.1 

FBI Terrorist Screening System 43.7 

FBI Digital Collection 43.2 

Transportation Federal Aviation Administration (FAA) Automatic Dependent Surveillance-Broadcast 301.5 

FAA En Route Automation Modernization 271.3 

FAA Data Communications NextGen Support 146.2 

FAA Telecommunications Infrastructure 143.0 

FAA Terminal Automation Modernization and Replacement Phase III 136.7 

FAA Wide Area Augmentation System 123.4 

FAA Oceanic Automation System: Advanced Technologies and Oceanic Procedures 102.7 

FAA Terminal Automation Replacement System 88.6 

FAA System Wide Information Management 74.3 

FAA Regulation and Certification Infrastructure for System Safety 60.0 

Treasury Internal Revenue Service (IRS) Main Frames and Servers Services and Support 449.3 

IRS Telecommunications Systems and Support 218.6 

IRS Customer Account Data Engine 2 210.3 

IRS End User Systems and Services 202.0 

Electronic Federal Tax Payment System 70.9 

Mainframe and Servers Support and Services 69.9 

Treasury Enterprise Identity, Credential, and Access Management 69.3 

Financial Management Service IT Infrastructure Mainframe and Servers Services and 

Support

59.7 

IRS Modernized e-File 49.1 

Retail Securities Services 45.9 

VA Medical IT Support 1,070.2 

Enterprise IT Support 563.5 

Benefits IT Support 199.8 

Corporate IT Support Enterprise Cyber Security and Privacy 172.9 

Medical Legacy 155.6 

Benefits 21st Century Paperless Delivery of Veterans Benefits 140.4 
Appendix II: Selected Investments

Page 38 GAO-14-64 IT Dashboard

Agency Investment title IT spending ($M)

InterAgency 21st Century-One Vet 136.9 

Medical 21st Century Development Core 85.5 

Benefits 21st Century Education 65.2 

InterAgency 21st Century Veterans Interoperability 52.0 

SSA Infrastructure - Data Center 407.4 

Infrastructure - Office Automation 221.9 

Infrastructure - Telecommunications 204.6 

Disability Case Processing System 68.5 

Telephone Systems Replacement Project 51.9 

Intelligent Disability 50.5 

Disability Determination Services Automation 31.1 

Citizen Access Routing Enterprise through 2020 25.1 

National Support Center 23.0 

Earnings Redesign 17.8 

Source: GAO analysis of data from OMB’s IT Dashboard.
Appendix III: Comments from the

Office of Management and Budget

Page 39 GAO-14-64 IT Dashboard

Appendix III: Comments from the

Office of Management and Budget
Appendix III: Comments from the

Office of Management and Budget

Page 40 GAO-14-64 IT Dashboard
Appendix III: Comments from the

Office of Management and Budget

Page 41 GAO-14-64 IT Dashboard
Appendix III: Comments from the

Office of Management and Budget

Page 42 GAO-14-64 IT Dashboard
Appendix IV: Comments from the Department 

of Agriculture

Page 43 GAO-14-64 IT Dashboard

Appendix IV: Comments from the 

Department of Agriculture
Appendix V: Comments from the

Department of Commerce

Page 44 GAO-14-64 IT Dashboard

Appendix V: Comments from the

Department of Commerce
Appendix V: Comments from the

Department of Commerce

Page 45 GAO-14-64 IT Dashboard
Appendix V: Comments from the

Department of Commerce

Page 46 GAO-14-64 IT Dashboard
Appendix VI: Comments from the Department 

of Energy

Page 47 GAO-14-64 IT Dashboard

Appendix VI: Comments from the 

Department of Energy
Appendix VII: Comments from the Department of the 

Treasury

Page 48 GAO-14-64 IT Dashboard

Appendix VII: Comments from the 

Department of the Treasury
Appendix VIII: Comments from the Department 

of Veterans Affairs

Page 49 GAO-14-64 IT Dashboard

Appendix VIII: Comments from the 

Department of Veterans Affairs
Appendix VIII: Comments from the Department 

of Veterans Affairs

Page 50 GAO-14-64 IT Dashboard
Appendix VIII: Comments from the Department 

of Veterans Affairs

Page 51 GAO-14-64 IT Dashboard
Appendix IX: Comments from the

Social Security Administration

Page 52 GAO-14-64 IT Dashboard

Appendix IX: Comments from the

Social Security Administration
Appendix IX: Comments from the

Social Security Administration

Page 53 GAO-14-64 IT Dashboard
Appendix X: GAO Contact and Staff 

Acknowledgments

Page 54 GAO-14-64 IT Dashboard

David A. Powner, (202) 512-9286 or pownerd@gao.gov

In addition to the contact named above, individuals making contributions 

to this report included Dave Hinchman (Assistant Director), Rebecca 

Eyler, Mike Mithani, Kevin Walsh, and Shawn Ward.

Appendix X: GAO Contact and Staff 

Acknowledgments

GAO Contact 

Staff 

Acknowledgments

(311285)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.