Federal Chief Information Officers: Reporting to OMB Can Be Improved by Further Streamlining and Better Focusing on Priorities
Released: 2015-04-02T00:00:00

Letter

Background

Over the years, Congress has enacted various laws in an attempt to improve the government’s management of its IT resources. In doing so, it has provided OMB with broad IT management and oversight responsibilities and given agencies a wide range of IT-related responsibilities. With regard to CIO responsibilities relative to IT management, we have previously identified major areas that are either statutory requirements or are critical to effective IT management.areas include: IT strategic planning: Plans for, among other things, using IT to help agencies improve the productivity, efficiency, and effectiveness of their business processes with the overall goal of achieving and supporting agency missions.

Capital planning and investment management: The process of selecting, controlling, and evaluating IT investments to produce business value, reduce investment-related risks, and increase accountability and transparency in the investment decision-making process. IT security: Establishment of a risk-based program that ensures agency-wide compliance with requirements to protect information and systems, including implementing requisite controls that prevent, limit, or detect access to computer networks, systems, or information.

Systems acquisitions, development, and integration: Obtain the skilled staff, disciplined processes, and tools necessary to develop and acquire IT system capabilities on time and within budget, including ensuring such capabilities interoperate as intended with existing (legacy) systems.

E-government initiatives: A wide range of activities across the federal government involving the use of the Internet and other emerging technologies to improve public access to government information and services.

To carry out their responsibilities, OMB (including its Office of E- Government and Information Technology, headed by the Federal CIO) issues directives to the agencies such as circulars, memorandums, and reporting instructions; these directives contain requirements for agency CIOs to, among other things, report on their IT activities. For example, OMB Circular A-11 requires agencies to provide information related to their IT investments, including agency exhibit 53s and capital asset plans and business cases (called exhibit 300s). In addition, in December 2011, OMB issued a memorandum that outlines the Federal Risk and Authorization Management Program guidance for agency adoption and use of cloud services.

For each reporting requirement, OMB typically identifies how agencies are to transmit the information. In particular, OMB operates and utilizes the following web-based systems that the agencies are to use to transmit their information:

CyberScope: Standardizes manual and automated data inputs for reporting on Federal Information Security Management Actcompliance and agency privacy programs. Integrated Data Collection: Allows reporting of structured information, including agency progress in meeting IT strategic goals, objectives and metrics, as well as cost savings and avoidances resulting from IT management actions. These data include information previously reported by agencies as well as data which agencies shall report on every 3 months. Updates are to be made on the last day of February, May, August, and November of subsequent fiscal years.

MAX Portal: Utilized by federal agencies to enter data and upload documentation related to a variety of reporting activities, including data required for the President’s Budget and Mid-Session review, and federal IT investment information.

Federal IT Dashboard: Allows federal agencies to upload cost, schedule, and performance data on agency major IT investments.

In addition to the reporting system mechanisms listed above, OMB occasionally requires agencies to provide information by e-mail (usually for ad hoc requests) or directs agencies to post information on their websites. For example, with regard to an OMB reporting requirement on open data policies, OMB directs the agencies to post this information on their websites.

To help CIOs prioritize their various roles and responsibilities, OMB has directed CIOs to focus their efforts on the following:

Governance. CIOs should have responsibility over the entire IT portfolio for the agency, including driving the investment review process for IT investments, working with CFOs and Chief Acquisition Officers to ensure IT portfolio analysis is part of the yearly budget process, and leading TechStat sessions.

Commodity IT. CIOs should focus on eliminating duplication in commodity IT services (e.g., data centers, e-mail, and web infrastructure) and rationalize their agency’s IT investments, including using shared services as a provider instead of standing up separate services.

Program management. CIOs should improve the overall management of federal IT projects by identifying, recruiting, and hiring top IT program management talent and be accountable for the performance of agency IT program managers. Information security. CIOs should have the authority and primary responsibility for implementing an agency-wide information security program, including having continuous monitoring and standardized risk assessment processes.

In addition, OMB has implemented a series of initiatives—commonly referred to by the agency as IT reforms—to, among other things, improve the oversight of underperforming investments, more effectively manage IT, and address duplicative investments. The initiatives include the following:

TechStat reviews. In January 2010, the Federal CIO began leading TechStat sessions—face-to-face meetings to terminate or turn around IT investments that are failing or are not producing results. These meetings involve OMB and agency leadership and are intended to increase accountability and transparency and improve performance. Subsequently, OMB empowered agency CIOs to hold their own TechStat sessions within their respective agencies. OMB has reported that these efforts to improve management and oversight of IT investments have resulted in almost $4 billion in savings.

Federal Data Center Consolidation Initiative. Concerned about the growing number of federal data centers, the Federal CIO (in February 2010) established the Federal Data Center Consolidation Initiative. The initiative’s four high-level goals were to promote the use of “green IT” by reducing the overall energy and real estate needs of government data centers; reduce the cost of data center hardware, software, and operations; increase the overall IT security posture of the government; and shift IT investments to more efficient computing platforms and technologies. OMB estimates that the initiative has the potential to provide about $3 billion in savings by the end of 2015.

PortfolioStat. In order to eliminate duplication, move to shared services, and improve portfolio management processes, OMB (in March 2012) launched its PortfolioStat initiative. It required agencies to conduct annual agency-wide IT portfolio reviews to, among other things, reduce commodity IT spending and demonstrate how IT investments align with agency mission and business functions. PortfolioStat is designed to assist agencies in (1) assessing the current maturity of their IT investment management process, (2) making decisions on eliminating duplicative investments, and (3) moving to shared solutions in order to maximize the return on IT investments across the portfolio. OMB estimates that the PortfolioStat effort has the potential to save $2.5 billion from fiscal year 2013 through fiscal year 2015 by, for example, consolidating duplicative systems.

Given the importance of these initiatives, OMB has established reporting requirements to, among other things, track the status of agencies’ implementation of these efforts. In addition, Congress recently incorporated key aspects of a number of these reforms into law.

Our extensive experience at federal agencies and in particular, our recent reports on TechStat, data center consolidation, and PortfolioStat, have shown that these reforms and the required information agencies are to report on to OMB as part of these efforts offer important opportunities to improve the efficiency and effectiveness of federal agency programs and operations, including producing financial savings. The reports also included recommendations to OMB to improve agency reporting on key initiatives; the recommendations highlighted the importance of OMB’s reporting requirements and the need for federal agencies to provide current and accurate information about the status of these initiatives. They also highlighted that the requirements are a critical component to ensuring OMB’s effective management and oversight of the initiatives.

Together, the responsibilities discussed above require CIOs to be key leaders in managing IT in a coordinated fashion in order to improve the efficiency and effectiveness of programs and operations.

OMB Has Periodically Changed CIO Requirements to Address Changes in Federal IT Management

Over the last several years, OMB has made changes to CIO reporting requirements to address, among other things, changes in federal IT management. Such changes included modifying how requirements are reported, updating what information is requested as part of existing requirements, and establishing new requirements. For example, in March 2013, OMB issued a memorandum which, among other things, established the Integrated Data Collection, which was a new way for agencies to submit information relating to IT reform initiatives such as PortfolioStat and data center consolidation.

OMB, Fiscal Year 2015 Guidance on Exhibits 53 and 300 – Information Technology and E-Government (Washington, D.C.: July 1, 2013). agencies provide information on investments that are to be reduced or eliminated as the result of new investments.

Moreover, in May 2014, OMB issued updated instructions for the Integrated Data Collection, aimed at improving the quality of data, which changed the format and information reported for several requirements, as well as adding a new reporting requirement on progress in using standard customer value methodologies to evaluate agencies’ highest impact IT services.

Prior GAO Reports Have Recommended Improvements to IT Reform Initiatives and Associated Reporting

During the past several years, we have reported on a variety of issues related to CIOs’ roles and responsibilities and OMB’s management and reporting of information obtained through federal agency reporting requirements. For example, in September 2011, we reported on the roles and responsibilities of agency CIOs. Specifically, we found that although most CIOs are responsible for major areas of IT (e.g., capital planning, IT strategic planning, and e-government initiatives), they are less frequently responsible for other information management areas (e.g., records management and privacy) that, despite being required by law, are considered not critical to effective IT management. We recommended that OMB update its guidance to establish measures of accountability for ensuring that CIOs’ responsibilities are fully implemented and require agencies to establish internal processes for documenting lessons learned. OMB agreed with our recommendations and stated that it had taken actions that it believed addressed the recommendations; we are currently in the process of validating whether these actions fully address our recommendations.

In addition, in November 2013, we found that of the 26 major federal agencies that were required to participate in the PortfolioStat initiative— an annual agency-wide portfolio review—only 1agency addressed all of the key requirements. We also determined that OMB’s estimate of $2.5 billion in savings from PortfolioStat was understated because it, among other things, did not include estimates from the Departments of Defense and Justice. Our analysis, which included these estimates, showed that agencies were reporting at least $5.8 billion in potential savings. Further, not all agencies provided sufficient support for their estimated potential savings on consolidation initiatives. We recommended that OMB and the agencies improve their PortfolioStat implementation, and the parties agreed in large part with our recommendations; we are currently following up to assess their progress in doing so.

Further, in December 2013, we reported that although the accuracy of ratings on the Federal IT Dashboard had improved, they were In addition, we found inconsistent among the agencies we reviewed. that the public version of the Dashboard was not updated for 15 of the past 24 months, and so was not available as a tool for investment oversight and decision making. We recommended that OMB make Dashboard information available independent of the budget process and agencies appropriately categorize IT investments and address identified weaknesses. OMB neither agreed nor disagreed with our recommendations. Nonetheless, we have ongoing work to assess the extent to which OMB and agencies have implemented the recommendations.

GAO-14-713. savings on as many as 67 data centers because of difficulties, such as calculating baseline data center costs. Further, we found that OMB had developed metrics, but these metrics do not address server utilization. Consequently, we recommended that OMB, among other things, develop and implement a metric for server utilization and agencies address their challenges in reporting costs savings. OMB and the agencies agreed with our recommendations. We have initiated follow-up efforts to assess agency progress in implementing our recommendations.

Agency CIOs Are to Address 36 IT Management Reporting Requirements for OMB

OMB directs agency CIOs to respond to 36 IT management reporting requirements. These 36 requirements—which we organized by key IT management areas such as IT strategic planning, IT security, and related initiatives—are shown in table 1 along with a description of each requirement; how often (i.e., the frequency) required information is to be reported (e.g., monthly, quarterly, annually); and how agencies are to report required information (the reporting mechanism). OMB specifies for each requirement, the reporting mechanism to be used, which range from posting information on an agency’s website or OMB’s IT Dashboard to transmitting it to OMB via the MAX Portal or the Integrated Data Collection system.

Additional details about these requirements, including when each requirement and its associated OMB guidance was initiated, are provided in appendix II.

As shown in figure 1, of the 36 requirements, the largest number are in IT security (12), and the fewest are in IT strategic planning (2).

In terms of reporting frequency, agency CIOs are largely required by OMB to report on the 36 requirements on a quarterly or annual basis. Specifically, 29 of the 36 requirements are required to be reported quarterly and/or annually. Further, several requirements are required to be reported at multiple periods. For example, the cybersecurity plan of action is required to be reported both quarterly and annually. In addition, agencies are required to submit major IT investment documentation annually and as needed (e.g., when significant changes occur to an investment).

The other seven are to be reported as follows: one biennially, one semi-annually, one monthly, one one-time, one continuously, and two as-needed.

According to officials from OMB’s Office of E-Government and Information Technology, OMB established the particular reporting periods to ensure it gets the information that it needs at the proper time to achieve its mission to, among other things, develop the President’s budget, make informed policy decisions, provide oversight, and meet statutory requirements. For example, OMB requires agencies to report quarterly on cost savings and avoidances, which assists OMB in publishing its quarterly report to Congress on progress with IT oversight and reform. Further, OMB requests that agencies report annually on the implementation status of their e-government initiatives, which assists OMB in developing its annual report to Congress on federal e- government. OMB also requests certain requirements to be reported at multiple periods. For example, agencies are required to report multiple iterations of their exhibits 53s and 300s in accordance with a schedule developed by OMB, which assists OMB in developing the President’s budget.

OMB has the agencies report these requirements via four primary mechanisms—namely, the federal IT Dashboard, the Department of Homeland Security’s CyberScope portal, OMB’s Integrated Data Collection system, and agency websites. More specifically, of the 36 requirements, agencies use

CyberScope for 5 requirements, the Integrated Data Collection system for 7 requirements, and agency websites for 9 requirements. the federal IT dashboard for 6 requirements, Agencies also report eight requirements using other mechanisms, including one via Data Point, one via a Department of Homeland Security information security one via the Federal Data Center Consolidation Initiative program management office portal, two via meetings with OMB officials, three via MAX Portal, and two via e-mail.

In terms of OMB’s IT reform initiatives, there are a number of the 36 requirements related to managing and overseeing OMB’s efforts in these areas. Key examples include the requirements on:

Agency TechStat outcomes,

Data center closures/status update,

PortfolioStat progress report,

PortfolioStat progress review, and

Commodity IT baseline updates.

Although OMB Uses Required Information, CIOs Reported That the Majority of Reporting Requirements Are Not Useful for Managing IT and Identified Areas for Improvement

See, for example, GAO-13-524, GAO-14-713, and GAO-14-65. consolidation, and PortfolioStat—still have important value. Accordingly, it is concerning that CIOs do not always see value in these reporting requirements. Consequently, effectively addressing proposed changes and aligning CIOs’ priorities to OMB’s (i.e., establishing a common understanding of what the priorities are) is important to, among other things, the success of OMB’s reforms and its goal of improving federal IT. Until this is done, there is a risk that the IT reforms will not succeed.

OMB Utilizes Reported Information to Meet Its Responsibilities

According to officials from OMB’s Office of E-Government and Information Technology, OMB utilizes all the information reported by agency CIOs to carry out, among other things, its budget development, policy formulation, and oversight roles and responsibilities. Specifically, it uses the information to undertake the following activities:

Development and execution of the President’s budget. Each year, OMB and federal agencies work together to determine how much the government plans to spend on IT projects and how these funds are to be allocated. OMB coordinates with federal agencies to obtain agency budget requests, and other information through the exhibit 53, exhibit 300, and annual FISMA reporting. OMB uses this information to analyze the requests, and prepare budget materials for the President’s review. These budget materials also include an analytical assessment that, among other things, provides details on the federal IT budget and the administration’s key federal IT initiatives.

Formulation of policies and guidance for the management of federal agency IT. OMB issues policy guidance and memorandums related to various aspects of IT management in order to improve the management, oversight, and transparency of the federal government’s IT. As part of these activities, OMB uses the information reported by agencies to inform policy decisions. For instance, OMB uses the reported information from the Federal Risk and Authorization Management Program to evaluate agency progress in implementing cloud services, which is an OMB policy priority.

Oversight of federal agency IT. OMB’s Office of E-Government and Information Technology is also responsible for oversight of federal information technology spending, and more than $80 billion is annually invested in federal IT. OMB provides oversight through several mechanisms including the Federal IT Dashboard, PortfolioStat reviews, and TechStat sessions. For instance, OMB requires agencies to report information on their IT portfolio, including commodity IT baselines, and information related to the Federal Information Security Management Act, as well as develop an Information Resources Management Strategic Plan and Enterprise Roadmap. OMB’s goal is to use the information it collects from agencies to monitor federal IT spending and help ensure programs and operations are efficient and effective.

Meeting statutory requirements. Under federal law, OMB’s Office of E-Government and Information Technology is required to report to Congress on certain IT management areas. In particular, the office is required to submit a report on the implementation of the E- Government Act of 2002, which summarizes information reported by agencies as required under the act. In addition, during the period of our review, OMB was also required to submit a report on the implementation of the Federal Information Security Management Act of 2002 by federal agencies. In order to prepare these reports, OMB requires agencies to submit information on their implementation efforts as required under the acts, which OMB then summarizes for Congress.

CIOs Identified Reporting Requirements Most Useful for Managing IT and Those That Were Less Useful

The 24 agency CIOs we surveyed reported that addressing certain reporting requirements assisted their agency in managing IT, while addressing other reporting requirements were not as useful. Specifically, a majority of 24 CIOs surveyed that responded reported that addressing 4 reporting requirements helped their agency to manage IT to a very great extent or great extent and 8 helped the agency to manage their IT to a moderate extent. They also reported that addressing the 24 remaining reporting requirements helped agencies only to some to no extent in managing IT.

Table 2 lists the 36 reporting requirements by the extent of assistance in managing IT as reported by agency CIOs. Specifically, it shows how the majority of CIOs (including the number) rated each requirement against our categories of usefulness—either very great to great, moderate, or some to no extent.

In addition, of the 24 requirements that CIOs found only assisted their agency in managing IT to some to no extent, our analysis showed that a number of them were associated with OMB’s IT reform initiatives. In particular, they include the requirements on

Agency TechStat outcomes,

Data center closures/status update,

PortfolioStat progress report,

PortfolioStat progress review, and

Commodity IT baseline updates.

CIOs Reported That Addressing Requirements That Were Not Always Useful Took a Significant Level of Effort to Implement

With regard to effort required to meet the 36 reporting requirements, the majority of the 24 agency CIOs reported that

16 requirements required a very great or great effort to meet,

14 required a moderate effort, and

6 required some to no effort to meet.

In addition, in terms of resources, agency CIOs estimated that they spend in total, approximately $150 million to $308 million annually to address the 36 reporting requirements. For individual requirements, agencies’ estimates of the range spent were generally at least a total of $1 million annually and as high as $19 million for one requirement (exhibit 300).

The majority of the 24 agency CIOs that responded also reported that the level of effort and resources reported to meet the requirement was sometimes greater than the extent to which addressing the reporting requirement assisted the agency in managing their IT. Specifically, of the 24 reporting requirements that provided some to no assistance in managing IT, 8 of these required very great or great effort to meet and 10 required moderate effort to meet. In addition, 4 of the 8 reporting requirements that helped agencies to a moderate extent in managing IT required a very great to great effort to meet.

Further, 26 out of 36 reporting requirements that CIOs reported assisted agencies in managing IT to some or no extent were estimated to cost agencies approximately $76 million to $164 million each year to meet. In addition, the 8 requirements that assisted agencies to a moderate extent were estimated to cost approximately $50 million to $92 million a year. However, the 4 key reporting requirements that helped agencies manage their IT to a very great or great extent were estimated to cost $24 million to $52 million a year.

Table 3 lists the 36 reporting requirements by the CIO reported usefulness of assistance in managing IT, the level of effort required, and the estimated annual cost to meet the requirement. More specifically, it shows how the majority of CIOs that responded rated each requirement against our categories of levels of effort—either very great to great, moderate, or some to no effort.

In addition to the fact that a majority of the 24 agency CIOs that responded reported that complying with many of the reporting requirements was not always commensurate with their usefulness in managing IT, they also generally indicated they would only collect at least some, but not all of the information, if addressing the requirements was optional. Specifically, for the 24 requirements that helped some to no extent, the majority of the CIOs reported that they would collect at least some but not all of the information if not required to do so.

According to comments from a number of CIOs, they did not always find that these requirements were useful because addressing them did not always clearly support departmental priorities. For example, with regard to reporting investment information using the exhibit 300, three CIOs said it had little value beyond reporting information that OMB needed to make decisions because their departments had their own processes for investment decisions. For IT security quarterly reporting and monthly IT security data feeds, three CIOs said these requirements were not commensurate with their usefulness because they were burdensome due in part, to the reporting format, frequency, and duplicative nature of certain elements. Other examples cited by CIOs include the following:

For government-wide tracking of cybersecurity resources, one agency CIO commented that it was helpful to determine how much funding was spent on cybersecurity, but providing the supporting detailed accounting of the resources called for in the requirement was difficult. Another CIO commented that tracking resources helped in understanding the investments made and historical data provided insight into whether prior allocated resources were impactful; however, the reporting requirement needed to be consolidated with annual Federal Information Security Management Act reporting.

Concerning commodity IT baseline updates, one agency CIO commented that it had helped with understanding the types of commodity spending that made up the agency’s portfolio and identified opportunities for optimization but reporting needed to be combined with other annual budget reporting. Another CIO commented that while reporting this information helped OMB provide oversight, the agency would prefer if OMB used the information to help agencies develop better strategies and operations plans that would result in cost reductions.

Regarding program management cost savings and avoidances, an agency CIO commented that while tracking cost savings provides a more robust understanding of its IT portfolio, the reporting is too frequent for capturing the cost savings. Another CIO noted that the reporting helps identify initiatives that are successful in driving down costs and those that are falling short of projected savings, but providing updates on changes to the cost savings and avoidance figures to OMB is burdensome.

CIOs Proposed Changes to Improve Reporting Requirements and OMB Feedback

Although agency CIOs surveyed generally found some of OMB’s initiatives valuable for managing IT resources, at least 8 or more proposed changes to improve (1) 13 reporting requirements and (2) OMB’s feedback to agencies on the reporting requirements generally.

With regard to the 13 reporting requirements, at least nine agency CIOs proposed changing what information should be reported under 3 reporting requirements, stating that data elements that do not add value in terms of what OMB needs or uses to make decisions or are no longer relevant should be removed. Agency CIOs also proposed changing the frequency of reporting for 5 requirements, moving from reporting on a quarterly basis to either a semi-annual or annual basis. In addition, at least eight of the CIOs proposed that 4 reporting requirements should be eliminated because they were generally either not useful to the agencies in managing their IT, information was duplicative with other reporting requirements, or OMB had not requested the information in recent years. Table 4 lists the reporting requirements, the agency CIOs’ proposed changes, and the number of CIOs that proposed them.

Officials from OMB’s Office of E-Government and Information Technology stated that they had received similar feedback on proposed changes to the reporting requirements in the past. They said that in some cases there was confusion among agency officials regarding the reporting requirements. Officials noted that the feedback on the requirements was useful information but provided no specific plan or date for addressing these suggestions. Effectively addressing such proposed changes is important to improving the efficiency and effectiveness of reporting requirements and could better position OMB to achieve its goal of improving management, oversight, and transparency of federal IT. Until this is done, OMB risks requiring agencies to implement and report on requirements that are duplicative, wasteful, or inefficient.

With regard to improving feedback on reported information, agency CIOs suggested that OMB’s feedback process could be improved. In particular, while agency CIOs reported that OMB provided feedback to them on the majority of the 36 reporting requirements, the majority of CIOs reported that the feedback was moderately effective to not effective for most reporting requirements. Six agency CIOs also reported that they were specifically interested in receiving better feedback on two reporting requirements—namely, the major IT investment documentation and the Open Government directive. The Office of E-Government and Information Technology officials stated that the information on the feedback, particularly those requirements agency CIOs were interested in receiving feedback on, was useful; nevertheless, the officials acknowledged that they do consistently not provide this level of feedback to the CIOs because in part, they did not know until now that the CIOs wanted feedback to this extent. Having a process that consistently provides effective feedback is key to helping agency CIOs better manage their IT resources and improve reporting; it is also consistent with OMB’s goals to improve federal IT management, oversight, and transparency. Until an effective feedback process is in place, there is a risk that agencies are managing their IT in a suboptimal manner.

As mentioned previously, our extensive experience at federal agencies and recent reports have shown that a critical component to ensuring OMB’s effective management and oversight of key IT reform initiatives— specifically, TechStat, data center consolidation, and PortfolioStat— is agency reporting of current and accurate information about the status of these initiatives, including the extent of any financial savings. However, agency CIOs surveyed reported that requirements related to these IT reform efforts, including agency TechStat outcomes, data center closures/status updates, commodity IT baseline updates, and PortfolioStat reviews and progress reports, helped their agency to only some or to no extent in managing IT. It is concerning that CIOs do not always see the value in reporting information that is essential to reform initiatives aimed at improving IT management effectiveness, saving money, and avoiding unnecessary costs, especially since key aspects of a number of the reforms have also been recently incorporated into law.

Consequently, establishing a common understanding between OMB and the CIOs on the priority of the reporting and related initiatives is key to the success of OMB reforms. As part of this understanding, it is also important to address underlying reasons cited by CIOs regarding the usefulness of requirements, including when department priorities are reportedly different than OMB’s and the burdensome and duplicative nature of requirements. Until such an understanding is established, there is a risk these important IT reforms, which are key to improving the efficiency and effectiveness of federal agency programs and operations, will not fully succeed.

OMB Has Initiated Efforts to Streamline Reporting, but They Do Not Address Challenges Reported by CIOs

OMB has taken steps to streamline CIO reporting requirements. Specifically, OMB has initiated efforts to identify opportunities to change the format for reporting (e.g., from narrative-intensive descriptions to specific performance data) with the goal of reducing CIOs’ reporting burden. Although these OMB efforts aim to streamline reporting, agency CIOs identified additional challenges with tracking what reporting requirements are currently in place, using multiple online tools to report required information, and using capital planning and investment reporting requirement information to make effective investment decisions, which are not addressed by OMB’s efforts. This is in part to the fact that OMB has not solicited feedback in these areas because its priority has been on streamlining reporting in other areas (e.g., changing report formatting and others discussed below). By not addressing these CIO-identified challenges, OMB is missing opportunities to help CIOs improve the requirements reporting process and to improve its use of information collected as part of this process to effectively manage IT.

OMB Is Working to Streamline Reporting

OMB has initiated several efforts to streamline reporting:

Changing the format and mechanism of information that is submitted. Officials from OMB’s Office of E-Government and Information Technology reported that they are examining transitioning the reporting of certain requirements from previous narrative-intensive plans or other documentation into structured forms (such as the Integrated Data Collection) that call for specific data associated with OMB-established performance metrics. These officials stated that they are continually re-evaluating the format in which agencies should provide information in order to improve the efficiency of OMB’s review process and the reliability of pertinent federal IT management data. For example, OMB expanded and refined its collection of key performance indicators for investment portfolio management with the Integrated Data Collection between 2013 and 2014.

Revising what information is currently required to be submitted as part of existing requirements. The Office of E-Government and Information Technology officials stated that they recently began reviewing existing reporting requirements in the Integrated Data Collection prior to issuing quarterly reporting instructions to identify whether any elements should be changed. For example, in May 2014, as the result of one of these reviews, OMB removed the commodity IT baseline portion of the Integrated Data Collection for the submission due in November 2014. In addition, OMB reported it is working with the CIOs to improve the value of agency reporting on the federal center consolidation initiative. Specifically, OMB reported that working via a task force that is part of the OMB-led CIO Council, OMB has helped to develop metrics and data collection requirements that best support administration goals associated with consolidating and optimizing data centers. OMB further reported that as a result of these efforts, it has made significant changes to both its reporting requirements and strategic approach associated with this initiative.

Incorporating lessons learned into the annual revision of PortfolioStat guidance. The officials from the Office of E- Government and Information Technology also stated that in 2013 they initiated an annual review of lessons learned from agency PortfolioStat sessions to identify whether additional information is needed to improve the office’s oversight of federal IT portfolio management. These officials added that they incorporated the results of these reviews into the 2014 PortfolioStat guidance, which may have included adding requirements or streamlining others. For example, OMB required agencies to identify IT investments that merit additional oversight and support for review and discussion during the 2014 PortfolioStat sessions. Selected agencies were required to develop an action plan with specific goals and targets for these high-impact investments.

Integrating requirements with other IT government-wide efforts. The Office of E-Government and Information Technology officials added that they review existing requirements to identify potential changes, including opportunities to streamline, when new government-wide IT initiatives are introduced. For example, they reported that their office is currently in the process of evaluating CIO and other reporting requirements to determine how they align to OMB’s strategic cross-agency priority goals associated with smarter IT delivery, cybersecurity, and Open Databe opportunities to further streamline existing reporting requirements.

In October 2013, OMB and the Federal CIO Council established a working group to streamline reporting requirements related to capital planning and investment control. The working group studied, among other things, whether IT investment performance information and related reporting to OMB could be streamlined and better aligned with what information was needed by CIOs to make informed investment decisions. As a result of the study, the working group made a number of short-term and long-term recommendations to OMB in April 2014 to improve its fiscal year 2016 IT budget capital planning guidance. According to OMB officials, they addressed certain short-term recommendations related to improving standard definitions and purchase provisions, and are working to address the long-term recommendations in future guidance. They also said they expect the working group to continue studying the issues associated with this area and to annually provide recommendations on improving and streamlining capital planning guidance.

OMB Efforts Do Not Address CIO-Identified Challenges in Meeting Reporting Requirements

Although OMB has initiated several efforts to further streamline CIO reporting requirements, its efforts do not address the following challenges agency CIOs identified in our survey in meeting reporting requirements: Tracking what reporting requirements are currently in place can be confusing. Agency CIOs expressed confusion in our survey on whether certain reporting requirements (e.g., the IT Capital Plan and compliance failures) were still in effect. This confusion was due in part to neither OMB nor the majority of federal agencies we surveyed having a comprehensive list of current CIO and related reporting requirements. In particular, officials from the Office of E-Government and Information Technology office noted that while they did not maintain a comprehensive list, they did have a spreadsheet they used for selected review processes (e.g., PortfolioStat) but that this list was not available to the federal agencies because it was an internal OMB working document. Having a comprehensive list of current CIO reporting requirements is important for agencies to effectively manage their resources and respond to these requirements and for OMB to effectively manage its streamlining efforts consistent with its goal of reducing CIO reporting burden. Without such a list, OMB may lack sufficient information to make informed decisions about streamlining efforts and agencies risk wasting resources responding to reporting requirements that have changed or are no longer in effect.

Having multiple online reporting tools takes additional time and resources to enter required information and can be duplicative. Agency CIOs reported in our survey that having to enter data into multiple online tools for reporting required information as part of capital planning, system integration, and IT security takes additional time and resources and can be duplicative. OMB requires agencies to use three online tools (Data Point, IT Dashboard, and the Integrated Data Collection) as well as e-mail for capital planning and system integration reporting requirements, and four online tools (Cyberscope, MAX Portal, Integrated Data Collection, and the Information Security Continuous Monitoring Dashboard) for IT security. In particular, agency officials noted that certain information for the capital planning reporting requirements (exhibit 53, exhibit 300) and cybersecurity reporting requirements (IT security key metrics, monthly IT security data feeds, IT security quarterly reporting) must be entered, in accordance with OMB’s direction, in each tool; some information must be reported by bureau or office; and there are limitations in uploading data using a spreadsheet or multiple attachments into the tools, all of which takes additional time and resources.

OMB officials stated that they were looking at what reporting requirements can be reported on through the tools, particularly the Integrated Data Collection, and what steps can be taken to simplify the tools. They also noted that there were challenges when determining whether to reuse an existing tool or create a new one for reporting requirements, particularly when they needed agencies to report information within a short time frame, but provided no specific date or plan for the completing the above described efforts, including addressing the related challenges. Having centralized tools for reporting required data is critical to reducing the reporting of duplicative data, as well as reducing the time and resources required to enter the data, and would be consistent with OMB’s goal of reducing CIO reporting burden. By not including efforts to streamline reporting tools as part of OMB’s overall requirements streamlining effort, there is a risk that agencies will continue to expend valuable time and resources entering potentially duplicative data into multiple reporting tools.

Using capital planning and investment reporting requirement information does not provide information to make effective investment decisions. As mentioned previously, agency CIOs reported in our survey that six of the seven capital planning and investment reporting requirements—namely the exhibit 53, exhibit 300, major IT investment documentation, IT Capital Plan, PortfolioStat progress report, and compliance failures—needed improvement. For example, agency CIOs reported that the two key capital planning reporting requirements— the exhibit 53 and exhibit 300—needed to be improved to remove data elements that did not provide value in making decisions. For the exhibit 53, this included eliminating non-financial data elements, such as those on cloud computing, and for the exhibit 300, eliminating the contracts and acquisition information since it was reported elsewhere. In addition, agency CIOs proposed eliminating three of the capital planning and investment reporting requirements (e.g., major IT investment documentation, IT Capital Plan, and compliance failures) because agencies had either not provided this information in recent years or information was reported elsewhere in accordance with other reporting requirements. By not focusing on efforts to streamline and improve capital planning and investment reporting requirements, there is a risk that OMB will not be able to adequately respond to the evolving nature of IT and ensure these key reporting requirements include data elements that provide the necessary value in making critical investment decisions.

With regard to the above challenges, the OMB officials told us that they were not currently addressing them because their top concern was the other requirement streamlining areas (i.e., changing the format and incorporating lessons learned) they were focusing on. They added that they had not solicited feedback from the agencies in the areas that were the focus of our survey and thus were not aware of the challenges. The officials also noted that the feedback we provided from the CIO surveys, including the challenges currently faced by CIOs, was useful information to consider as part of their further streamlining efforts. Nonetheless, until these challenges are addressed, OMB risks not achieving its goal of reducing CIO reporting burden and is missing opportunities to help CIOs improve the reporting process, and their use of information collected as part of this process, to effectively manage IT.

Conclusions

OMB has established an extensive framework of IT management reporting requirements to aid in carrying out its mission and to help federal agency CIOs manage their IT resources. Although OMB uses reported information to meet its responsibilities, the majority of the 24 CIOs for the most part, do not, citing that (1) two thirds of the reporting requirements are not very useful to them in managing IT and (2) addressing reporting requirements entails a significant effort on the part of the CIOs, including an annual multi-hundred-million-dollar financial commitment. Nonetheless, our extensive experience at federal agencies and recent work have shown that reporting requirements—in particular, those relating to TechStat, data center consolidation, and PortfolioStat— are key to carrying out OMB’s IT reforms and their goal of improving federal agency programs and operations, including producing financial savings. Consequently, it is of particular concern that OMB and the CIOs are not fully aligned on the utility of reporting requirements integral to the success of OMB’s reforms. This misalignment is due in part to a lack of a common understanding between OMB and the CIOs on the priority nature of these reporting requirements and their associated reforms. Without such a common understanding, OMB and the CIOs risk missing key opportunities to improve federal programs and operations and produce savings. The CIOs did propose a number of changes aimed at increasing the usefulness of reporting requirements and providing for effective feedback, but OMB had not yet established an effective approach to address them. Until it does, OMB risks requiring agencies to report on and manage IT in a suboptimal manner, which is inconsistent with its goal of improving federal IT management.

While OMB’s efforts to streamline CIO reporting are steps in the right direction, they largely do not address the CIO-identified challenges discussed in this report. These challenges are not being addressed by OMB in part because it is focusing on other aspects of requirement streamlining as its top priority and also because of a lack of awareness of these specific CIO challenges. Until these challenges are addressed, OMB is missing opportunities to help CIOs improve the requirements reporting process and its use of information collected as part of this process to effectively manage IT.

Recommendations for Executive Action

To improve the effectiveness of OMB streamlining efforts and ensure agency CIOs are better able to carry out their responsibilities in managing IT, including implementing OMB’s IT reform initiatives, we recommend the Director of OMB direct the Federal CIO, in collaboration with agency CIOs, to take the following four actions:

Ensure there is a common understanding with agency CIOs on the priority of the current reporting requirements and related IT reform initiatives. This should include addressing underlying reasons cited by CIOs regarding the usefulness of requirements, including when department priorities are reportedly different than OMB’s and the burdensome and duplicative nature of requirements.

Address and incorporate, as appropriate, the 13 proposed improvements to reporting requirements made by agency CIOs in our survey. This should include developing milestones and associated plans for completing this effort.

Ensure the feedback process on information reported by CIOs consistently provides effective and constructive responses to these officials on their reported information.

Assess, as part of ongoing streamlining efforts, the reporting challenges identified in our report. This should include determining whether to (1) have a comprehensive list of current IT reporting requirements that is publically available to agency CIOs; (2) reduce the number of reporting tools; and (3) improve the utility of capital planning and investment reporting requirements, and taking steps to implement where appropriate.

Agency Comments and Our Evaluation

In written comments provided by the Federal CIO and reprinted in appendix IV, OMB stated it respectfully neither agreed nor disagreed with our recommendations. According to OMB, it chose to take no position on the recommendations due to its concerns that: (1) our draft report’s count of reporting requirements was not currently accurate, (2) our survey approach did not fully support the report’s findings and recommendations, (3) the second objective to solicit CIO views did not allow for sufficient context setting, and (4) OMB has taken steps to solicit feedback and streamline requirements that are not reflected in the draft report. OMB added that it agreed with the principles contained in the recommendations that stress the importance of a common understanding with CIOs and other key stakeholders on the importance and priority of reporting requirements, ensuring effective and constructive feedback to CIOs on the reported information, and streamlining reporting requirements. OMB further noted that it has actively incorporated these principles in its work and continuously works to enhance its efforts in these areas.

We evaluated OMB’s concerns and found them, as discussed below, in large part either unfounded or as having no material impact on our results and recommendations. Consequently, we stand by our findings on OMB’s key IT reform initiatives—and their associated reporting requirements— which are critical to improving federal IT management effectiveness. Further, ensuring the success of these reforms is a key reason why Congress, in December 2014, incorporated key aspects of a number of these reforms into law. Accordingly, we believe our recommendations to OMB are still valid and essential.

With regard to OMB’s first concern about the accuracy of the number of requirements identified in our report, OMB took issue with our inclusion of seven requirements, although it provided no documentation to support its comments; specifically, OMB said (1) two requirements did not exist at the time of GAO’s audit (specifically, Information Security Continuous Monitoring Dashboard and Cybersecurity Plans of Action), (2) two had been eliminated by OMB during or shortly after the audit time frame—namely, the PortfolioStat Review and the Commodity IT Baseline, and (3) three appeared to have been had been double-counted (specifically, the IT Capital Plan and Cloud First are components of the Exhibit 53 requirement, and Open Data Policy Public Data Listing is a subset of Open Data Enterprise Inventory).

Regarding the first two requirements (Information Security Continuous Monitoring Dashboard and Cybersecurity Plan of Action), our analysis shows these requirements were in effect at the time of our audit, and this is why we therefore included them in our list. Specifically,

OMB memorandum M-14-03 (dated November 2013)agencies, in addition to submitting data to this dashboard once it is deployed, to report on the status of their preparation for meeting this requirement starting in 2014, which was in the time frame specified in our scope and methodology.

OMB memorandum M-12-20 (dated September 2012) stated that The Department of Homeland Security will “ask agencies to complete a Plan of Action for improving specific cybersecurity responsibilities. Agencies will provide quarterly and fiscal year targets and demonstrate progress toward these targets as they mature their programs.” Although OMB assigned responsibility to the Department of Homeland Security for these plans and subsequent quarterly updates, we considered it to be an OMB reporting requirement because the reporting requirement was included in an OMB memorandum.

In addition, for the two requirements that OMB said were eliminated (i.e., PortfolioStat review and Commodity IT Baseline), our evaluation shows that both had not been eliminated as of March 2014, which is the “as-of” date specified in our scope and methodology. Specifically,

OMB memorandum M-13-09 (dated March 2013) required agencies to submit a consolidated document of successes, challenges, and lessons learned to OMB no later than 2 weeks after the transmittal of the President’s Budget for fiscal year 2015 to Congress (which was sent on March 4, 2014). Since the requirement was in effect as of March 2014, we included it in our list.

OMB memorandum M-13-09 also required agencies to provide a quarterly update of their commodity IT baseline to OMB. Because this requirement was not eliminated until May 2014, as OMB’s comments and our report indicated, we therefore included it in our list since the requirement was in effect as of March 2014.

Thirdly, although OMB stated that we double counted three requirements (i.e., IT capital plan, Cloud First, and the Open Data Policy Public Data Listing), the evidence shows otherwise. Specifically, these requirements are defined in OMB circulars or memoranda (as noted in appendix II) as separate reporting initiatives, and we therefore treated them as such. For example, the requirement for an IT capital plan is outlined in OMB Circular A-130 (dated November 2000) and details several activities that the agency must provide annually to OMB as part of the budget submission.

In addition to the above, it is important to note that (as described in our report’s scope and methodology) after we developed our list of requirements, we asked all 24 CFO agencies as well as OMB to review the list and provide feedback as a means to validate that our inventory was accurate. Specifically, we had officials from the 24 agencies review our list to ensure it was complete and accurate, adjusted our list as appropriate based on their feedback, and reached consensus on the number of requirements in the inventory. In the case of OMB, we had officials from the Office of E-Government and Information Technology review our list multiple times and adjusted our list as appropriate based on their feedback. During these OMB meetings, the officials never raised the specific issues OMB noted in its written comments.

It is also important to note that we took these validation steps because, as we noted in our report, OMB does not maintain a comprehensive list of current CIO reporting requirements. This has left agencies confused regarding what requirements were currently in place, and is therefore why we recommended that OMB develop such a list. This inconsistency among OMB’s written response, what the agencies told us, and what OMB representatives previously told us confirms the need for GAO’s recommendation.

OMB’s second concern was that our survey approach did not fully support the report’s findings and conclusions. Specifically, OMB stated that many agency CIOs told OMB through the CIO Council that they delegated full responsibility for completing the survey to lower-level staff and because of this, OMB did not have full confidence in some of our findings and felt our attributions was not appropriate. In addition to this concern, OMB said that it believed GAO gave disproportionate weight to the estimated costs reported, which appeared, in its opinion, to be unsubstantiated because they were based on a survey question that did not require the basic characteristics found in GAO’s cost estimating and assessment guide.OMB further questioned the reliability of these estimates because they believed staff other than the CIO had completed the survey and were therefore not experienced enough to provide knowledgeable estimates.

We disagree with OMB’s opinion regarding the validity of our survey results for the following reasons. First, in meetings with CIOs and selected senior officials from agencies in our review to discuss our plans to survey the CIOs, we consistently expressed our expectation that the CIOs were to complete the survey. If this was not feasible, CIOs could delegate it to other staff, especially when agencies wanted staff that worked directly on addressing the reporting requirements to complete the survey because they were best qualified to respond to our questions. In these cases, we nevertheless told the agencies the CIOs were ultimately responsible for reviewing the accuracy of the survey results.

Second, we pretested the survey at seven agencies, again reinforcing our expectation that CIOs were to complete the survey. Specifically, to minimize errors that might occur from respondents interpreting our questions differently from our intended purpose, we pretested the questionnaire in person and by phone in three rounds of testing with officials from the office of the CIO at seven agencies. The selection of agencies for pretesting was based on agency availability to assist us with pretesting, variation in size of agency, and variation in agency CIO models (i.e., centralized or decentralized). During these pretests, we asked agency officials to complete the questionnaire for one reporting requirement as we listened to the process. We then interviewed the respondents to check whether the questions were applicable, clear, unambiguous, and easy to understand.

Third, in transmitting the surveys to the CIOs, we sent them directly to each of the 24 agency CIOs with instructions indicating that they were to complete it, with assistance from other officials as appropriate, and that the results would be summarized in a report to our congressional requesters. Specifically, in coordination with GAO survey methodology experts, we developed and administered a web-based survey that we sent to the CIOs of the 24 CFO Act agencies. All 24 CFO Act agencies completed the final survey. We then reviewed all responses, and followed up by phone and e-mail to clarify the responses as appropriate.

The above three points notwithstanding, we have revised the report to note that CIO responses may have in some cases, come from agency officials completing the surveys on behalf of the CIOs.

In terms of the estimated costs of meeting reporting requirements, the application of GAO’s cost guide in this context would be inappropriate, according to an internal GAO subject matter expert who co-authored the guide. This expert said the guide is reserved for the evaluation of government acquisition program cost estimates. The expert added that given that GAO was not reviewing such programs in this case, OMB’s assertion is without merit. In addition, we initially asked OMB whether it had cost information on the reporting requirements but was told it did not and were directed to the 24 agencies as the best source. Further, we believe that we have appropriately qualified the range of costs reported by the agencies and note that we made no recommendations on the cost of such reporting.

Further, regarding OMB’s comment about the experience of the staff providing estimates, we expressed our expectation that the CIOs complete this and the other parts of our survey. However, as previously discussed, in some cases CIOs and selected staff from the 24 agencies delegated this responsibility to agency subject matter experts who they thought were more knowledgeable in this area. In these cases, we again expressed our expectation that the CIOs at a minimum review the accuracy of such estimates.

With regard to the OMB concern about our report’s contextual framing of certain information related to its efforts, the agency stated that it appreciated our draft report acknowledging that agency reporting is a critical component of ensuring effective management and oversight of key IT reform initiatives and that such information is essential to the success of IT reform efforts; however, it said our draft lacked contextual framing because it did not the fully address the value of the reporting requirements beyond agency CIOs’ use of the data. In particular, OMB stated that these requirements helped ensure the successful implementation of federal law and/or supported administration priorities, including IT-related cross-agency priority goals (which OMB commonly refers to as CAP goalscover areas—specifically, cybersecurity, Open Data, and smarter IT delivery—in which increased cross-agency collaboration is needed to improve progress towards shared, complex priorities reaching far beyond the scope of the priorities of individual federal agencies. ). According to OMB’s comments, CAP goals We included a description in our report of the value of the information to OMB in carrying out federal laws and its statutory roles and responsibilities. Specifically, consistent with OMB’s comments, we point out that OMB utilizes reported information to meet statutory responsibilities. For example, our draft states that under federal law, OMB’s Office of E-Government and Information Technology is required to submit a report on the implementation of the E-Government Act of 2002, which summarizes information reported by agencies as required under the act. The draft also says that during the period of our review, OMB was required to submit a report on federal agencies’ implementation of the Federal Information Security Management Act of 2002. Further, the draft notes that in order to prepare such reports, OMB requires agencies to submit information on their implementation efforts as required under the acts, which OMB then summarizes for Congress.

Our report also discusses the administration’s priorities such as the IT- related cross-agency priority goals. OMB’s written comments acknowledge this point by stating that our draft report mentions the CAP goals. However, OMB notes that our draft does not provide essential context as to how the goals relate to reporting requirements. We disagree. Our draft stated that the CIO reporting requirements help the Federal CIO in its efforts to meet the CAP goals. Specifically, the draft noted that Office of E-Government and Information Technology officials review existing requirements to identify potential changes, including opportunities to streamline, when new government-wide IT initiatives are introduced. For example, these officials reported that their office is currently in the process of evaluating CIO and other reporting requirements to determine how they align to OMB’s strategic cross- agency priority goals associated with smarter IT delivery, cybersecurity, and Open Data and whether there might be opportunities to further streamline existing reporting requirements Regarding OMB’s fourth concern that our report does not reflect steps it has taken to solicit feedback and streamline requirements, the agency commented that it holds quarterly review (feedback) sessions with agency officials about IDC requirements. It added that these sessions have resulted in streamlining OMB collection requirements. OMB also noted that it meets with CIO council members (via a Federal Data Center Consolidation Initiative task force) to, among other things, help develop metric and data collection requirements that best support administration goals in this area. According to OMB, it has changed reporting requirements and its strategic approach with regard to the data center initiative as a result of feedback from the task force. In addition, OMB cited the annual capital planning and investment control guidance it issues, which includes Exhibits 53 and 300 reporting requirements, as another example of the agency soliciting feedback.

The streamlining efforts described in OMB’s comments are discussed in our report, including the role OMB played in soliciting of feedback. For this reason, we agree with and believe our draft is consistent with OMB’s comments as it relates to IDC requirements and the annual capital and control guidance it issues. Regarding its data center consolidation efforts, we are aware of this effort and have updated our report to show it as another example of OMB’s efforts to streamline and solicit feedback. Nonetheless, OMB’s comments provided no explanation or documentation to show it was addressing the CIO-identified challenges in our draft that are not currently being addressed by its streamlining efforts.

The above efforts aside, agency CIOs suggested, as noted in our report, that OMB’s feedback process could nonetheless be improved. In particular, while agency CIOs reported that OMB provided feedback to them on the majority of the 36 reporting requirements, the majority of CIOs reported that the feedback was moderately effective to not effective for most reporting requirements. Six agency CIOs also reported that they were specifically interested in receiving better feedback on two reporting requirements—namely, the major IT investment documentation and the Open Government directive. Office of E-Government and Information Technology officials stated that the information on the feedback, particularly those requirements agency CIOs were interested in receiving feedback on, was useful.

Nevertheless, the officials acknowledged that they do not consistently provide this level of feedback to the CIOs because, in part, they did not know until now that the CIOs wanted feedback to this extent. Having a process that consistently provides effective feedback is key to helping agency CIOs better manage their IT resources and improve reporting; it is also consistent with OMB’s goals to improve federal IT management, oversight, and transparency. Until an effective feedback process is in place, there is a risk that agencies are managing their IT in a suboptimal manner.

We are sending copies of this report to interested congressional committees, the Director of the Office of Management and Budget, the secretaries and agency heads of the departments and agencies addressed in this report, and other interested parties. In addition, the report is available at no charge on GAO’s website at http://www.gao.gov.

If you or your staffs have any questions on the matters discussed in this report, please contact me at (202) 512-9286 or pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) identify the current information technology (IT) reporting requirements that agency chief information officers (CIO) are to address for the Office of Management and Budget (OMB); (2) evaluate the extent to which OMB and agency CIOs use the required information to manage IT, including CIOs’ views on the utility of the requirements; and (3) assess any OMB efforts to streamline this reporting.

To identify the current CIO IT management reporting requirements, we obtained and analyzed OMB circulars, memorandums, and other issued guidance to develop a current list of requirements to report information to OMB; the resulting list included regular, recurring, or one-time requirements that were in effect as of March 2014. We focused on the requirements that were in effect by the end of the second quarter in fiscal year 2014 (i.e., March 2014) so that we could use these in our survey of In addition, since federal agency CIOs, which was issued in May 2014.there could be several requirements for information in multiple OMB memorandums for one initiative, we grouped the requirements to report information together by initiative and the frequency of reporting rather than list each as its own separate requirement. Requirements related to activities such as information collection and control of paperwork; records management; privacy and compliance with the Privacy Act; and information disclosure and compliance with the Freedom of Information Act were not included because these activities are not directly related to IT management responsibilities. Further, requirements directed to other agency officials but to which the agency CIO contributes were not included. Further, although OMB occasionally makes ad hoc requests to agency CIOs to provide information, we did not include those items in our list because they did not originate from issued guidance.

We categorized the requirements based on their best fit in the following areas that are typically identified as key CIO IT management responsibilities: IT strategic planning; capital planning and investment management; IT security; and system acquisition, development, and integration. We also had the agencies in our review (the 24 Chief Financial Officer (CFO) Act agenciesprovide feedback to help ensure it was complete and accurate. Based on their feedback, we made the necessary changes to our list, including adding several requirements and removing several requirements that were no longer in effect. ) and OMB review our list and To evaluate the extent to which agency CIOs and OMB use the required information to manage IT and to assess CIOs’ views on the utility of the requirements, we first obtained and analyzed OMB documentation and interviewed OMB officials; we then compared it to OMB’s goal of using CIO-reported information to improve the management, oversight, and transparency of federal IT. We also, in coordination with our survey methodology expert, developed and administered a web-based survey that we sent to the CIOs of the 24 CFO Act agencies. In meetings with CIOs and selected senior officials from agencies in our review to discuss our plans to survey the CIOs, we told them we expected the CIOs to complete the survey. If this was not feasible, CIOs could delegate the task to other staff, especially when agencies wanted staff that worked directly on addressing the reporting requirements to complete the survey because they were best qualified to respond to our questions. In these cases, we nevertheless told the agencies the CIOs were ultimately responsible for reviewing the accuracy of the survey results.

Using the survey, we requested information on, among other things, the usefulness of the reporting requirements in managing IT, the level of effort required to meet them, and whether CIOs thought the requirements should be changed. For our analysis and reporting of the questions on usefulness and level of effort, we grouped the reported responses into three categories (i.e., very great to great, moderate, and some to no extent) and reported the category that had a majority response, or the largest number of responses in cases where the total number of respondents for a particular requirement was less than 24. In instances where the number of responses was evenly, or close to evenly, divided between the “very great to great” and “some to no extent” categories, we chose the moderate category, as it provided the best overall representation of the views of the agency CIOs for that requirement. For the question on proposed changes to the reporting requirements, we provided six options of possible change for agency CIOs to choose from for each reporting requirement (i.e., frequency, mechanism, information provided, consolidation, elimination, or other). We determined that at least eight responses or one third of respondents suggesting a particular change represented a sufficient consensus of agency CIO views for reporting purposes.

In addition, for each requirement, we asked agency CIOs to estimate the annual amount (in government and/or contractor dollars) that agencies spent on meeting each reporting requirement. For each reporting requirement, we provided four choices in the survey to choose from ($1- $99,000; $100,000-$299,000; $300,000-$1,000,000; and more than $1,000,000). We decided not to offer more precise dollar amount options to choose from because, among other things, agencies do not typically keep track of funds spent on CIO reporting requirements because this is not required by OMB and therefore the benefits of having more precise amounts were outweighed by the time and effort agencies would have had to expend to develop such precise estimates. For our analysis of how much agencies spend annually on reporting requirements, since the option “more than $1,000,000” did not include an upper range, we worked with a survey methodology expert to define the lower and upper range of this choice to be the same, which is $1 million plus one dollar or $1,000,001. See appendix III for the list of survey questions.

To minimize errors that might occur from respondents interpreting our questions differently from our intended purpose, we pretested the questionnaire in person and by phone in three rounds of testing with officials from the office of the CIO at seven agencies. The selection of agencies for pretesting was based on agency availability to assist us with pretesting, variation in size of agency, and variation in agency CIO models (i.e., centralized or decentralized). During these pretests, we asked agency officials to complete the questionnaire for one reporting requirement as we listened to the process. We then interviewed the respondents to check whether the questions were applicable, clear, unambiguous, and easy to understand. All 24 CFO Act agencies completed the final survey, although not all survey respondents answered every question. We then reviewed all responses, and followed up by phone and e-mail to clarify the responses as appropriate.

The practical difficulties of conducting any survey may introduce non- sampling errors. For example, differences in how a particular question is interpreted, the sources of information available to respondents, or the types of respondents who do not respond to a question can introduce errors into the survey results. We included steps in both the data collection and data analysis stages to minimize such non-sampling errors. We examined the survey results and performed computer analyses to identify inconsistencies and other indications of error, and addressed such issues as necessary. We analyzed responses to closed-ended questions by counting the response for all agencies. For questions that asked respondents to provide a narrative answer, we compiled the answers in one document that was analyzed and used as examples in the report.

To assess any OMB efforts to streamline this reporting, we obtained and analyzed OMB and Federal CIO Council documentation and interview information to summarize OMB’s current and future plans for streamlining CIO reporting and compared these efforts with OMB’s goal to reduce CIO reporting burden. As part of this, we interviewed officials from OMB and the Federal CIO’s Capital Planning and Investment Control Community of Practice key working group to identify current and future actions taken to streamline CIO reporting requirements.

We conducted this performance audit from December 2013 to April 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions base on our audit objectives.

Appendix II: Chief Information Officer IT Management Reporting Requirements

The following table lists the 36 OMB requirements for agency Chief Information Officer reporting on IT management, along with the source of requirement and the year that the requirement was established.

Appendix III: Survey of Federal Agency Chief Information Officers

The following is an abridged version of the CIO reporting requirements survey. This version includes the set of questions for one requirement, which was repeated for each of the requirements in our survey.

Appendix IV: Comments from the Office of Management and Budget

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, the following staff made key contributions to this report: Gary Mountjoy (Assistant Director); Scott Borre; Chris Businsky; Valerie Hopkins; Stuart Kaufman; Lee McCracken; Tarunkant Mithani; and Teresa Smith.