Federal Chief Information Officers: Reporting to OMB Can Be Improved by Further Streamlining and Better Focusing on Priorities
Published: 2015-04-02T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Federal agencies annually invest over $80 billion on IT. As part of overseeing this spending, OMB directs federal CIOs to report on their management of IT in such areas as capital planning and investment management, security, and strategic planning.
GAO was asked to review the usefulness of such CIO reporting requirements. Its objectives were to (1) identify the current IT reporting requirements that agency CIOs are to address for OMB, (2) evaluate the extent to which OMB and agency CIOs use the required information to manage IT, including CIOs' views on the utility of the requirements, and (3) assess any OMB efforts to streamline this reporting. To do so, GAO analyzed OMB memorandums and other guidance to develop a list of CIO requirements and surveyed 24 major agency CIOs on how they used the required information to manage IT. Further, it analyzed OMB documentation and interviewed officials to identify plans to streamline reporting.
section_title: What GAO Found
paragraphs: The Office of Management and Budget (OMB) directs agency chief information officers (CIO) to respond to 36 information technology (IT) management reporting requirements, largely on a quarterly or annual basis, that address several areas key to effective IT management (see figure).
OMB uses the information reported by CIOs to help it oversee the federal government's use of IT, including implementation of OMB's IT reform initiatives such as consolidating data centers and eliminating duplication. A majority of 24 CIOs surveyed that responded reported that 24 of the 36 reporting requirements help only to some to no extent in managing IT and that meeting them took significant effort and cost approximately $150 million to $308 million annually. A number of CIOs further noted that these requirements were not always helpful because, among other things, addressing them did not support agency priorities. Nonetheless, GAO has previously emphasized the importance of OMB's reforms and their associated reporting requirements to improving federal IT management and producing savings. Thus it is concerning that CIOs do not always see value in reporting information essential to these reforms. Establishing a common understanding between OMB and CIOs on the priority of these initiatives and their related reporting requirements will help ensure their success.
OMB has taken steps to streamline CIO reporting requirements, such as changing reporting formats from narratives to performance data. Nonetheless, OMB's efforts do not address challenges identified by CIOs, such as tracking all current requirements and having to use multiple online tools to report information. This is partly because OMB has not solicited feedback in these areas, due to its focus on streamlining reporting in other areas. By not addressing these challenges, OMB is missing opportunities to help CIOs improve the requirements reporting process and its use of information collected to effectively manage and oversee federal IT.
section_title: What GAO Recommends
paragraphs: GAO is recommending that OMB, in collaboration with CIOs, ensure a common understanding of priority IT reforms and their reporting requirements and address proposed reporting improvements and challenges. OMB neither agreed nor disagreed with GAO's recommendations, citing concerns with, among other things, GAO's survey methodology, stating it did not fully support the report's findings and recommendations. GAO believes these concerns are largely unfounded and that its recommendations are still valid.

section_title: Letter
paragraphs: The federal government annually invests more than $80 billion on  information technology (IT). As part of budgeting for and overseeing this  spending, the Office of Management and Budget (OMB) directs federal  agency chief information officers (CIO) to report on their management of  IT in such areas as capital planning and investment management,  security, and strategic planning. The goal of these reports is to, among  other things, optimize investment of IT funds and address long-standing  federal agency IT management problems.
You asked that we review the usefulness of such CIO reporting  requirements. Our objectives were to (1) identify the current IT reporting  requirements that agency CIOs are to address for OMB; (2) evaluate the  extent to which OMB and agency CIOs use the required information to  manage IT, including CIOs’ views on the utility of the requirements; and  (3) assess any OMB efforts to streamline this reporting.
To address these objectives, we obtained and analyzed OMB  memorandums and other guidance to develop a list of CIO requirements  that were regular, repeating, or one-time requests. Since there could be  several requirements for information in multiple OMB memorandums for  one initiative, we grouped the requirements to report information together  by initiative and the frequency of reporting rather than list each as its own  separate requirement. In doing this, we had all agencies in our review  (the 24 Chief Financial Officer (CFO) Act agencies) and OMB review our  list and provide feedback to help ensure the list was complete and  accurate. Requirements related to activities such as information collection  and control of paperwork; records management; privacy and compliance  with the Privacy Act; and information disclosure and compliance with the  Freedom of Information Act were not included because these activities  are not directly related to IT management responsibilities. In addition, we  obtained and analyzed OMB documentation and interviewed OMB  officials to determine the extent to which they use the information  reported by agencies to further the goal of improving the management of  federal IT; we also conducted a web-based survey of the 24 CFO Act  agencies to obtain information on how they used the required information  to manage IT. All 24 agencies completed the survey, although not all  survey respondents answered every question. Further, we analyzed OMB  and Federal CIO Councilassess current and future plans to streamline CIO reporting and the  extent to which these efforts assist OMB’s goal of reducing CIO reporting  burden.   documentation and interviewed officials to  We conducted this performance audit from December 2013 to April 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Details on our objectives,  scope, and methodology are in appendix I.
subsections: section_title: Background
paragraphs: Over the years, Congress has enacted various laws in an attempt to  improve the government’s management of its IT resources. In doing so, it  has provided OMB with broad IT management and oversight  responsibilities and given agencies a wide range of IT-related  responsibilities. With regard to CIO responsibilities relative to IT  management, we have previously identified major areas that are either  statutory requirements or are critical to effective IT management.areas include:  IT strategic planning: Plans for, among other things, using IT to help  agencies improve the productivity, efficiency, and effectiveness of  their business processes with the overall goal of achieving and  supporting agency missions.
Capital planning and investment management: The process of  selecting, controlling, and evaluating IT investments to produce  business value, reduce investment-related risks, and increase  accountability and transparency in the investment decision-making  process.  IT security: Establishment of a risk-based program that ensures  agency-wide compliance with requirements to protect information and  systems, including implementing requisite controls that prevent, limit,  or detect access to computer networks, systems, or information.
Systems acquisitions, development, and integration: Obtain the  skilled staff, disciplined processes, and tools necessary to develop  and acquire IT system capabilities on time and within budget,  including ensuring such capabilities interoperate as intended with  existing (legacy) systems.
E-government initiatives: A wide range of activities across the  federal government involving the use of the Internet and other  emerging technologies to improve public access to government  information and services.
To carry out their responsibilities, OMB (including its Office of E- Government and Information Technology, headed by the Federal CIO)  issues directives to the agencies such as circulars, memorandums, and  reporting instructions; these directives contain requirements for agency  CIOs to, among other things, report on their IT activities. For example,  OMB Circular A-11 requires agencies to provide information related to  their IT investments, including agency exhibit 53s and capital asset plans  and business cases (called exhibit 300s). In addition, in December 2011,  OMB issued a memorandum that outlines the Federal Risk and  Authorization Management Program guidance for agency adoption and  use of cloud services.
For each reporting requirement, OMB typically identifies how agencies  are to transmit the information. In particular, OMB operates and utilizes  the following web-based systems that the agencies are to use to transmit  their information:
CyberScope: Standardizes manual and automated data inputs for  reporting on Federal Information Security Management Actcompliance and agency privacy programs.  Integrated Data Collection: Allows reporting of structured  information, including agency progress in meeting IT strategic goals,  objectives and metrics, as well as cost savings and avoidances  resulting from IT management actions. These data include information  previously reported by agencies as well as data which agencies shall  report on every 3 months. Updates are to be made on the last day of  February, May, August, and November of subsequent fiscal years.
MAX Portal: Utilized by federal agencies to enter data and upload  documentation related to a variety of reporting activities, including  data required for the President’s Budget and Mid-Session review, and  federal IT investment information.
Federal IT Dashboard: Allows federal agencies to upload cost,  schedule, and performance data on agency major IT investments.
In addition to the reporting system mechanisms listed above, OMB  occasionally requires agencies to provide information by e-mail (usually  for ad hoc requests) or directs agencies to post information on their  websites. For example, with regard to an OMB reporting requirement on  open data policies, OMB directs the agencies to post this information on  their websites.
To help CIOs prioritize their various roles and responsibilities, OMB has  directed CIOs to focus their efforts on the following:
Governance. CIOs should have responsibility over the entire IT  portfolio for the agency, including driving the investment review  process for IT investments, working with CFOs and Chief Acquisition  Officers to ensure IT portfolio analysis is part of the yearly budget  process, and leading TechStat sessions.
Commodity IT. CIOs should focus on eliminating duplication in  commodity IT services (e.g., data centers, e-mail, and web  infrastructure) and rationalize their agency’s IT investments, including  using shared services as a provider instead of standing up separate  services.
Program management. CIOs should improve the overall  management of federal IT projects by identifying, recruiting, and hiring  top IT program management talent and be accountable for the  performance of agency IT program managers.  Information security. CIOs should have the authority and primary  responsibility for implementing an agency-wide information security  program, including having continuous monitoring and standardized  risk assessment processes.
In addition, OMB has implemented a series of initiatives—commonly  referred to by the agency as IT reforms—to, among other things, improve  the oversight of underperforming investments, more effectively manage  IT, and address duplicative investments. The initiatives include the  following:
TechStat reviews. In January 2010, the Federal CIO began leading  TechStat sessions—face-to-face meetings to terminate or turn around  IT investments that are failing or are not producing results. These  meetings involve OMB and agency leadership and are intended to  increase accountability and transparency and improve performance.  Subsequently, OMB empowered agency CIOs to hold their own  TechStat sessions within their respective agencies. OMB has reported  that these efforts to improve management and oversight of IT  investments have resulted in almost $4 billion in savings.
Federal Data Center Consolidation Initiative. Concerned about the  growing number of federal data centers, the Federal CIO (in February  2010) established the Federal Data Center Consolidation Initiative.  The initiative’s four high-level goals were to promote the use of “green  IT” by reducing the overall energy and real estate needs of  government data centers; reduce the cost of data center hardware,  software, and operations; increase the overall IT security posture of  the government; and shift IT investments to more efficient computing  platforms and technologies. OMB estimates that the initiative has the  potential to provide about $3 billion in savings by the end of 2015.
PortfolioStat. In order to eliminate duplication, move to shared  services, and improve portfolio management processes, OMB (in  March 2012) launched its PortfolioStat initiative. It required agencies  to conduct annual agency-wide IT portfolio reviews to, among other  things, reduce commodity IT spending and demonstrate how IT  investments align with agency mission and business functions.  PortfolioStat is designed to assist agencies in (1) assessing the  current maturity of their IT investment management process, (2)  making decisions on eliminating duplicative investments, and (3)  moving to shared solutions in order to maximize the return on IT  investments across the portfolio. OMB estimates that the PortfolioStat  effort has the potential to save $2.5 billion from fiscal year 2013  through fiscal year 2015 by, for example, consolidating duplicative  systems.
Given the importance of these initiatives, OMB has established reporting  requirements to, among other things, track the status of agencies’  implementation of these efforts. In addition, Congress recently  incorporated key aspects of a number of these reforms into law.
Our extensive experience at federal agencies and in particular, our recent  reports on TechStat, data center consolidation, and PortfolioStat,  have shown that these reforms and the required information agencies are  to report on to OMB as part of these efforts offer important opportunities  to improve the efficiency and effectiveness of federal agency programs  and operations, including producing financial savings. The reports also  included recommendations to OMB to improve agency reporting on key  initiatives; the recommendations highlighted the importance of OMB’s  reporting requirements and the need for federal agencies to provide  current and accurate information about the status of these initiatives.  They also highlighted that the requirements are a critical component to  ensuring OMB’s effective management and oversight of the initiatives.
Together, the responsibilities discussed above require CIOs to be key  leaders in managing IT in a coordinated fashion in order to improve the  efficiency and effectiveness of programs and operations.
subsections: section_title: OMB Has Periodically Changed CIO Requirements to Address Changes in Federal IT Management
paragraphs: Over the last several years, OMB has made changes to CIO reporting  requirements to address, among other things, changes in federal IT  management. Such changes included modifying how requirements are  reported, updating what information is requested as part of existing  requirements, and establishing new requirements. For example, in March  2013, OMB issued a memorandum which, among other things,  established the Integrated Data Collection, which was a new way for  agencies to submit information relating to IT reform initiatives such as  PortfolioStat and data center consolidation.
OMB, Fiscal Year 2015 Guidance on Exhibits 53 and 300 – Information Technology and  E-Government (Washington, D.C.: July 1, 2013). agencies provide information on investments that are to be reduced or  eliminated as the result of new investments.
Moreover, in May 2014, OMB issued updated instructions for the  Integrated Data Collection, aimed at improving the quality of data, which  changed the format and information reported for several requirements, as  well as adding a new reporting requirement on progress in using standard  customer value methodologies to evaluate agencies’ highest impact IT  services.
section_title: Prior GAO Reports Have Recommended Improvements to IT Reform Initiatives and Associated Reporting
paragraphs: During the past several years, we have reported on a variety of issues  related to CIOs’ roles and responsibilities and OMB’s management and  reporting of information obtained through federal agency reporting  requirements. For example, in September 2011, we reported on the  roles and responsibilities of agency CIOs. Specifically, we found that  although most CIOs are responsible for major areas of IT (e.g., capital  planning, IT strategic planning, and e-government initiatives), they are  less frequently responsible for other information management areas (e.g.,  records management and privacy) that, despite being required by law, are  considered not critical to effective IT management. We recommended  that OMB update its guidance to establish measures of accountability for  ensuring that CIOs’ responsibilities are fully implemented and require  agencies to establish internal processes for documenting lessons  learned. OMB agreed with our recommendations and stated that it had  taken actions that it believed addressed the recommendations; we are  currently in the process of validating whether these actions fully address  our recommendations.
In addition, in November 2013, we found that of the 26 major federal  agencies that were required to participate in the PortfolioStat initiative— an annual agency-wide portfolio review—only 1agency addressed all of  the key requirements. We also determined that OMB’s estimate of $2.5  billion in savings from PortfolioStat was understated because it, among  other things, did not include estimates from the Departments of Defense  and Justice. Our analysis, which included these estimates, showed that  agencies were reporting at least $5.8 billion in potential savings. Further,  not all agencies provided sufficient support for their estimated potential  savings on consolidation initiatives. We recommended that OMB and the  agencies improve their PortfolioStat implementation, and the parties  agreed in large part with our recommendations; we are currently following  up to assess their progress in doing so.
Further, in December 2013, we reported that although the accuracy of  ratings on the Federal IT Dashboard had improved, they were   In addition, we found inconsistent among the agencies we reviewed. that the public version of the Dashboard was not updated for 15 of the  past 24 months, and so was not available as a tool for investment  oversight and decision making. We recommended that OMB make  Dashboard information available independent of the budget process and  agencies appropriately categorize IT investments and address identified  weaknesses. OMB neither agreed nor disagreed with our  recommendations. Nonetheless, we have ongoing work to assess the  extent to which OMB and agencies have implemented the  recommendations.
GAO-14-713. savings on as many as 67 data centers because of difficulties, such as  calculating baseline data center costs. Further, we found that OMB had  developed metrics, but these metrics do not address server utilization.  Consequently, we recommended that OMB, among other things, develop  and implement a metric for server utilization and agencies address their  challenges in reporting costs savings. OMB and the agencies agreed with  our recommendations. We have initiated follow-up efforts to assess  agency progress in implementing our recommendations.
section_title: Agency CIOs Are to Address 36 IT Management Reporting Requirements for OMB
paragraphs: OMB directs agency CIOs to respond to 36 IT management reporting  requirements. These 36 requirements—which we organized by key IT  management areas such as IT strategic planning, IT security, and related  initiatives—are shown in table 1 along with a description of each requirement; how often (i.e., the frequency) required information is to be reported  (e.g., monthly, quarterly, annually); and how agencies are to report required information (the reporting  mechanism). OMB specifies for each requirement, the reporting  mechanism to be used, which range from posting information on an  agency’s website or OMB’s IT Dashboard to transmitting it to OMB via  the MAX Portal or the Integrated Data Collection system.
Additional details about these requirements, including when each  requirement and its associated OMB guidance was initiated, are provided  in appendix II.
As shown in figure 1, of the 36 requirements, the largest number are in IT  security (12), and the fewest are in IT strategic planning (2).
In terms of reporting frequency, agency CIOs are largely required by  OMB to report on the 36 requirements on a quarterly or annual basis.  Specifically, 29 of the 36 requirements are required to be reported  quarterly and/or annually. Further, several requirements are required to  be reported at multiple periods. For example, the cybersecurity plan of  action is required to be reported both quarterly and annually. In addition,  agencies are required to submit major IT investment documentation  annually and as needed (e.g., when significant changes occur to an  investment).
The other seven are to be reported as follows: one biennially, one semi-annually, one monthly, one one-time, one continuously, and two as-needed.
According to officials from OMB’s Office of E-Government and  Information Technology, OMB established the particular reporting periods  to ensure it gets the information that it needs at the proper time to achieve  its mission to, among other things, develop the President’s budget, make  informed policy decisions, provide oversight, and meet statutory  requirements. For example, OMB requires agencies to report quarterly on  cost savings and avoidances, which assists OMB in publishing its  quarterly report to Congress on progress with IT oversight and reform.  Further, OMB requests that agencies report annually on the  implementation status of their e-government initiatives, which assists  OMB in developing its annual report to Congress on federal e- government. OMB also requests certain requirements to be reported at  multiple periods. For example, agencies are required to report multiple  iterations of their exhibits 53s and 300s in accordance with a schedule  developed by OMB, which assists OMB in developing the President’s  budget.
OMB has the agencies report these requirements via four primary  mechanisms—namely, the federal IT Dashboard, the Department of  Homeland Security’s CyberScope portal, OMB’s Integrated Data  Collection system, and agency websites. More specifically, of the 36  requirements, agencies use
CyberScope for 5 requirements, the Integrated Data Collection system for 7 requirements, and agency websites for 9 requirements.  the federal IT dashboard for 6 requirements,  Agencies also report eight requirements using other mechanisms,  including one via Data Point, one via a Department of Homeland Security information security one via the Federal Data Center Consolidation Initiative program  management office portal,  two via meetings with OMB officials,  three via MAX Portal, and  two via e-mail.
In terms of OMB’s IT reform initiatives, there are a number of the 36  requirements related to managing and overseeing OMB’s efforts in these  areas. Key examples include the requirements on:
Agency TechStat outcomes,
Data center closures/status update,
PortfolioStat progress report,
PortfolioStat progress review, and
Commodity IT baseline updates.
section_title: Although OMB Uses Required Information, CIOs Reported That the Majority of Reporting Requirements Are Not Useful for Managing IT and Identified Areas for Improvement
paragraphs: See, for example, GAO-13-524, GAO-14-713, and GAO-14-65. consolidation, and PortfolioStat—still have important value. Accordingly, it  is concerning that CIOs do not always see value in these reporting  requirements. Consequently, effectively addressing proposed changes  and aligning CIOs’ priorities to OMB’s (i.e., establishing a common  understanding of what the priorities are) is important to, among other  things, the success of OMB’s reforms and its goal of improving federal IT.  Until this is done, there is a risk that the IT reforms will not succeed.
subsections: section_title: OMB Utilizes Reported Information to Meet Its Responsibilities
paragraphs: According to officials from OMB’s Office of E-Government and  Information Technology, OMB utilizes all the information reported by  agency CIOs to carry out, among other things, its budget development,  policy formulation, and oversight roles and responsibilities. Specifically, it  uses the information to undertake the following activities:
Development and execution of the President’s budget. Each  year, OMB and federal agencies work together to determine how  much the government plans to spend on IT projects and how these  funds are to be allocated. OMB coordinates with federal agencies to  obtain agency budget requests, and other information through the  exhibit 53, exhibit 300, and annual FISMA reporting. OMB uses this  information to analyze the requests, and prepare budget materials for  the President’s review. These budget materials also include an  analytical assessment that, among other things, provides details on  the federal IT budget and the administration’s key federal IT  initiatives.
Formulation of policies and guidance for the management of  federal agency IT. OMB issues policy guidance and memorandums  related to various aspects of IT management in order to improve the  management, oversight, and transparency of the federal  government’s IT. As part of these activities, OMB uses the information  reported by agencies to inform policy decisions. For instance, OMB  uses the reported information from the Federal Risk and Authorization  Management Program to evaluate agency progress in implementing  cloud services, which is an OMB policy priority.
Oversight of federal agency IT. OMB’s Office of E-Government and  Information Technology is also responsible for oversight of federal  information technology spending, and more than $80 billion is  annually invested in federal IT. OMB provides oversight through  several mechanisms including the Federal IT Dashboard, PortfolioStat  reviews, and TechStat sessions. For instance, OMB requires  agencies to report information on their IT portfolio, including  commodity IT baselines, and information related to the Federal  Information Security Management Act, as well as develop an  Information Resources Management Strategic Plan and Enterprise  Roadmap. OMB’s goal is to use the information it collects from  agencies to monitor federal IT spending and help ensure programs  and operations are efficient and effective.
Meeting statutory requirements. Under federal law, OMB’s Office of  E-Government and Information Technology is required to report to  Congress on certain IT management areas. In particular, the office is  required to submit a report on the implementation of the E- Government Act of 2002, which summarizes information reported by  agencies as required under the act. In addition, during the period of  our review, OMB was also required to submit a report on the  implementation of the Federal Information Security Management Act  of 2002 by federal agencies. In order to prepare these reports, OMB  requires agencies to submit information on their implementation  efforts as required under the acts, which OMB then summarizes for  Congress.
section_title: CIOs Identified Reporting Requirements Most Useful for Managing IT and Those That Were Less Useful
paragraphs: The 24 agency CIOs we surveyed reported that addressing certain  reporting requirements assisted their agency in managing IT, while  addressing other reporting requirements were not as useful. Specifically,  a majority of 24 CIOs surveyed that responded reported that addressing 4  reporting requirements helped their agency to manage IT to a very great  extent or great extent and 8 helped the agency to manage their IT to a  moderate extent. They also reported that addressing the 24 remaining  reporting requirements helped agencies only to some to no extent in  managing IT.
Table 2 lists the 36 reporting requirements by the extent of assistance in  managing IT as reported by agency CIOs. Specifically, it shows how the  majority of CIOs (including the number) rated each requirement against  our categories of usefulness—either very great to great, moderate, or  some to no extent.
In addition, of the 24 requirements that CIOs found only assisted their  agency in managing IT to some to no extent, our analysis showed that a  number of them were associated with OMB’s IT reform initiatives. In  particular, they include the requirements on
Agency TechStat outcomes,
Data center closures/status update,
PortfolioStat progress report,
PortfolioStat progress review, and
Commodity IT baseline updates.
section_title: CIOs Reported That Addressing Requirements That Were Not Always Useful Took a Significant Level of Effort to Implement
paragraphs: With regard to effort required to meet the 36 reporting requirements, the  majority of the 24 agency CIOs reported that
16 requirements required a very great or great effort to meet,
14 required a moderate effort, and
6 required some to no effort to meet.
In addition, in terms of resources, agency CIOs estimated that they spend  in total, approximately $150 million to $308 million annually to address the  36 reporting requirements. For individual requirements, agencies’  estimates of the range spent were generally at least a total of $1 million  annually and as high as $19 million for one requirement (exhibit 300).
The majority of the 24 agency CIOs that responded also reported that the  level of effort and resources reported to meet the requirement was  sometimes greater than the extent to which addressing the reporting  requirement assisted the agency in managing their IT. Specifically, of the  24 reporting requirements that provided some to no assistance in  managing IT, 8 of these required very great or great effort to meet and 10  required moderate effort to meet. In addition, 4 of the 8 reporting  requirements that helped agencies to a moderate extent in managing IT  required a very great to great effort to meet.
Further, 26 out of 36 reporting requirements that CIOs reported assisted  agencies in managing IT to some or no extent were estimated to cost  agencies approximately $76 million to $164 million each year to meet. In  addition, the 8 requirements that assisted agencies to a moderate extent  were estimated to cost approximately $50 million to $92 million a year.  However, the 4 key reporting requirements that helped agencies manage  their IT to a very great or great extent were estimated to cost $24 million  to $52 million a year.
Table 3 lists the 36 reporting requirements by the CIO reported  usefulness of assistance in managing IT, the level of effort required, and  the estimated annual cost to meet the requirement. More specifically, it  shows how the majority of CIOs that responded rated each requirement  against our categories of levels of effort—either very great to great,  moderate, or some to no effort.
In addition to the fact that a majority of the 24 agency CIOs that  responded reported that complying with many of the reporting  requirements was not always commensurate with their usefulness in  managing IT, they also generally indicated they would only collect at least  some, but not all of the information, if addressing the requirements was  optional. Specifically, for the 24 requirements that helped some to no  extent, the majority of the CIOs reported that they would collect at least  some but not all of the information if not required to do so.
According to comments from a number of CIOs, they did not always find  that these requirements were useful because addressing them did not  always clearly support departmental priorities. For example, with regard  to reporting investment information using the exhibit 300, three CIOs said  it had little value beyond reporting information that OMB needed to make  decisions because their departments had their own processes for  investment decisions. For IT security quarterly reporting and monthly IT  security data feeds, three CIOs said these requirements were not  commensurate with their usefulness because they were burdensome due  in part, to the reporting format, frequency, and duplicative nature of  certain elements. Other examples cited by CIOs include the following:
For government-wide tracking of cybersecurity resources, one agency  CIO commented that it was helpful to determine how much funding  was spent on cybersecurity, but providing the supporting detailed  accounting of the resources called for in the requirement was difficult.  Another CIO commented that tracking resources helped in  understanding the investments made and historical data provided  insight into whether prior allocated resources were impactful;  however, the reporting requirement needed to be consolidated with  annual Federal Information Security Management Act reporting.
Concerning commodity IT baseline updates, one agency CIO  commented that it had helped with understanding the types of  commodity spending that made up the agency’s portfolio and  identified opportunities for optimization but reporting needed to be  combined with other annual budget reporting. Another CIO  commented that while reporting this information helped OMB provide  oversight, the agency would prefer if OMB used the information to  help agencies develop better strategies and operations plans that  would result in cost reductions.
Regarding program management cost savings and avoidances, an  agency CIO commented that while tracking cost savings provides a  more robust understanding of its IT portfolio, the reporting is too  frequent for capturing the cost savings. Another CIO noted that the  reporting helps identify initiatives that are successful in driving down  costs and those that are falling short of projected savings, but  providing updates on changes to the cost savings and avoidance  figures to OMB is burdensome.
section_title: CIOs Proposed Changes to Improve Reporting Requirements and OMB Feedback
paragraphs: Although agency CIOs surveyed generally found some of OMB’s  initiatives valuable for managing IT resources, at least 8 or more  proposed changes to improve (1) 13 reporting requirements and (2)  OMB’s feedback to agencies on the reporting requirements generally.
With regard to the 13 reporting requirements, at least nine agency CIOs  proposed changing what information should be reported under 3 reporting  requirements, stating that data elements that do not add value in terms of  what OMB needs or uses to make decisions or are no longer relevant  should be removed. Agency CIOs also proposed changing the frequency  of reporting for 5 requirements, moving from reporting on a quarterly  basis to either a semi-annual or annual basis. In addition, at least eight of  the CIOs proposed that 4 reporting requirements should be eliminated  because they were generally either not useful to the agencies in  managing their IT, information was duplicative with other reporting  requirements, or OMB had not requested the information in recent years.  Table 4 lists the reporting requirements, the agency CIOs’ proposed  changes, and the number of CIOs that proposed them.
Officials from OMB’s Office of E-Government and Information Technology  stated that they had received similar feedback on proposed changes to  the reporting requirements in the past. They said that in some cases there  was confusion among agency officials regarding the reporting  requirements. Officials noted that the feedback on the requirements was  useful information but provided no specific plan or date for addressing  these suggestions. Effectively addressing such proposed changes is  important to improving the efficiency and effectiveness of reporting  requirements and could better position OMB to achieve its goal of  improving management, oversight, and transparency of federal IT. Until  this is done, OMB risks requiring agencies to implement and report on  requirements that are duplicative, wasteful, or inefficient.
With regard to improving feedback on reported information, agency CIOs  suggested that OMB’s feedback process could be improved. In particular,  while agency CIOs reported that OMB provided feedback to them on the  majority of the 36 reporting requirements, the majority of CIOs reported  that the feedback was moderately effective to not effective for most  reporting requirements. Six agency CIOs also reported that they were  specifically interested in receiving better feedback on two reporting  requirements—namely, the major IT investment documentation and the  Open Government directive. The Office of E-Government and Information  Technology officials stated that the information on the feedback,  particularly those requirements agency CIOs were interested in receiving  feedback on, was useful; nevertheless, the officials acknowledged that  they do consistently not provide this level of feedback to the CIOs  because in part, they did not know until now that the CIOs wanted  feedback to this extent. Having a process that consistently provides  effective feedback is key to helping agency CIOs better manage their IT  resources and improve reporting; it is also consistent with OMB’s goals to  improve federal IT management, oversight, and transparency. Until an  effective feedback process is in place, there is a risk that agencies are  managing their IT in a suboptimal manner.
As mentioned previously, our extensive experience at federal agencies  and recent reports have shown that a critical component to ensuring  OMB’s effective management and oversight of key IT reform initiatives— specifically, TechStat, data center consolidation, and PortfolioStat— is agency reporting of current and accurate information about the status  of these initiatives, including the extent of any financial savings. However,  agency CIOs surveyed reported that requirements related to these IT  reform efforts, including agency TechStat outcomes, data center  closures/status updates, commodity IT baseline updates, and  PortfolioStat reviews and progress reports, helped their agency to only  some or to no extent in managing IT. It is concerning that CIOs do not  always see the value in reporting information that is essential to reform  initiatives aimed at improving IT management effectiveness, saving  money, and avoiding unnecessary costs, especially since key aspects of  a number of the reforms have also been recently incorporated into law.
Consequently, establishing a common understanding between OMB and  the CIOs on the priority of the reporting and related initiatives is key to the  success of OMB reforms. As part of this understanding, it is also  important to address underlying reasons cited by CIOs regarding the  usefulness of requirements, including when department priorities are  reportedly different than OMB’s and the burdensome and duplicative  nature of requirements. Until such an understanding is established, there  is a risk these important IT reforms, which are key to improving the  efficiency and effectiveness of federal agency programs and operations,  will not fully succeed.
section_title: OMB Has Initiated Efforts to Streamline Reporting, but They Do Not Address Challenges Reported by CIOs
paragraphs: OMB has taken steps to streamline CIO reporting requirements.  Specifically, OMB has initiated efforts to identify opportunities to change  the format for reporting (e.g., from narrative-intensive descriptions to  specific performance data) with the goal of reducing CIOs’ reporting  burden. Although these OMB efforts aim to streamline reporting, agency  CIOs identified additional challenges with tracking what reporting  requirements are currently in place, using multiple online tools to report  required information, and using capital planning and investment reporting  requirement information to make effective investment decisions, which  are not addressed by OMB’s efforts. This is in part to the fact that OMB  has not solicited feedback in these areas because its priority has been on  streamlining reporting in other areas (e.g., changing report formatting and  others discussed below). By not addressing these CIO-identified  challenges, OMB is missing opportunities to help CIOs improve the  requirements reporting process and to improve its use of information  collected as part of this process to effectively manage IT.
subsections: section_title: OMB Is Working to Streamline Reporting
paragraphs: OMB has initiated several efforts to streamline reporting:
Changing the format and mechanism of information that is  submitted. Officials from OMB’s Office of E-Government and  Information Technology reported that they are examining transitioning  the reporting of certain requirements from previous narrative-intensive  plans or other documentation into structured forms (such as the  Integrated Data Collection) that call for specific data associated with  OMB-established performance metrics. These officials stated that  they are continually re-evaluating the format in which agencies should  provide information in order to improve the efficiency of OMB’s review  process and the reliability of pertinent federal IT management data.  For example, OMB expanded and refined its collection of key  performance indicators for investment portfolio management with the  Integrated Data Collection between 2013 and 2014.
Revising what information is currently required to be submitted  as part of existing requirements. The Office of E-Government and  Information Technology officials stated that they recently began  reviewing existing reporting requirements in the Integrated Data  Collection prior to issuing quarterly reporting instructions to identify  whether any elements should be changed. For example, in May 2014,  as the result of one of these reviews, OMB removed the commodity IT  baseline portion of the Integrated Data Collection for the submission  due in November 2014. In addition, OMB reported it is working with  the CIOs to improve the value of agency reporting on the federal  center consolidation initiative. Specifically, OMB reported that working  via a task force that is part of the OMB-led CIO Council, OMB has  helped to develop metrics and data collection requirements that best  support administration goals associated with consolidating and  optimizing data centers. OMB further reported that as a result of these  efforts, it has made significant changes to both its reporting  requirements and strategic approach associated with this initiative.
Incorporating lessons learned into the annual revision of  PortfolioStat guidance. The officials from the Office of E- Government and Information Technology also stated that in 2013 they  initiated an annual review of lessons learned from agency  PortfolioStat sessions to identify whether additional information is  needed to improve the office’s oversight of federal IT portfolio  management. These officials added that they incorporated the results  of these reviews into the 2014 PortfolioStat guidance, which may have  included adding requirements or streamlining others. For example,  OMB required agencies to identify IT investments that merit additional  oversight and support for review and discussion during the 2014  PortfolioStat sessions. Selected agencies were required to develop an  action plan with specific goals and targets for these high-impact  investments.
Integrating requirements with other IT government-wide efforts.  The Office of E-Government and Information Technology officials  added that they review existing requirements to identify potential  changes, including opportunities to streamline, when new  government-wide IT initiatives are introduced. For example, they  reported that their office is currently in the process of evaluating CIO  and other reporting requirements to determine how they align to  OMB’s strategic cross-agency priority goals associated with smarter  IT delivery, cybersecurity, and Open Databe opportunities to further streamline existing reporting requirements.
In October 2013, OMB and the Federal CIO Council established a  working group to streamline reporting requirements related to capital  planning and investment control. The working group studied, among other  things, whether IT investment performance information and related  reporting to OMB could be streamlined and better aligned with what  information was needed by CIOs to make informed investment decisions.  As a result of the study, the working group made a number of short-term  and long-term recommendations to OMB in April 2014 to improve its fiscal  year 2016 IT budget capital planning guidance. According to OMB officials, they addressed certain short-term recommendations related to  improving standard definitions and purchase provisions, and are working  to address the long-term recommendations in future guidance. They also  said they expect the working group to continue studying the issues  associated with this area and to annually provide recommendations on  improving and streamlining capital planning guidance.
section_title: OMB Efforts Do Not Address CIO-Identified Challenges in Meeting Reporting Requirements
paragraphs: Although OMB has initiated several efforts to further streamline CIO  reporting requirements, its efforts do not address the following challenges  agency CIOs identified in our survey in meeting reporting requirements:  Tracking what reporting requirements are currently in place can be  confusing. Agency CIOs expressed confusion in our survey on whether  certain reporting requirements (e.g., the IT Capital Plan and compliance  failures) were still in effect. This confusion was due in part to neither OMB  nor the majority of federal agencies we surveyed having a comprehensive  list of current CIO and related reporting requirements. In particular,  officials from the Office of E-Government and Information Technology  office noted that while they did not maintain a comprehensive list, they did  have a spreadsheet they used for selected review processes (e.g.,  PortfolioStat) but that this list was not available to the federal agencies  because it was an internal OMB working document. Having a  comprehensive list of current CIO reporting requirements is important for  agencies to effectively manage their resources and respond to these  requirements and for OMB to effectively manage its streamlining efforts  consistent with its goal of reducing CIO reporting burden. Without such a  list, OMB may lack sufficient information to make informed decisions  about streamlining efforts and agencies risk wasting resources  responding to reporting requirements that have changed or are no longer  in effect.
Having multiple online reporting tools takes additional time and  resources to enter required information and can be duplicative.  Agency CIOs reported in our survey that having to enter data into multiple  online tools for reporting required information as part of capital planning,  system integration, and IT security takes additional time and resources  and can be duplicative. OMB requires agencies to use three online tools  (Data Point, IT Dashboard, and the Integrated Data Collection) as well as  e-mail for capital planning and system integration reporting requirements,  and four online tools (Cyberscope, MAX Portal, Integrated Data  Collection, and the Information Security Continuous Monitoring  Dashboard) for IT security. In particular, agency officials noted that  certain information for the capital planning reporting requirements (exhibit  53, exhibit 300) and cybersecurity reporting requirements (IT security key  metrics, monthly IT security data feeds, IT security quarterly reporting)  must be entered, in accordance with OMB’s direction, in each tool; some  information must be reported by bureau or office; and there are limitations  in uploading data using a spreadsheet or multiple attachments into the  tools, all of which takes additional time and resources.
OMB officials stated that they were looking at what reporting  requirements can be reported on through the tools, particularly the  Integrated Data Collection, and what steps can be taken to simplify the  tools. They also noted that there were challenges when determining  whether to reuse an existing tool or create a new one for reporting  requirements, particularly when they needed agencies to report  information within a short time frame, but provided no specific date or  plan for the completing the above described efforts, including addressing  the related challenges. Having centralized tools for reporting required  data is critical to reducing the reporting of duplicative data, as well as  reducing the time and resources required to enter the data, and would be  consistent with OMB’s goal of reducing CIO reporting burden. By not  including efforts to streamline reporting tools as part of OMB’s overall  requirements streamlining effort, there is a risk that agencies will continue  to expend valuable time and resources entering potentially duplicative  data into multiple reporting tools.
Using capital planning and investment reporting requirement  information does not provide information to make effective  investment decisions. As mentioned previously, agency CIOs reported  in our survey that six of the seven capital planning and investment  reporting requirements—namely the exhibit 53, exhibit 300, major IT  investment documentation, IT Capital Plan, PortfolioStat progress report,  and compliance failures—needed improvement. For example, agency  CIOs reported that the two key capital planning reporting requirements— the exhibit 53 and exhibit 300—needed to be improved to remove data  elements that did not provide value in making decisions. For the exhibit  53, this included eliminating non-financial data elements, such as those  on cloud computing, and for the exhibit 300, eliminating the contracts and  acquisition information since it was reported elsewhere. In addition,  agency CIOs proposed eliminating three of the capital planning and  investment reporting requirements (e.g., major IT investment  documentation, IT Capital Plan, and compliance failures) because  agencies had either not provided this information in recent years or  information was reported elsewhere in accordance with other reporting  requirements. By not focusing on efforts to streamline and improve capital  planning and investment reporting requirements, there is a risk that OMB  will not be able to adequately respond to the evolving nature of IT and  ensure these key reporting requirements include data elements that  provide the necessary value in making critical investment decisions.
With regard to the above challenges, the OMB officials told us that they  were not currently addressing them because their top concern was the  other requirement streamlining areas (i.e., changing the format and  incorporating lessons learned) they were focusing on. They added that  they had not solicited feedback from the agencies in the areas that were  the focus of our survey and thus were not aware of the challenges. The  officials also noted that the feedback we provided from the CIO surveys,  including the challenges currently faced by CIOs, was useful information  to consider as part of their further streamlining efforts. Nonetheless, until  these challenges are addressed, OMB risks not achieving its goal of  reducing CIO reporting burden and is missing opportunities to help CIOs  improve the reporting process, and their use of information collected as  part of this process, to effectively manage IT.
section_title: Conclusions
paragraphs: OMB has established an extensive framework of IT management  reporting requirements to aid in carrying out its mission and to help  federal agency CIOs manage their IT resources. Although OMB uses  reported information to meet its responsibilities, the majority of the 24  CIOs for the most part, do not, citing that (1) two thirds of the reporting  requirements are not very useful to them in managing IT and (2)  addressing reporting requirements entails a significant effort on the part of  the CIOs, including an annual multi-hundred-million-dollar financial  commitment. Nonetheless, our extensive experience at federal agencies  and recent work have shown that reporting requirements—in particular,  those relating to TechStat, data center consolidation, and PortfolioStat— are key to carrying out OMB’s IT reforms and their goal of improving  federal agency programs and operations, including producing financial  savings. Consequently, it is of particular concern that OMB and the CIOs  are not fully aligned on the utility of reporting requirements integral to the  success of OMB’s reforms. This misalignment is due in part to a lack of a  common understanding between OMB and the CIOs on the priority nature  of these reporting requirements and their associated reforms. Without  such a common understanding, OMB and the CIOs risk missing key  opportunities to improve federal programs and operations and produce  savings. The CIOs did propose a number of changes aimed at increasing  the usefulness of reporting requirements and providing for effective  feedback, but OMB had not yet established an effective approach to  address them. Until it does, OMB risks requiring agencies to report on  and manage IT in a suboptimal manner, which is inconsistent with its goal  of improving federal IT management.
While OMB’s efforts to streamline CIO reporting are steps in the right  direction, they largely do not address the CIO-identified challenges  discussed in this report. These challenges are not being addressed by  OMB in part because it is focusing on other aspects of requirement  streamlining as its top priority and also because of a lack of awareness of  these specific CIO challenges. Until these challenges are addressed,  OMB is missing opportunities to help CIOs improve the requirements  reporting process and its use of information collected as part of this  process to effectively manage IT.
section_title: Recommendations for Executive Action
paragraphs: To improve the effectiveness of OMB streamlining efforts and ensure  agency CIOs are better able to carry out their responsibilities in managing  IT, including implementing OMB’s IT reform initiatives, we recommend the  Director of OMB direct the Federal CIO, in collaboration with agency  CIOs, to take the following four actions:
Ensure there is a common understanding with agency CIOs on the  priority of the current reporting requirements and related IT reform  initiatives. This should include addressing underlying reasons cited by  CIOs regarding the usefulness of requirements, including when  department priorities are reportedly different than OMB’s and the  burdensome and duplicative nature of requirements.
Address and incorporate, as appropriate, the 13 proposed  improvements to reporting requirements made by agency CIOs in our  survey. This should include developing milestones and associated  plans for completing this effort.
Ensure the feedback process on information reported by CIOs  consistently provides effective and constructive responses to these  officials on their reported information.
Assess, as part of ongoing streamlining efforts, the reporting  challenges identified in our report. This should include determining  whether to (1) have a comprehensive list of current IT reporting  requirements that is publically available to agency CIOs; (2) reduce  the number of reporting tools; and (3) improve the utility of capital  planning and investment reporting requirements, and taking steps to  implement where appropriate.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments provided by the Federal CIO and reprinted in  appendix IV, OMB stated it respectfully neither agreed nor disagreed with  our recommendations. According to OMB, it chose to take no position on  the recommendations due to its concerns that: (1) our draft report’s count  of reporting requirements was not currently accurate, (2) our survey  approach did not fully support the report’s findings and recommendations,  (3) the second objective to solicit CIO views did not allow for sufficient  context setting, and (4) OMB has taken steps to solicit feedback and  streamline requirements that are not reflected in the draft report. OMB  added that it agreed with the principles contained in the recommendations  that stress the importance of a common understanding with CIOs and  other key stakeholders on the importance and priority of reporting  requirements, ensuring effective and constructive feedback to CIOs on  the reported information, and streamlining reporting requirements. OMB  further noted that it has actively incorporated these principles in its work  and continuously works to enhance its efforts in these areas.
We evaluated OMB’s concerns and found them, as discussed below, in  large part either unfounded or as having no material impact on our results  and recommendations. Consequently, we stand by our findings on OMB’s  key IT reform initiatives—and their associated reporting requirements— which are critical to improving federal IT management effectiveness.  Further, ensuring the success of these reforms is a key reason why  Congress, in December 2014, incorporated key aspects of a number of  these reforms into law. Accordingly, we believe our recommendations to  OMB are still valid and essential.
With regard to OMB’s first concern about the accuracy of the number of  requirements identified in our report, OMB took issue with our inclusion of  seven requirements, although it provided no documentation to support its  comments; specifically, OMB said (1) two requirements did not exist at the time of GAO’s audit (specifically,  Information Security Continuous Monitoring Dashboard and Cybersecurity  Plans of Action), (2) two had been eliminated by OMB during or shortly after the audit time  frame—namely, the PortfolioStat Review and the Commodity IT Baseline,  and  (3) three appeared to have been had been double-counted (specifically,  the IT Capital Plan and Cloud First are components of the Exhibit 53  requirement, and Open Data Policy Public Data Listing is a subset of  Open Data Enterprise Inventory).
Regarding the first two requirements (Information Security Continuous  Monitoring Dashboard and Cybersecurity Plan of Action), our analysis  shows these requirements were in effect at the time of our audit, and this  is why we therefore included them in our list. Specifically,
OMB memorandum M-14-03 (dated November 2013)agencies, in addition to submitting data to this dashboard once it is  deployed, to report on the status of their preparation for meeting this  requirement starting in 2014, which was in the time frame specified in  our scope and methodology.
OMB memorandum M-12-20 (dated September 2012) stated that The  Department of Homeland Security will “ask agencies to complete a  Plan of Action for improving specific cybersecurity responsibilities.  Agencies will provide quarterly and fiscal year targets and  demonstrate progress toward these targets as they mature their  programs.” Although OMB assigned responsibility to the Department  of Homeland Security for these plans and subsequent quarterly  updates, we considered it to be an OMB reporting requirement  because the reporting requirement was included in an OMB  memorandum.
In addition, for the two requirements that OMB said were eliminated (i.e.,  PortfolioStat review and Commodity IT Baseline), our evaluation shows  that both had not been eliminated as of March 2014, which is the “as-of”  date specified in our scope and methodology. Specifically,
OMB memorandum M-13-09 (dated March 2013) required agencies to  submit a consolidated document of successes, challenges, and  lessons learned to OMB no later than 2 weeks after the transmittal of  the President’s Budget for fiscal year 2015 to Congress (which was  sent on March 4, 2014). Since the requirement was in effect as of  March 2014, we included it in our list.
OMB memorandum M-13-09 also required agencies to provide a  quarterly update of their commodity IT baseline to OMB. Because this  requirement was not eliminated until May 2014, as OMB’s comments  and our report indicated, we therefore included it in our list since the  requirement was in effect as of March 2014.
Thirdly, although OMB stated that we double counted three requirements  (i.e., IT capital plan, Cloud First, and the Open Data Policy Public Data  Listing), the evidence shows otherwise. Specifically, these requirements  are defined in OMB circulars or memoranda (as noted in appendix II) as  separate reporting initiatives, and we therefore treated them as such. For  example, the requirement for an IT capital plan is outlined in OMB  Circular A-130 (dated November 2000) and details several activities that  the agency must provide annually to OMB as part of the budget  submission.
In addition to the above, it is important to note that (as described in our  report’s scope and methodology) after we developed our list of  requirements, we asked all 24 CFO agencies as well as OMB to review  the list and provide feedback as a means to validate that our inventory  was accurate. Specifically, we had officials from the 24 agencies review  our list to ensure it was complete and accurate, adjusted our list as  appropriate based on their feedback, and reached consensus on the  number of requirements in the inventory. In the case of OMB, we had  officials from the Office of E-Government and Information Technology  review our list multiple times and adjusted our list as appropriate based  on their feedback. During these OMB meetings, the officials never raised  the specific issues OMB noted in its written comments.
It is also important to note that we took these validation steps because, as  we noted in our report, OMB does not maintain a comprehensive list of  current CIO reporting requirements. This has left agencies confused  regarding what requirements were currently in place, and is therefore why  we recommended that OMB develop such a list. This inconsistency  among OMB’s written response, what the agencies told us, and what  OMB representatives previously told us confirms the need for GAO’s  recommendation.
OMB’s second concern was that our survey approach did not fully support  the report’s findings and conclusions. Specifically, OMB stated that many  agency CIOs told OMB through the CIO Council that they delegated full  responsibility for completing the survey to lower-level staff and because of  this, OMB did not have full confidence in some of our findings and felt our  attributions was not appropriate. In addition to this concern, OMB said  that it believed GAO gave disproportionate weight to the estimated costs  reported, which appeared, in its opinion, to be unsubstantiated because  they were based on a survey question that did not require the basic  characteristics found in GAO’s cost estimating and assessment guide.OMB further questioned the reliability of these estimates because they  believed staff other than the CIO had completed the survey and were  therefore not experienced enough to provide knowledgeable estimates.
We disagree with OMB’s opinion regarding the validity of our survey  results for the following reasons. First, in meetings with CIOs and  selected senior officials from agencies in our review to discuss our plans  to survey the CIOs, we consistently expressed our expectation that the  CIOs were to complete the survey. If this was not feasible, CIOs could  delegate it to other staff, especially when agencies wanted staff that  worked directly on addressing the reporting requirements to complete the  survey because they were best qualified to respond to our questions. In  these cases, we nevertheless told the agencies the CIOs were ultimately  responsible for reviewing the accuracy of the survey results.
Second, we pretested the survey at seven agencies, again reinforcing our  expectation that CIOs were to complete the survey. Specifically, to  minimize errors that might occur from respondents interpreting our  questions differently from our intended purpose, we pretested the  questionnaire in person and by phone in three rounds of testing with  officials from the office of the CIO at seven agencies. The selection of  agencies for pretesting was based on agency availability to assist us with  pretesting, variation in size of agency, and variation in agency CIO  models (i.e., centralized or decentralized). During these pretests, we  asked agency officials to complete the questionnaire for one reporting  requirement as we listened to the process. We then interviewed the  respondents to check whether the questions were applicable, clear,  unambiguous, and easy to understand.
Third, in transmitting the surveys to the CIOs, we sent them directly to  each of the 24 agency CIOs with instructions indicating that they were to  complete it, with assistance from other officials as appropriate, and that  the results would be summarized in a report to our congressional  requesters. Specifically, in coordination with GAO survey methodology  experts, we developed and administered a web-based survey that we  sent to the CIOs of the 24 CFO Act agencies. All 24 CFO Act agencies  completed the final survey. We then reviewed all responses, and followed  up by phone and e-mail to clarify the responses as appropriate.
The above three points notwithstanding, we have revised the report to  note that CIO responses may have in some cases, come from agency  officials completing the surveys on behalf of the CIOs.
In terms of the estimated costs of meeting reporting requirements, the  application of GAO’s cost guide in this context would be inappropriate,  according to an internal GAO subject matter expert who co-authored the  guide. This expert said the guide is reserved for the evaluation of  government acquisition program cost estimates. The expert added that  given that GAO was not reviewing such programs in this case, OMB’s  assertion is without merit. In addition, we initially asked OMB whether it  had cost information on the reporting requirements but was told it did not  and were directed to the 24 agencies as the best source. Further, we  believe that we have appropriately qualified the range of costs reported  by the agencies and note that we made no recommendations on the cost  of such reporting.
Further, regarding OMB’s comment about the experience of the staff  providing estimates, we expressed our expectation that the CIOs  complete this and the other parts of our survey. However, as previously  discussed, in some cases CIOs and selected staff from the 24 agencies  delegated this responsibility to agency subject matter experts who they  thought were more knowledgeable in this area. In these cases, we again  expressed our expectation that the CIOs at a minimum review the  accuracy of such estimates.
With regard to the OMB concern about our report’s contextual framing of  certain information related to its efforts, the agency stated that it  appreciated our draft report acknowledging that agency reporting is a  critical component of ensuring effective management and oversight of key  IT reform initiatives and that such information is essential to the success  of IT reform efforts; however, it said our draft lacked contextual framing  because it did not the fully address the value of the reporting  requirements beyond agency CIOs’ use of the data. In particular, OMB  stated that these requirements helped ensure the successful  implementation of federal law and/or supported administration priorities,  including IT-related cross-agency priority goals (which OMB commonly  refers to as CAP goalscover areas—specifically, cybersecurity, Open Data, and smarter IT  delivery—in which increased cross-agency collaboration is needed to  improve progress towards shared, complex priorities reaching far beyond  the scope of the priorities of individual federal agencies.  ). According to OMB’s comments, CAP goals  We included a description in our report of the value of the information to  OMB in carrying out federal laws and its statutory roles and  responsibilities. Specifically, consistent with OMB’s comments, we point  out that OMB utilizes reported information to meet statutory  responsibilities. For example, our draft states that under federal law,  OMB’s Office of E-Government and Information Technology is required to  submit a report on the implementation of the E-Government Act of 2002,  which summarizes information reported by agencies as required under  the act. The draft also says that during the period of our review, OMB was  required to submit a report on federal agencies’ implementation of the  Federal Information Security Management Act of 2002. Further, the draft  notes that in order to prepare such reports, OMB requires agencies to  submit information on their implementation efforts as required under the  acts, which OMB then summarizes for Congress.
Our report also discusses the administration’s priorities such as the IT- related cross-agency priority goals. OMB’s written comments  acknowledge this point by stating that our draft report mentions the CAP  goals. However, OMB notes that our draft does not provide essential  context as to how the goals relate to reporting requirements. We  disagree. Our draft stated that the CIO reporting requirements help the  Federal CIO in its efforts to meet the CAP goals. Specifically, the draft  noted that Office of E-Government and Information Technology officials  review existing requirements to identify potential changes, including  opportunities to streamline, when new government-wide IT initiatives are  introduced. For example, these officials reported that their office is  currently in the process of evaluating CIO and other reporting  requirements to determine how they align to OMB’s strategic cross- agency priority goals associated with smarter IT delivery, cybersecurity,  and Open Data and whether there might be opportunities to further  streamline existing reporting requirements Regarding OMB’s fourth concern that our report does not reflect steps it  has taken to solicit feedback and streamline requirements, the agency  commented that it holds quarterly review (feedback) sessions with agency  officials about IDC requirements. It added that these sessions have  resulted in streamlining OMB collection requirements. OMB also noted  that it meets with CIO council members (via a Federal Data Center  Consolidation Initiative task force) to, among other things, help develop  metric and data collection requirements that best support administration  goals in this area. According to OMB, it has changed reporting  requirements and its strategic approach with regard to the data center  initiative as a result of feedback from the task force. In addition, OMB  cited the annual capital planning and investment control guidance it  issues, which includes Exhibits 53 and 300 reporting requirements, as  another example of the agency soliciting feedback.
The streamlining efforts described in OMB’s comments are discussed in  our report, including the role OMB played in soliciting of feedback. For  this reason, we agree with and believe our draft is consistent with OMB’s  comments as it relates to IDC requirements and the annual capital and  control guidance it issues. Regarding its data center consolidation efforts,  we are aware of this effort and have updated our report to show it as  another example of OMB’s efforts to streamline and solicit feedback.  Nonetheless, OMB’s comments provided no explanation or  documentation to show it was addressing the CIO-identified challenges in  our draft that are not currently being addressed by its streamlining efforts.
The above efforts aside, agency CIOs suggested, as noted in our report,  that OMB’s feedback process could nonetheless be improved. In  particular, while agency CIOs reported that OMB provided feedback to  them on the majority of the 36 reporting requirements, the majority of  CIOs reported that the feedback was moderately effective to not effective  for most reporting requirements. Six agency CIOs also reported that they  were specifically interested in receiving better feedback on two reporting  requirements—namely, the major IT investment documentation and the  Open Government directive. Office of E-Government and Information  Technology officials stated that the information on the feedback,  particularly those requirements agency CIOs were interested in receiving  feedback on, was useful.
Nevertheless, the officials acknowledged that they do not consistently  provide this level of feedback to the CIOs because, in part, they did not  know until now that the CIOs wanted feedback to this extent. Having a  process that consistently provides effective feedback is key to helping  agency CIOs better manage their IT resources and improve reporting; it is  also consistent with OMB’s goals to improve federal IT management,  oversight, and transparency. Until an effective feedback process is in  place, there is a risk that agencies are managing their IT in a suboptimal  manner.
We are sending copies of this report to interested congressional  committees, the Director of the Office of Management and Budget, the  secretaries and agency heads of the departments and agencies  addressed in this report, and other interested parties. In addition, the  report is available at no charge on GAO’s website at http://www.gao.gov.
If you or your staffs have any questions on the matters discussed in this  report, please contact me at (202) 512-9286 or pownerd@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) identify the current information technology (IT)  reporting requirements that agency chief information officers (CIO) are to  address for the Office of Management and Budget (OMB); (2) evaluate  the extent to which OMB and agency CIOs use the required information  to manage IT, including CIOs’ views on the utility of the requirements; and  (3) assess any OMB efforts to streamline this reporting.
To identify the current CIO IT management reporting requirements, we  obtained and analyzed OMB circulars, memorandums, and other issued  guidance to develop a current list of requirements to report information to  OMB; the resulting list included regular, recurring, or one-time  requirements that were in effect as of March 2014. We focused on the  requirements that were in effect by the end of the second quarter in fiscal  year 2014 (i.e., March 2014) so that we could use these in our survey of   In addition, since  federal agency CIOs, which was issued in May 2014.there could be several requirements for information in multiple OMB  memorandums for one initiative, we grouped the requirements to report  information together by initiative and the frequency of reporting rather  than list each as its own separate requirement. Requirements related to  activities such as information collection and control of paperwork; records  management; privacy and compliance with the Privacy Act; and  information disclosure and compliance with the Freedom of Information  Act were not included because these activities are not directly related to  IT management responsibilities. Further, requirements directed to other  agency officials but to which the agency CIO contributes were not  included. Further, although OMB occasionally makes ad hoc requests to  agency CIOs to provide information, we did not include those items in our  list because they did not originate from issued guidance.
We categorized the requirements based on their best fit in the following  areas that are typically identified as key CIO IT management  responsibilities: IT strategic planning; capital planning and investment  management; IT security; and system acquisition, development, and  integration. We also had the agencies in our review (the 24 Chief  Financial Officer (CFO) Act agenciesprovide feedback to help ensure it was complete and accurate. Based on  their feedback, we made the necessary changes to our list, including  adding several requirements and removing several requirements that  were no longer in effect.  ) and OMB review our list and  To evaluate the extent to which agency CIOs and OMB use the required  information to manage IT and to assess CIOs’ views on the utility of the  requirements, we first obtained and analyzed OMB documentation and  interviewed OMB officials; we then compared it to OMB’s goal of using  CIO-reported information to improve the management, oversight, and  transparency of federal IT. We also, in coordination with our survey  methodology expert, developed and administered a web-based survey  that we sent to the CIOs of the 24 CFO Act agencies. In meetings with  CIOs and selected senior officials from agencies in our review to discuss  our plans to survey the CIOs, we told them we expected the CIOs to  complete the survey. If this was not feasible, CIOs could delegate the  task to other staff, especially when agencies wanted staff that worked  directly on addressing the reporting requirements to complete the survey  because they were best qualified to respond to our questions. In these  cases, we nevertheless told the agencies the CIOs were ultimately  responsible for reviewing the accuracy of the survey results.
Using the survey, we requested information on, among other things, the  usefulness of the reporting requirements in managing IT, the level of  effort required to meet them, and whether CIOs thought the requirements  should be changed. For our analysis and reporting of the questions on  usefulness and level of effort, we grouped the reported responses into  three categories (i.e., very great to great, moderate, and some to no  extent) and reported the category that had a majority response, or the  largest number of responses in cases where the total number of  respondents for a particular requirement was less than 24. In instances  where the number of responses was evenly, or close to evenly, divided  between the “very great to great” and “some to no extent” categories, we  chose the moderate category, as it provided the best overall  representation of the views of the agency CIOs for that requirement. For  the question on proposed changes to the reporting requirements, we  provided six options of possible change for agency CIOs to choose from  for each reporting requirement (i.e., frequency, mechanism, information  provided, consolidation, elimination, or other). We determined that at least  eight responses or one third of respondents suggesting a particular  change represented a sufficient consensus of agency CIO views for  reporting purposes.
In addition, for each requirement, we asked agency CIOs to estimate the  annual amount (in government and/or contractor dollars) that agencies  spent on meeting each reporting requirement. For each reporting  requirement, we provided four choices in the survey to choose from ($1- $99,000; $100,000-$299,000; $300,000-$1,000,000; and more than  $1,000,000). We decided not to offer more precise dollar amount options  to choose from because, among other things, agencies do not typically  keep track of funds spent on CIO reporting requirements because this is  not required by OMB and therefore the benefits of having more precise  amounts were outweighed by the time and effort agencies would have  had to expend to develop such precise estimates. For our analysis of how  much agencies spend annually on reporting requirements, since the  option “more than $1,000,000” did not include an upper range, we worked  with a survey methodology expert to define the lower and upper range of  this choice to be the same, which is $1 million plus one dollar or  $1,000,001. See appendix III for the list of survey questions.
To minimize errors that might occur from respondents interpreting our  questions differently from our intended purpose, we pretested the  questionnaire in person and by phone in three rounds of testing with  officials from the office of the CIO at seven agencies. The selection of  agencies for pretesting was based on agency availability to assist us with  pretesting, variation in size of agency, and variation in agency CIO  models (i.e., centralized or decentralized). During these pretests, we  asked agency officials to complete the questionnaire for one reporting  requirement as we listened to the process. We then interviewed the  respondents to check whether the questions were applicable, clear,  unambiguous, and easy to understand. All 24 CFO Act agencies  completed the final survey, although not all survey respondents answered  every question. We then reviewed all responses, and followed up by  phone and e-mail to clarify the responses as appropriate.
The practical difficulties of conducting any survey may introduce non- sampling errors. For example, differences in how a particular question is  interpreted, the sources of information available to respondents, or the  types of respondents who do not respond to a question can introduce  errors into the survey results. We included steps in both the data  collection and data analysis stages to minimize such non-sampling errors.  We examined the survey results and performed computer analyses to  identify inconsistencies and other indications of error, and addressed  such issues as necessary. We analyzed responses to closed-ended  questions by counting the response for all agencies. For questions that  asked respondents to provide a narrative answer, we compiled the  answers in one document that was analyzed and used as examples in the  report.
To assess any OMB efforts to streamline this reporting, we obtained and  analyzed OMB and Federal CIO Council documentation and interview  information to summarize OMB’s current and future plans for streamlining  CIO reporting and compared these efforts with OMB’s goal to reduce CIO  reporting burden. As part of this, we interviewed officials from OMB and  the Federal CIO’s Capital Planning and Investment Control Community of  Practice key working group to identify current and future actions taken to  streamline CIO reporting requirements.
We conducted this performance audit from December 2013 to April 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions base on our audit objectives.
section_title: Appendix II: Chief Information Officer IT Management Reporting Requirements
paragraphs: The following table lists the 36 OMB requirements for agency Chief  Information Officer reporting on IT management, along with the source of  requirement and the year that the requirement was established.
section_title: Appendix III: Survey of Federal Agency Chief Information Officers
paragraphs: The following is an abridged version of the CIO reporting requirements  survey. This version includes the set of questions for one requirement,  which was repeated for each of the requirements in our survey.
section_title: Appendix IV: Comments from the Office of Management and Budget
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, the following staff made key  contributions to this report: Gary Mountjoy (Assistant Director); Scott  Borre; Chris Businsky; Valerie Hopkins; Stuart Kaufman; Lee McCracken;  Tarunkant Mithani; and Teresa Smith.