Data Act: OMB and Treasury Have Issued Additional Guidance and Have Improved Pilot Design but Implementation Challenges Remain
Published: 2016-12-08T00:00:00

Fast Facts:
The DATA Act requires more transparency in federal spending data and increases the types of spending data that agencies make available to the public. We found that agencies face challenges in implementing the common data standards required by the act and in preparing their data for submission to Treasury ahead of the act's May 2017 deadline.
We also looked at a pilot project that is intended to reduce the reporting burden for entities that receive federal funding. We recommended that the Office of Management and Budget clearly document how the results of the procurement portion of the pilot can be applied to other required federal reporting.

Highlights:
section_title: Why GAO Did This Study
paragraphs: Effective implementation of the DATA Act will allow federal funds to be better tracked and greatly increase the types of data made publicly available. OMB and Treasury have taken significant steps to implement the act, but challenges remain as the critical deadline of May 2017 approaches. Consistent with GAO's mandate under the act, this report is one in a series of products GAO will provide to Congress providing oversight of DATA Act implementation. This report examines (1) steps taken to establish a clear data governance structure which is important during the upcoming transition to a new administration, (2) challenges reported by major agencies in their implementation plan updates, (3) the operationalization of government-wide data standards and technical specifications for data reporting, and (4) updated designs for the Section 5 pilot for reducing recipient reporting burden and progress made in its implementation. GAO reviewed key implementation documents, compared the Section 5 Pilot to leading practices, and interviewed staff at OMB, Treasury, and other selected agencies.
section_title: What GAO Found
paragraphs: Data governance and the transition to the new administration. Consistent with a July 2015 GAO recommendation to establish clear policies and processes that follow leading practices for data governance under the Digital Accountability and Transparency Act of 2014 (DATA Act), the Office of Management and Budget (OMB) and the Department of the Treasury (Treasury) have taken the initial step of convening a committee to maintain established standards and identify new standards. Although this represents progress, more needs to be done to establish a data governance structure that is consistent with key practices to ensure the integrity of standards over time. The upcoming transition to a new administration presents risks to implementing the DATA Act, potentially including shifted priorities or lost momentum. The lack of a data governance structure for managing efforts going forward jeopardizes the ability to sustain progress as priorities shift over time.
Implementation plan updates. The 24 Chief Financial Officers Act agencies continue to face challenges implementing the DATA Act, according to information in their implementation plan updates. GAO identified four categories of challenges reported by agencies that may impede their ability to implement the act: systems integration issues, lack of resources, evolving and complex reporting requirements, and inadequate guidance. To address these challenges, agencies reported changing internal policies and procedures; leveraging existing resources; and using external resources, and manual and temporary workarounds, among other actions.
Operationalizing data standards and technical specifications for data reporting. OMB issued additional guidance on how agencies should report data involving specific transactions, such as intragovernmental transfers, and how agencies should provide quality assurances for submitted data. However, this guidance does not provide sufficient detail in areas such as the process for providing assurance on data submissions and it does not address others, such as how agencies should operationalize the definitions for data elements (e.g., primary place of performance and award description). Treasury released a new version of the DATA Act Broker—a system that collects and validates agency data—in October 2016 and is making minor adjustments to its functionality. Agencies have reported making progress creating and testing their data submissions, but some report needing to rely on interim solutions for initial reporting while they wait for automated processes to be developed.
Pilot to reduce recipient reporting burden. GAO reviewed the revised design for both the grants and procurement portions of the pilot and found that they partly met each of the leading practices for effective pilot design. Although this represented significant progress since April 2016, GAO identified an area where further improvement is still needed. Specifically, the plan for the procurement portion of the pilot does not clearly document how findings related to the centralized certified payroll reporting portal will be applicable to other types of required procurement reporting. This is a particular concern given the diversity of federal procurement reporting requirements. To date, all six components of the grant portion are underway. Data collection for the procurement portion is delayed and is not expected to begin until January or February 28, 2017.
section_title: What GAO Recommends
paragraphs: GAO is making one new recommendation: that for the Section 5 Pilot, OMB clearly document in its design of the procurement portion how data collected through the centralized certified payroll reporting portal will be applied to other required procurement reporting. Moving forward, additional progress needs to be made to address GAO's 11 previous DATA Act recommendations that remain open. OMB neither agreed nor disagreed with GAO's recommendation.

section_title: Letter
paragraphs: Full and effective implementation of the Digital Accountability and  Transparency Act of 2014 (DATA Act) offers the promise of a much more  complete and accurate understanding of federal spending by enabling— for the first time— the federal government as a whole to track these funds  at multiple points in the federal spending lifecycle, and significantly  increasing the types and transparency of data available to agencies,  Congress, and the general public. Since the DATA Act became law in  May 2014, the Office of Management and Budget (OMB) and the  Department of the Treasury (Treasury) have taken significant steps to  implement it. However, the transition to a new administration may present  risks tom the implementation of the DATA Act, including potential shifting  of priorities or a loss of momentum.
OMB, Treasury, and federal agencies need to address a range of  evolving and complex policy and technical issues to ensure the DATA Act  is effectively implemented. As we have previously reported, agencies  have identified several areas of concern including inadequate guidance,  tight time frames, competing priorities, a lack of funding, and system  integration issues. Our prior work has identified concerns related to  standardizing data element definitions and developing a technical  schema, concerns that, if not addressed, could lead to agencies  inconsistently and inaccurately reporting data and delaying  implementation. Finally, we also reported that although OMB appears to  be on track with designing its pilot for developing recommendations to  reduce recipient reporting burden, much of the work of actually  implementing those plans remains. Addressing these challenges will  require ongoing and focused commitment to maintain progress  implementing key provisions of the DATA Act.
This review is part of an ongoing effort to provide interim reports on the  progress being made in implementing the DATA Act and meets the  reporting requirements mandated by the act. This report addresses the  following areas: (1) steps taken to establish a clear data governance  structure, which is particularly important for the transition to a new  administration, (2) challenges reported by Chief Financial Officers Act of  1990 (CFO Act) agencies in their implementation plan updates, (3) the  operationalization of government-wide data standards and the technical  specifications for data reporting, and (4) updated designs for the pilot for  reducing recipient reporting burden and progress made in its  implementation.
To describe the extent to which OMB’s and Treasury’s efforts to  implement a data governance structure for the DATA Act were consistent  with key practices, we assessed OMB’s and Treasury’s efforts against a  set of common key practices for establishing effective data governance  structures. We identified this common set of key practices from a range of  organizations, including domestic and international standard-setting  organizations, industry groups or associations, and federal agencies, to  ensure we had a comprehensive understanding of data governance key  practices across several domains. We also met with OMB staff and  Treasury officials to obtain information on the status of their efforts to  address our previous recommendation that they establish a data  governance structure.
To identify implementation challenges reported by agencies, we reviewed  implementation plan updates and supplemental information submitted by  federal agencies and assessed it against new OMB guidance and the  revised Treasury DATA Act Implementation Playbook. We compared the  implementation plan updates we received to the initial implementation  plans submitted by the CFO Act agencies in 2015. We also interviewed  OMB staff and Treasury officials and reviewed documentation of their  processes and controls for reviewing the updated implementation  information and monitoring agencies’ progress. We met with OMB and  Treasury to obtain information on the status of efforts to address our  previous recommendations related to agency implementation plans.
To assess the operationalization of data standards and technical  specifications for reporting, we reviewed applicable technical guidance  and documentation related to DATA Act Information Model Schema  (DAIMS), version 1.0, and the DATA Act Broker. We reviewed various  versions of the broker made available by Treasury through open source  code posted on a public website. In addition, we observed several  demonstrations of how agencies submit their data to a prototype of the  broker. We also interviewed knowledgeable officials from OMB, Treasury,  and selected federal agencies and staff from their offices of inspector  general, as well as enterprise resource planning (ERP) vendors assisting  federal agencies with technical implementation.
To obtain additional information on agencies’ use of the technical  guidance, we selected three agencies—the Department of Health and  Human Services (HHS), the Department of Agriculture (USDA), and the  Corporation for National and Community Service (CNCS). Although the  information obtained from these three agencies is not generalizable to all  agencies, they illustrate a range of conditions under which agencies are  implementing the act. At each agency, we reviewed DATA Act  implementation plan updates and interviewed officials responsible for  implementing the act, including DATA Act implementation team members.  We met with OMB and Treasury to obtain information on the status of  efforts to address our previous recommendations related to providing  policy and technical guidance.
To assess the design of the pilot for reducing recipient reporting burden  (called the Section 5 Pilot), we reviewed Section 5 of the Federal Funding  Accountability and Transparency Act of 2006, as amended by the DATA  Act; assessed OMB and its partners’ draft design documents; and spoke  with cognizant staff implementing these pilots at OMB, HHS, and General  Services Administration (GSA). We met with OMB to obtain information  on the status of efforts to address our previous recommendations related  to designing the pilot for reducing recipient reporting burden. Additional  details regarding our objectives, scope, and methodology are provided in  appendix I.
We conducted this performance audit from May 2016 to December 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Signed into law on May 9, 2014, the DATA Act expanded on previous  federal transparency legislation to link federal agency spending to federal  program activities so that taxpayers and policymakers can more  effectively track federal spending. The DATA Act requires government- wide reporting on a greater variety of federal funds as well as tracking of  these funds at multiple points in the federal spending lifecycle. The act  also calls for the federal government to set government-wide data  standards, identify ways to reduce reporting burdens for grantees and  contractors (Section 5 Pilot), and regularly review data quality to help  improve the transparency and accountability of federal spending data.
OMB and Treasury have taken significant steps toward implementing the  act’s various requirements including standardizing data element  definitions, issuing guidance to help agencies develop their  implementation plans, and designing a pilot for developing  recommendations to reduce recipient reporting burden. We have  previously reported on these efforts and others and have identified a  number of ongoing challenges that will need to be addressed in order to  successfully meet the act’s requirements. Throughout our ongoing  oversight of OMB’s and Treasury’s efforts to implement the act, we have  coordinated closely with OMB and Treasury to provide timely feedback  and have made a number of recommendations that, if addressed, could  help ensure the full and effective implementation of the act. OMB and  Treasury have made progress implementing 5 of our recommendations  related to DATA Act implementation. However, additional effort is needed  to address 11 previous GAO recommendations that remain open. See  appendix II for a list of our previous recommendations relating to the  DATA Act and their implementation status.
section_title: Initial Steps Taken to Develop a Governance Structure but Further Steps Needed to Maintain the Integrity of Data Standards
paragraphs: OMB and Treasury are developing a governance structure, but more work  will be needed to ensure that this structure is consistent with key  practices for developing and maintaining the integrity of data standards  over time. In July 2015, we reported that OMB and Treasury took initial  steps to develop organizational structures for project governance but had  not yet established a formal framework for providing data governance  throughout the lifecycle of developing and implementing standards. Such  a framework is key for ensuring that the integrity of data standards is  maintained over time. Accordingly, we recommended that OMB and  Treasury establish a clear set of policies and procedures for developing  and maintaining data standards that are consistent with leading  practices.
OMB and Treasury generally agreed with our recommendation and, in  response, engaged a contractor to interview key stakeholders and  develop a set of potential next steps. The first of these steps was to  establish a new Data Standards Committee that will be responsible for  maintaining established standards and developing new data elements or  data definitions that could affect more than one functional community  (e.g., financial management, financial assistance, and procurement).  According to OMB staff, the Data Standards Committee held its inaugural  meeting on September 15, 2016, and will meet on a monthly basis. The  committee has also drafted a charter that will delineate the scope of the  committee’s work, as well as the composition and responsibilities of its  members. According to OMB staff, members include representatives from  a range of federal communities including the grants, procurement,  financial management and human resources communities, as well as  representatives of several interagency councils including the Chief  Information Officers Council and the Performance Improvement Council.  OMB staff told us that the committee will focus on clarifying existing data  standard definitions, including the definition of predominant place of  performance, and identifying new standards that may be needed going  forward. In October 2016, according to OMB staff, the charter was under  review by the DATA Act Executive Steering Committee.
Several data governance models exist that could inform OMB’s and  Treasury’s efforts to ensure the integrity of the data standards over time.
These models define data governance as an institutionalized system of  decision rights and accountabilities for planning, overseeing, and  controlling data management. Many of these models promote a common  set of key practices that include establishing clear policies and  procedures for developing, managing, and enforcing data standards. A  common set of key practices endorsed by standards setting organizations  recommend that data governance structures should include the key  practices shown in the text box below. We have shared these key  practices with OMB and Treasury.
Key Practices for Data Governance Structures  i.
Developing and approving data standards. ii.
Managing, controlling, monitoring, and enforcing consistent application of data  standards. iii.
Making decisions about changes to existing data standards and resolving  conflicts related to the application of data standards. iv.
Obtaining input from stakeholders and involving them in key decisions, as  appropriate. v.
Delineating roles and responsibilities for decision-making and accountability,  including roles and responsibilities for stakeholder input on key decisions.
OMB and Treasury have not yet institutionalized and clearly documented  policies and procedures that are consistent with these key practices. For  example, processes have not been developed to both approve new  standards and ensure that already established standards are consistently  applied and enforced across the federal government.
One reason why having a robust, institutionalized data governance  structure is important is to provide consistent data management during  times of change and transition. The transition to a new administration  presents one such situation. We have previously reported that, given the  importance of continuity when implementing complex, government-wide  initiatives, the potential for gaps in leadership that can occur as  administrations change can impact the effectiveness and efficiency of  such efforts, potentially resulting in delays and missed deadlines. Such  transitions may disrupt the momentum for meeting implementation  timeframes or cause the government to fail to continue to build on  previous accomplishments. The absence of a robust and institutionalized  data governance structure presents additional potential risks regarding  the integrity of data standards over time and the ability of agencies to  meet their statutory timelines in the event that priorities shift with the  incoming administration or momentum is lost.
section_title: Agencies’ Implementation Plan Updates Indicate Continuing Challenges to Meeting the DATA Act Reporting Deadline
paragraphs: In June 2016, OMB directed the 24 CFO Act agencies to update their  initial DATA Act implementation plans that they submitted in response to  OMB’s May 2015 request. Each agency was to (1) update its timeline  and milestones and explain the agency’s progress to date and the  remaining actions it would take to implement the act in accordance with  the suggested steps in Treasury’s DATA Act Implementation Playbook  (Version 2.0) (Playbook 2.0), (2) report costs to date and estimated total  future costs, and (3) explain any new challenges and mitigation  strategies.
In reviewing the 24 CFO Act agencies’ implementation plan updates that  we obtained from the agencies, we found the following:
Each of the 24 CFO Act agencies’ updates included timelines and  milestones and most of the updates included most of the OMB  required information. For example, most of the 24 CFO Act agencies  included remaining actions the agencies would take to implement the  suggested steps in Playbook 2.0.
Some of the CFO Act agencies did not include information about  some of the remaining actions to implement the suggested steps in  Playbook 2.0. For example, 5 of the 24 CFO Act agencies did not  include information about testing for completeness and accuracy of  data elements submitted to Treasury, 11 CFO Act agencies did not  include information about workflows for addressing validation errors  and revisions needed to agency data submissions, and 13 CFO Act  agencies did not include information about testing linkages of program  and financial data or possible interim solutions to link such data, if  needed. Without such information in agencies’ updates, it may be  more difficult for OMB and Treasury to determine where to target their  monitoring and assistance efforts to help ensure the DATA Act is  successfully implemented.
Our review of the CFO Act agencies’ August 2016 implementation  plan updates found that 21 of the 24 CFO Act agencies reported costs  to date and future estimated costs to implement the DATA Act  reporting requirements. One agency reported future estimated  costs, but did not report costs to date. Two agencies did not provide  any cost estimates. Total cumulative and future estimated costs for  full DATA Act implementation that was reported by 22 CFO Act  agencies in their implementation plan updates ranged from  approximately $1.0 million to $59.1 million, for a total of about $202.4  million. This total estimated cost reported by CFO Act agencies to  implement the DATA Act includes costs for systems integration and  modernization. It is important to note that the estimated total costs  reported by CFO Act agencies to implement the DATA Act  requirements is relatively small when compared to the almost $81  billion spent on information technology by the CFO Act agencies in  fiscal year 2016 alone.
See appendix III for more details about the information that OMB required  CFO Act agencies to include in their implementation plan updates,  remaining actions to implement the suggested steps in Playbook 2.0, and  the number of CFO Act agencies that included the information.
In our July 2016 report, we reported on challenges agencies included in  their initial implementation plans. The implementation plan updates  indicate that 19 of the 24 CFO Act agencies continue to face challenges  in their efforts to implement the DATA Act. Based on our review of the 24  CFO Act agency implementation plan updates, we identified four  overarching categories of challenges reported by agencies that may  impede their ability to effectively and efficiently implement the DATA Act:  systems integration issues, lack of resources, evolving and complex  reporting requirements, and inadequate guidance. See table 4 in  appendix III, which describes the categories of challenges and the  number of CFO Act agencies reporting challenges in each category.  Some of the challenges reported by the CFO Act agencies in their  updates include the following:  Systems integration. Nineteen of the 24 CFO Act agencies reported  challenges related to systems integration, which include concerns with  systems limitations, modernization efforts, and timing. For example, one  agency reported that validation presents challenges because its financial  systems are not properly integrated with procurement and grant systems.  Similarly, the agency reported that several of its components are  undergoing grant, procurement, or financial system improvements that  coincide with implementing the DATA Act, which could pose a risk to  timely DATA Act implementation if the improvements are delayed.  Another agency reported that, for one of its legacy systems, obtaining the  unique identifier to generate award financial data will likely be a manual  process. The lack of properly integrated systems increases the risk that  agencies may have difficulty compiling and validating the information they  are required to report under the DATA Act by the May 2017 reporting  deadline for agencies to submit their financial and payment information.
Resources. Fourteen of the 24 CFO Act agencies reported challenges  related to staffing issues or funding constraints. For example, one agency  reported that expertise related to feeder systems and data will be needed  to successfully implement the DATA Act, but such subject matter experts  may not be available. Another agency reported that meeting the reporting  deadline is highly dependent on receiving requisite funding and  resources. The lack of sufficient resources, including staff expertise and  proper funding, increases the risk that agencies may have difficulty taking  all the actions needed in a timely manner to fully implement the  requirements of the DATA Act.
Reporting. Thirteen of the 24 CFO Act agencies reported challenges  related to mandatory DATA Act reporting requirements, including  concerns with data quality and their ability to report all the required data  elements in their initial DATA Act submissions, as well as senior  accountable officials (SAO) certification and reporting non-financial  data. For example, one agency reported that its SAO may be unable to  certify the quality of data if OMB’s guidance for the SAO certification  cannot be supported by existing processes. Another agency reported  concerns with the burden of reconciling account data with financial and  award data. In addition, two agencies reported challenges with reporting  beginning balances at the level of detail required by the DATA Act. Two  agencies reported concerns with protecting sensitive and classified data.  One agency also reported ongoing issues with inconsistent quality of data  submitted from their financial systems. Another agency reported that  certain data elements are not currently available for all document types,  and is considering pulling these data elements from other source systems  to the extent possible. A lack of complete and accurate agency data  increases the risk that agencies may not be able to meet the DATA Act  reporting requirements within the mandated timeframes.
Guidance. Eleven of the 24 CFO Act agencies reported ongoing  challenges related to the timely issuance of, and ongoing changes to,  OMB policy and Treasury guidance. Eight agencies reported that if policy  or technical guidance continues to evolve or be delayed, the agencies’  ability to comply with the May 2017 reporting deadline could be affected.  Some agencies also reported concerns about the requirement for SAOs  to certify the data reported quarterly. For example, one agency reported  that if guidance clarifying certification procedures is delayed, it may not  have time to implement appropriate validation steps needed to give  assurance over the data. Because of the lack of timely and consistent  guidance, agencies may need to continuously update or change their  processes, which could adversely affect their ability to meet the DATA Act  requirements.
As noted above, the information reported by the CFO Act agencies in  their implementation plan updates indicates that some agencies are at  increased risk of not meeting the May 2017 reporting deadline because of  these challenges. In addition, inspectors general for some agencies, such  as the Departments of Labor and Housing and Urban Development, have  issued readiness review reports that also indicate their respective  agencies are at risk of not meeting the reporting deadline. As discussed  further below, the technical software requirements for agency reporting  are still evolving, so any changes to the technical requirements over the  next few months could also affect agencies’ abilities to meet the reporting  deadline.
In August 2016, in response to our prior recommendation, OMB  established procedures for reviewing and using agency implementation  plan updates that include procedures for identifying ongoing challenges.  In its procedures document, OMB states that it has received input from a  significant number of agency staff via office hours, emails, regular  meetings, agency visits, and other methods regarding the challenges  agencies are experiencing as they work toward implementation since the  submission of their original plans. OMB’s document also states that it has  worked to address these challenges and provide both policy and technical  guidance as needed. Further, the document stated that requiring  agencies to update plans will allow OMB to address challenges that  agencies are not directly reaching out to OMB about or that numerous  agencies are experiencing. According to the procedures document, OMB  will also be monitoring progress toward the statutory deadline and setting  up meetings with any of the 24 CFO Act agencies that OMB identifies as  being at risk of not meeting the implementation deadline. OMB will  schedule these visits by reviewing the implementation plan updates and  discerning which agencies appear to be experiencing the most challenges  to implementation.
To help address their challenges, 16 of the 24 CFO Act agencies reported  that they use certain mitigating strategies in their implementation plan  updates. Based on our review, we identified seven overarching categories  of mitigating strategies reported by these agencies to address DATA Act  implementation challenges: making changes to internal policies and  procedures, leveraging existing resources, using external resources,  continuing communications, employing manual and temporary  workarounds, monitoring and developing guidance, and enhancing  existing systems. These strategies, as a whole, were similar to the  mitigating strategies reported by agencies in their initial implementation  plans. The most commonly reported categories of mitigating strategies  were changing internal policies and procedures and leveraging existing  resources. See table 5 in appendix III for descriptions of the categories of  mitigating strategies and the number of CFO Act agencies that report  using strategies from each category.
section_title: Progress Made in Data Standards and Technical Framework, but Uncertainties Persist as Implementation Proceeds
subsections: section_title: OMB Released Additional Policy Guidance to Clarify Agency Reporting Requirements
paragraphs: In May 2016, in response to our prior recommendation, OMB released  additional guidance on reporting financial and award information required  under the act to address potential clarity, consistency, and quality issues  with the definitions of standardized data elements. In January 2016 we  reported that ensuring that data definitions are generally consistent with  leading practices is important because limitations with the definitions  could lead to inconsistent or inaccurate reporting, among other issues.  We also reported that although the standardized data element definitions  issued by OMB largely adhered to leading practices for establishing data  definitions, several definitions had limitations that could lead to  inconsistent reporting. While OMB’s additional guidance addresses some  of the limitations we identified, it does not address all the clarity issues we  identified.
Specifically, OMB’s additional guidance addresses (1) reporting financial  and award level data, (2) establishing linkage between agency award and  financial systems using a unique award identifier, and (3) providing  assurances that data submitted to Treasury for publication on  USASpending.gov is sufficiently valid and reliable. For example, OMB’s  Management Procedures Memorandum No. 2016-03 directs agencies to  leverage existing procedures for providing assurances of the quality of  their DATA Act data submissions and directs agency SAOs to provide  reasonable assurance that their internal controls support the reliability  and validity of the data submitted to Treasury for publication on  USASpending.gov. OMB’s memorandum notes that assurance means that, at a minimum, the data reported are based on appropriate internal  control and risk management strategies identified in OMB Circular A- 123. OMB expects SAO assurance of the data through this process  would mean that data submitted to Treasury by May 2017 complies with  existing controls for ensuring the data quality.
However, our prior work has shown that relying on these quality  assurance processes is not sufficient to address the accuracy and  completeness challenges that we have previously identified.  Additionally, as we reported in August 2016, Offices of Inspector General,  which are required to assess the completeness, timeliness, quality, and  accuracy of data submitted under the act, have expressed concerns  about agencies’ abilities to provide assurances of the quality of their  data. The inspectors general are particularly concerned about their  agencies’ ability to provide quality assurances for data that are not  directly provided by the agency, such as data submitted by non-federal  entities who receive federal awards.
To address these concerns, OMB released draft guidance in August 2016  that specifies DATA Act reporting responsibilities when an  intragovernmental transfer (both allocation transfers and buy/sell  transfers) is involved, explains how to report financial assistance awards  with personally identifiable information (PII), and clarifies the SAO  assurance process over the data submitted to the broker. OMB staff told  us that this most recent policy guidance was drafted in response to  questions and concerns reported by agencies in their implementation plan  updates, as well in meetings with senior OMB and Treasury officials  intended to assess agency implementation status. Among other  challenges, agencies indicated the need for additional guidance on  reporting intergovernmental transfers, providing assurances over their  data, and reporting insurance information. For example, officials from  USDA, one of our case example agencies, told us that they are waiting  for guidance on insurance and indemnity reporting, but no guidance has  been issued. Absent any new guidance, they plan to report insurance as  they have under the Federal Funding and Accountability and  Transparency Act of 2006 (FFATA). OMB staff told us that they received  feedback from 30 different agencies and reviewed over 200 comments on  the draft guidance. The final guidance, OMB M-17-04, was issued on  November 4, 2016.
Although OMB has made some progress with these efforts, other data  definitions lack clarity—including primary place of performance and award  description—which still needs to be addressed to ensure agencies report  consistent and comparable data. These challenges, as well as the  challenges identified by agencies, underscore the need for OMB and  Treasury to fully address our prior recommendation to provide agencies  with additional guidance to address potential clarity issues. OMB staff told  us that the newly established Data Standards Committee will be  responsible for developing guidance to provide additional operational  clarity regarding these data definitions; however, they were unable to  provide a specific timeframe for when this would be done.
section_title: Late Release of the Schema Version 1.0 May Pose Risks for Implementation Delays at Some Agencies
paragraphs: Treasury released the schema version 1.0 on April 29, 2016—4 months  later than planned and approximately a year before reporting is required  to begin under the act. The schema version 1.0 is intended to  standardize the way financial assistance awards, contracts, and other  financial data will be collected and reported under the DATA Act.  Treasury expects the guidance provided in the schema version 1.0 will  provide a stable base for agencies to develop the necessary data  submission procedures. We have previously reported that a significant  delay in releasing version 1.0 of the schema would likely have  consequences for timely implementation of the act. Agencies are using  schema version 1.0 to plan what changes are needed to systems and  business processes to be able to capture and submit the required data.  Under the act, agencies must report data in compliance with established  standards by May 2017. Toward that end, OMB and Treasury have  directed agencies to begin submitting data by the beginning of the second  quarter of fiscal year 2017 (January 2017) with the intention of publically  reporting that data by May 2017.
OMB’s summary of agencies’ implementation plan updates  acknowledged that the delay in the release of schema version 1.0  delayed agency timelines for implementation. This document also  recognized that the iterative approach being used to develop and release  guidance has posed challenges for some agencies as changes in the  guidance may require them to re-work some of their implementation  project plans. Our analysis of the implementation plan updates submitted  by the agencies to OMB confirms this. We found that 11 of the 24 CFO  Act agencies highlighted challenges related to the guidance provided by  OMB and Treasury in their implementation plan updates. One of the  commonly cited challenges concerned complications arising due to the  iterative nature or late release of the guidance. For example, one agency  reported that developing its implementation plan was highly dependent  upon the concurrent development of the schema version 1.0 and  technical guidance being developed by Treasury. This agency stated that  any delays or changes to these components will significantly affect its  solution design, development and testing schedule, and cost estimate.
section_title: Treasury Continues to Test and Revise the DATA Act Broker
paragraphs: A key component of the reporting framework laid out in the schema  version 1.0 is the DATA Act Broker, a system to standardize data  formatting and assist reporting agencies in validating their data prior to  submitting it to Treasury. See figure 1 for a depiction of how Treasury  expects the broker to operate.
Treasury’s software development team has been iteratively testing and  developing the broker using what Treasury describes as an agile  development process. Treasury released the first version of the broker  in spring 2016 and it continues to develop the system’s capabilities  through 2-week software development cycles, called sprints. On  September 30, 2016, Treasury released a version of the broker, which it  stated was fully capable of performing the key functions of extracting and  validating agency data. Treasury officials told us that although they plan  to continue to refine the broker to improve its functionality and overall  user experience, they have no plans to alter these key functions.
According to Treasury guidance documents, agencies are expected to  use the broker to upload three files containing data pulled from the  agencies’ internal financial and award management systems. These files  will undergo two types of validation checks in the broker before being  submitted to Treasury: data element validations and complex validations.  Data element validations check whether data elements comply with  specific format requirements such as field type and character length.  Complex validations perform tasks such as checking data against other  sources or using calculation rules to verify whether certain data elements  sum up to each other. Treasury has configured these complex validation  rules so that if a rule is not met, the broker can either produce a warning  message while still accepting the data for submission or produce a fatal  error, which prevents submission of the data altogether. As of September  30, 2016, data uploaded to the broker needs to successfully meet less  than a quarter of these complex validation checks in order to be accepted  for submission to Treasury. Treasury officials said that this choice was  made in order to allow agencies more flexibility to test the broker, and that  the data submissions will be required to pass more of these validation  rules at a later date. According to Treasury documents, in a future  release of the broker, data uploaded to the broker will need to  successfully meet about half of the complex validation checks in order to  be accepted for submission to Treasury. Treasury officials said that  having about half of the validation rules produce warnings rather than  fatal errors would provide agency officials with the flexibility to correct  issues flagged by the broker or not to do so, depending on their  knowledge of the context and situation of specific data elements. For  example, for some of the programs, grants award-level information may  not be reported for security reasons.
In addition to assisting agencies in collecting and validating agency- generated data, the broker also extracts award and sub-award  information from existing government-wide award reporting systems and  helps ensure these files are in the standard format. This function was  added during software development efforts in late September and early  October 2016. Unlike the files submitted by agencies, these extracted  files with award and sub-award information are not subject to any  validations in the broker. However, Treasury implemented additional  validation checks on the file containing agency financial assistance award  information through its source system, the Award Submission Portal.  These checks include verifying that required information is present and  formatted correctly.
Treasury officials told us that the responsibility for ensuring the accuracy  of these files lies with the DATA Act SAO at each agency. For example,  OMB Management Procedures Memorandum 2016-03 specifies that  SAOs must provide reasonable assurance that their internal controls  support the reliability and validity of the agency account-level and award- level data submitted to Treasury for publication. Before final submission  of the data files in the broker, the SAO is responsible for assuring that, at  a minimum, the data reported are based on appropriate internal control  and risk management strategies identified in OMB Circular A-123.  Treasury officials said that if SAOs are not able to provide this assurance,  their agency will be prevented from submitting the files and their data will  not be included in the data reporting based on the current broker design.  Currently, the broker does not allow agencies to submit their data with  qualifications, such as known quality limitations, so data that does not  completely meet the criteria for SAO assurance will not be reported, even  with qualifications. OMB staff and Treasury officials said that they are  reconsidering this position and are exploring ways that agencies can  submit data with qualifications and how these qualifications can be  conveyed to the public.
section_title: Agency Data Submissions are Under Development
paragraphs: Agencies have made progress toward creating their data submissions  and testing them in the broker, but work remains to be done before actual  reporting can begin. Treasury has made empty sample files available to  agencies so they can begin testing their data files in the broker without  having completed building all of them. As of October 2016, 21 of 24 CFO  Act agencies reported that they had begun testing their data files in the  broker, but only the National Aeronautics and Space Administration had  completed testing the broker and revised its data files accordingly.  Treasury also collects data from the four shared service providers that are  helping to manage data submissions for their agency clients. As of  October 2016, two of these shared service providers reported to Treasury  that they had finished building the data files for submission to the broker.
In August 2016, we reported that agencies we reviewed are relying on a  series of software patches from their enterprise resource planning (ERP)  vendors to facilitate their data submissions. ERP vendors are  developing patches that will extract data to help their clients develop files  that comply with DATA Act requirements. According to vendors, these  patches will help link an agency’s financial and award systems, create  additional fields in existing systems to report new data elements, and  extract data files formatted for submission to Treasury. Patches that will  facilitate the generation of agency file submissions were planned to be  completed between August 2016 and February 2017. As of September  2016, the release of one of these patches has been delayed. Oracle, one  of the ERP vendors developing these patches, had planned to release a  patch that would allow award attributes to be captured in their clients’  core purchasing systems and general ledger journals in August 2016, but  the release was delayed until September 13, 2016. Representatives from  SAP, another such ERP vendor, said that they were able to deliver one of  the needed patches to their clients in August 2016 and an additional  patch in October. But, they also said that changes and adjustments to the  broker had delayed their progress towards creating a patch that can  format their clients’ data files for submission.
It will be more difficult for agencies that are relying on ERP vendor  patches to test their data files in the broker until the patches have been  implemented since the patches will enable them to construct and format  their files for submission to the broker. Two agencies reported in their  implementation plan updates that a delay in the release of the patches  could jeopardize complete and timely data submission for May 2017.  Treasury officials told us that agencies should still be able to create and  submit the required files to the broker without these patches. These  officials said that when designing the schema and broker, they chose to  use a simple file format for data submissions so that agencies would be  able to create these files without a specialized software solution. Treasury  officials acknowledged that patches will make the submission process  easier, but also pointed out that not every agency is able to take  advantage of software patches. Some agencies reported in their  implementation plan updates that they developed plans for interim  solutions to construct these files until the patches can be developed,  tested, and configured. However, some of these interim solutions rely on  manual processing, which can be burdensome and increase the risk for  errors. For example, USDA officials said that the effort to create an  interim solution has been very resource intensive. This process involved  surveying USDA’s bureaus to identify how their systems are configured  and using that information to modify the financial system. HHS has also  developed an interim reporting solution that can generate the required  files without depending on a patch. However, HHS officials said this  interim solution is complex and their processes cannot be fully automated  until the Oracle patch is released. Furthermore, since these processes  are not fully automated, they carry a risk of errors being introduced  though human error. Agencies that are developing interim solutions will  only have until May 2017 to test the data before the reporting deadline.  An OMB document commended these agencies for developing robust  contingency plans since this will better position them for timely  implementation, but acknowledged that that long-term reporting solutions  are still needed.
section_title: Updated Design of Section 5 Pilot Meets DATA Act Requirements and Mostly Reflects Leading Practices, but the Procurement Portion Does Not Clearly Document How its Focus on Certified Payroll Reporting Will Apply to Other Reporting Requirements
subsections: section_title: Revised Pilot Design Meets DATA Act Requirements
paragraphs: As required by the DATA Act, OMB is conducting a pilot program, known  as the Section 5 Pilot, aimed at developing recommendations for reducing  reporting burden for grantees and contractors. The Section 5 Pilot has  two primary focus areas—federal grants and federal contracts  (procurements). OMB partnered with HHS to design and implement the  grants portion of the pilot and with the General Services Administration  (GSA) to implement the procurement portion. As the executing agent for  the grants portion of the pilot, HHS developed six “test models” to  evaluate different approaches to potentially reducing grantee reporting  burden. On the procurement portion of the pilot, OMB’s Office of Federal  Procurement Policy (OFPP) worked with GSA’s 18F to develop and test  a proof of concept reporting portal for reports required by the Federal  Acquisition Regulation (FAR) and is piloting it with the centralized  reporting of certified payroll by contractors working construction projects  in the United States. In March 2016, a revised plan describing the  design of the grants portion of the pilot was released, which updated the  November 2015 version we previously reviewed. This was followed, in  July 2016, by a revised version of the design for the procurement portion.  See table 1 for a summary of the test models components that comprise  the grants and procurement portions of the Section 5 Pilot.
We determined that the updated design for both portions of the Section 5  Pilot meets the statutory requirements for the pilot established under the  DATA Act. Specifically, the DATA Act requires that the pilot program  should include the following design features: (1) collect data during a 12- month reporting cycle; (2) include a diverse group of federal award  recipients and, to the extent practicable, recipients that receive federal  awards from multiple programs across multiple agencies; and (3) include  a combination of federal contracts, grants, and subawards with an  aggregate value between $1 billion and $2 billion. Based on our review  of design documents as well as interviews with cognizant agency staff,  there has been substantial improvement in this area since our last review,  when the design lacked specifics in the procurement portion of the pilot,  which made it difficult to determine whether the design of the overall pilot  would meet these requirements.
section_title: Grants and Procurement Portions of the Pilot Partially Reflect All 5 Leading Practices for Pilot Design
paragraphs: Both the grants and procurement portions of the pilot showed substantial  improvements in the extent to which they reflect leading practices for pilot  design (shown in the textbox below). We found that HHS’s March 2016  revised design for the grants portion of the pilot partly reflects all five of  the leading practices for effective pilot design—an improvement from our  prior assessment. For example, in our April 2016 review we found that  the grants design lacked specific details regarding how the data will be  analyzed and how conclusions will be reached about integrating the pilot  activities into overall grant reporting efforts. Based on our feedback, OMB  and HHS developed a plan to analyze survey and other data prior to the  start of data analysis. This plan specifies the types of quantitative and  qualitative data analysis HHS intends to conduct for each test model and  how that assessment links back to the stated hypotheses. HHS also  added a sampling plan and information on participant outreach efforts to  the design of the grants portion of the pilot which helped it to meet the  leading practices for effective pilot design.
Leading Practices for Effective Pilot Design  1.  Establish well-defined, appropriate, clear, and measurable objectives.  2.  Clearly articulate an assessment methodology and data gathering strategy that  addresses all components of the pilot program and includes key features of a sound  plan.
3.
Identify criteria or standards for identifying lessons about the pilot to inform decisions  about scalability and whether, how, and when to integrate pilot activities into overall  efforts.  4.  Develop a detailed data-analysis plan to track the pilot program’s implementation  and performance and evaluate the final results of the project and draw conclusions  on whether, how, and when to integrate pilot activities into overall efforts.  5.  Ensure appropriate two-way stakeholder communication and input at all stages of  the pilot project, including design, implementation, data gathering, and assessment.
OMB’s July 2016 revision of the design of the procurement portion of the  Section 5 Pilot also showed substantial improvements in reflecting the  leading practices for effective pilot design. Compared to the previous  version, dated November 2015, we identified progress in several areas.  For example, the revised procurement design identified hypotheses for  each objective and contained objectives that were linked to metrics that  should facilitate OMB’s ability to collect appropriate evaluation data. The  revised design also provides additional details regarding the procurement  portion’s intended assessment methodology. It specifies that participants  will submit payroll information to the centralized test portal on a weekly  basis and that OMB will use focus groups to collect qualitative data from  agency staff that use these data for contract management and oversight  purposes. Furthermore, the revised design includes a data-analysis plan  that describes how OMB will collect, track, and analyze data produced by  the pilot. Finally, the revised procurement design provides additional  detail about how potential findings could be scalable from the experiences  of the individual pilot participants to the larger population of contractors  required to submit certified payroll reports in compliance with Davis  Bacon requirements. Toward that end, the revised procurement pilot  design contains a sampling plan that provides criteria for selecting a  diverse group of participants.
section_title: Design of the Procurement Portion of the Pilot Does Not Clearly Document How its Focus on Certified Payroll Reporting Will Apply to Other Procurement Reporting Requirements
paragraphs: We found some areas where the revised procurement design does not  fully reflect leading practices for effective pilot design. These largely relate  to how OMB intends to broaden the pilot’s initial focus on centralizing  certified payroll reporting to other types of FAR-required reporting. The  procurement design presents a reasonable set of factors for why OMB  decided to initially select certified payroll reporting for testing the potential  usefulness of a centralized reporting portal to reduce reporting burden.  However, the plan does not take the next step of clearly describing and  documenting how findings related to centralized certified payroll reporting  will be more broadly applicable to the many other types of required  reporting under the FAR beyond citing general concepts such as data  pre-population and system integration. More specifically, the current  design lacks a plan for testing the assumption that the experiences  contractors have with centralized certified payroll reporting will be similar  when they use the system to meet different reporting requirements and  other databases. This is of particular concern given the diversity of  reporting requirements contained in the FAR. In fact, OMB staff have  identified over 100 different types of FAR reporting requirements with  different reporting frequencies, mechanisms, and required information.
OMB staff told us that they expect to test the centralized portal on other  types of FAR-required reporting and the revised design briefly mentions  other FAR requirements such as those for service contracts and  affirmative action plans. However, the revised design does not provide  any details on how this will be done. The absence of an assessment  methodology and an approach to test the scalability of the design when  applied to procurement reporting requirements beyond certified payroll  reporting is inconsistent with leading practices for pilot design and raises  questions about whether the pilot design will meet its stated objective of  reducing procurement reporting burden more broadly.
section_title: Implementation of Grants Portion of the Pilot is Underway, but the Procurement Portion is Not Scheduled to Begin Until Early 2017
paragraphs: HHS has taken a number of steps to begin implementing the design of  the grants portion of the pilot. For example, they are recruiting  participants for all of the test models and have begun administering data  collection instruments for all of the test models. HHS has engaged in a  number of outreach efforts to recruit participants for its test models.  These officials told us that they have attended an estimated 70 events  since 2015 to discuss the grants pilot, during which they provided  information to interested attendees on how to get involved. Additionally,  as of August 2016, HHS officials reported e-mailing almost 8,000  potential participants and plan on emailing additional prospects, if  needed, in order to reach an established minimum number of participants  for each test model.
GSA’s 18F completed a prototype for the procurement portion of the  Section 5 Pilot at the end of May 2016 and presented it to OMB in June  2016. 18F’s role was to explore how an electronic certified payroll  reporting portal could reduce contractor burden for federal Davis-Bacon  contracts. In August 2016, GSA’s Federal Acquisition Service, the  implementation lead for the pilot, awarded NuAxis the contract to build the  prototype from information obtained as a result of the 18F prototype  process. GSA officials told us that starting in September 2016, NuAxis  began developing a web-based reporting interface that will allow users to  centrally enter and submit certified payroll data. They plan to make this  interface compatible with other existing systems, such as the System for  Award Management (SAM) and Wage Determination Online (WDOL) to  access relevant data sources.
In late November 2016, OMB staff and GSA officials informed us that they  decided to delay launching the portal to conduct the procurement portion  of the pilot in order to ensure that security procedures designed to protect  personally identifiable information (PII) were in place. GSA officials told us  that the centralized reporting portal that would be used to collect data on  certified payroll did not receive the required Authority to Operate because  it did not include necessary security measures to protect the PII that  would be submitted by contractors participating in the pilot. Before the  portal can be used to collect PII, GSA officials said they needed to issue a  System of Records Notice and redesign the certified payroll reporting  platform so that it conforms to agency security procedures. As a result  of these additional steps, GSA officials expect to be able to begin  collecting data through the centralized reporting portal sometime between  late January 2017 and late February 2017. OMB staff said that despite  the security-related delay, they still plan on collecting 12 months of data  through the procurement pilot as required by the act.
In order to meet the act’s requirement that OMB deliver a report to  Congress on ways to reduce recipient reporting burden by August 2017,  OMB staff told us that they plan to only include data collected up to June  or July 2017 in order to allow for sufficient time to analyze the results and  incorporate them into the report’s findings. However, these staff said that  that they plan to continue to collect data through the procurement portion  of the pilot until they obtain a full 12 months of contractors’ experiences  with centralized payroll reporting. Afterwards, OMB plans to analyze this  data, compare it to the smaller data set produced for the August 2017  report to Congress and, if necessary, make any needed revisions to the  findings and recommendations contained in the report previously  submitted to Congress.
section_title: Conclusions
paragraphs: Across the federal government, agencies have efforts under way to  implement the DATA Act by the May 2017 deadline and the success of  these efforts will depend on, among other things, OMB and Treasury’s  efforts to address agency-reported challenges and build an infrastructure  to effectively support government-wide implementation. OMB and  Treasury have made progress but still need to fully address the  recommendations we have made in our previous reports. For example,  OMB and Treasury can build upon the initial step of establishing a data  standards committee responsible for maintaining already established  standards and identifying new standards towards the goal of establishing  an institutionalized system of data management that follows key practices  and ensures the integrity of the data standards over time. In this context,  implementing our prior recommendations will be critical to OMB’s and  Treasury’s progress.
Among the areas where progress has been made in setting a foundation  for successfully implementing the act is the Section 5 Pilot to reduce  reporting burden. In particular, the design of the procurement portion of  the pilot has improved substantially, including the extent to which it  reflects leading practices of pilot design. However, despite advances in  several areas, the current design remains limited by its lack of specifics  regarding how a pilot focused on assessing contractors’ experiences with  a centralized portal designed for certified payroll reporting will be  applicable to many other federal procurement reporting requirements. By  addressing issues such as this and continuing to focus on implementing  the act, the administration greatly increases the likelihood of creating a  system that will achieve the goals of the act—to increase the  transparency of financial information and improve the usefulness of that  data to Congress, federal managers, and the American people.
section_title: Recommendation for Executive Action
paragraphs: In order to ensure that the procurement portion of the Section 5 Pilot  better reflects leading practices for effective pilot design, we recommend  that the Director of OMB clearly document in the pilot’s design how data  collected through the centralized certified payroll reporting portal will be  used to test hypotheses related to reducing reporting burden involving  other procurement reporting requirements. This should include  documenting the extent to which recommendations based on data  collected for certified payroll reporting would be scalable to other FAR- required reporting and providing additional details about the methodology  that would be used to assess this expanded capability in the future.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Secretaries of Agriculture, Health  and Human Services, and the Treasury; the Director of OMB; the Chief  Executive Officer of the Corporation of National and Community Service;  and the Administrator of the General Services Administration for review  and comment. OMB, Treasury, CNCS, HHS, and GSA provided us with  technical comments which we incorporated as appropriate. USDA had no  comments. OMB and Treasury also provided written comments which are  summarized below and reproduced in appendices IV and V, respectively.
In written comments submitted to us, OMB provided an overview of their  implementation efforts since the passage of the DATA Act. These efforts  include issuing three memoranda providing implementation guidance to  federal agencies; finalizing 57 data standards for use on  USASpending.gov; establishing the Data Standards Committee to  develop and maintain standards for federal spending; and developing and  executing the Section 5 pilot. The OMB response also noted that OMB  and Treasury met with each of the 24 CFO Act agencies to discuss their  implementation timelines, risks, and mitigation strategies, and took steps  to address issues that could affect successful implementation. Through  these meetings, OMB staff learned that 19 of the 24 CFO Act agencies  expect that they will fully meet the May 2017 deadline for DATA Act  implementation. OMB neither agreed nor disagreed with GAO’s  recommendation.
In their written response, Treasury provided an overview of the steps they  have taken to implement the DATA Act’s requirements and assist  agencies in meeting their requirements under the act including OMB’s  and Treasury’s issuance of uniform data standards, Treasury’s DATA Act  Implementation Playbook, version 2.0, and the DATA Act Information  Model Schema version 1.0. The Treasury response also noted that as a  result of the aggressive implementation timelines specified in the act and  the complexity associated with linking hundreds of disconnected data  elements across the federal government, they made the decision to use  an iterative approach to provide incremental technical guidance to  agencies. According to Treasury, among other things, this iterative  approach enabled agencies and other key stakeholders to provide  feedback and contribute to improving the technical guidance and the  public website.
We are sending copies of this report to the Secretaries of Agriculture,  Health and Human Services, and the Treasury; the Director of OMB; the  Chief Executive Officer of the Corporation of National and Community  Service; the Administrator of the General Services Administration; as well  as interested congressional committees and other interested parties.
This report will be available at no charge on our website at  http://www.gao.gov.If you or your staff has any questions about this  report, please contact J. Christopher Mihm at (202) 512-6806 or  Mihmj@gao.gov or Paula M. Rascona at (202) 512-9816 or   Rasconap@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of our report.  Key contributors to this report are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This review is part of an ongoing effort to provide interim reports on the  progress being made in implementing the Digital Accountability and  Transparency Act of 2014 (DATA Act), while also meeting the reporting  requirements for us mandated by the act. This report examines: (1) steps  taken to establish a clear data governance structure which is particularly  important during the upcoming transition to a new administration; (2)  challenges reported by Chief Financial Officers Act of 1990 (CFO Act)  agencies in their implementation plan updates; (3) the operationalization  of government-wide data standards and the technical specifications for  data reporting; and (4) updated designs for the Section 5 Pilot for  reducing recipient reporting burden and progress made in its  implementation.
To describe the Office of Management and Budget’s (OMB) and the  Department of the Treasury’s (Treasury) efforts to implement a data  governance structure for the DATA Act, we identified common key  practices for establishing effective data governance structures. To identify  key practices for data governance we reviewed our past reports to identify  applicable laws, regulations, and guidance, as well as reports from other  entities that could inform our work. To select the sources we used to  identify key practices for establishing an effective data governance  program, we identified organizations that had data governance expertise,  had previously published work on data governance, were frequently cited  as a primary source, or some combination of these qualifications. In  addition, because the DATA Act requires that established data standards  incorporate widely accepted common data elements such as those  developed by international voluntary consensus standards bodies, federal  agencies with authority over contracting and financial assistance, and  accounting standards organizations, we selected a range of  organizations, including domestic and international standards-setting  organizations, industry groups or associations, and federal agencies, to  ensure we had a comprehensive understanding of data governance key  practices across several domains.
All of the organizations we identified endorse establishing and using a  governance structure to oversee how data standards, digital content, and  other data assets are developed, managed and implemented. Based on  these selection factors we drew on work from the following organizations  to help us identify data governance key practices: American Institute of  Certified Public Accountants, American National Standards Institute,  Carnegie-Mellon University-Software Engineering Institute, Data  Governance Institute, Data Management Association International,  Oracle, National Association of State Chief Information Officers, National  Institute of Standards and Technology, Digital Services Advisory Group  and the Department of Education-Privacy Technical Assistance Center.  We also met with OMB and Treasury to obtain information on the status  of their efforts to address our previous recommendation that they  establish a data governance structure.
To determine the implementation challenges reported by CFO Act  agencies in their DATA Act implementation plan updates, we requested  and received the updates from the 24 CFO Act agencies. We reviewed  these implementation plan updates and assessed the information against  OMB’s requirements and the revised guidance in Treasury’s DATA Act  Implementation Playbook (Version 2.0) (Playbook 2.0) to determine  whether the updates contained the information required by OMB—(1) an  updated timeline and milestones with an explanation of the agency’s  progress to date and the remaining actions it would take to implement the  act in accordance with the suggested steps in Playbook 2.0, (2) costs to  date and estimated total future costs, and (3) an explanation of any new  challenges and mitigation strategies. We analyzed the agency-reported  challenges and mitigating strategies and categorized them. We compared  the categories of challenges reported by the CFO Act agencies in their  implementation plan updates to the challenges that had been reported in  their initial implementation plans in 2015 to identify any new categories of  challenges. We interviewed cognizant OMB and Treasury officials and  obtained any supporting documentation to further understand the  implementation challenges reported by agencies in their implementation  plan updates and OMB and Treasury’s processes and controls for  reviewing the updated implementation information and monitoring  agencies’ progress. We also met with OMB and Treasury to obtain  information on the status of efforts to address our previous  recommendations related to agency implementation plans.
To assess efforts to date to operationalize government-wide standards  we reviewed OMB policy guidance intended to facilitate agency reporting  as well as guidance intended to respond to agency requests that OMB  clarify how to report specific transactions. We also interviewed OMB staff  and Treasury officials to obtain information about plans for additional  guidance as well as to assess the extent to which issued guidance is  responsive to agency questions, requests for additional clarity on their  reporting requirements, or both. We met with OMB and Treasury to obtain  information on the status of efforts to address our previous  recommendation related to the provision of policy guidance.
To examine the technical structure and specifications for reporting, we  assessed Treasury’s processes for developing technical guidance and  reviewed applicable technical documentation related to the schema  version, 1.0 and the broker. We reviewed the broker made available by  Treasury through open source code posted on a public website (GitHub  repositories associated with the DATA Act) in order to understand its  functionality and validations. In addition, we observed several  demonstrations of how agencies submit their data to a prototype of the  broker and the feedback produced by the system regarding data  verification. We also interviewed knowledgeable officials from OMB,  Treasury, and selected federal agencies and inspectors general, as well  as enterprise resource planning (ERP) vendors assisting federal agencies  with technical implementation.
To obtain specific information on how agencies use the technical  guidance, we selected three agencies based on whether they were in  compliance with existing federal requirements for federal financial  management systems, the type of federal funding provided (such as  grants, loans, or procurements), and their status as a federal shared  service provider for financial management. Based on these selection  factors, we chose the Department of Health and Human Services (HHS),  the Department of Agriculture (USDA), and the Corporation for National  and Community Service (CNCS). Although the information obtained from  these three agencies is not generalizable to all agencies, they illustrate a  range of conditions under which agencies are implementing the act.  These are the same three agencies we selected for our January 2016  and August 2016 reports. This allowed us to assess progress in DATA  Act implementation at these agencies since our last review. At each  agency, we reviewed DATA Act implementation plan updates and  interviewed officials responsible for implementation and DATA Act  implementation team members. We met with OMB and Treasury to obtain  information on the status of efforts to address our recommendation  related to providing technical guidance.
To assess whether the Section 5 pilot designs meet statutory design  requirements, we reviewed Section 5 of the Federal Funding  Accountability and Transparency Act of 2006, as amended by the DATA  Act, to understand the deadlines and design requirements. We reviewed  the draft design documents to assess OMB and its partners’ plans for  meeting these requirements. To supplement our review of those plans,  we also spoke with cognizant staff implementing these pilots at OMB,  HHS, and General Services Administration.
To assess the extent to which the Section 5 pilot designs adhered to  leading practices for effective pilot design, we reviewed the documented  designs for both the grants and procurement portions of the pilot. To  evaluate the grants portion of the pilot, we reviewed the draft design  document from March 2016 as well as data collection instruments such  as surveys and quizzes. We supplemented our assessment with  information HHS officials provided to us during subsequent interviews, as  appropriate. For the procurement portion, we reviewed the draft design  document from July 2016. Additionally, we supplemented our assessment  with information officials from OMB’s Office of Federal Procurement  Policy (OFPP) provided to us during subsequent interviews, as  appropriate. To assess the grants and procurement portions of the pilot,  we applied the five leading practices for effective pilot design we identified  to both portion’s design documents. Each of these analyst assessments  were subsequently verified by a second analyst. We determined that the  design met the criteria when we saw evidence that all aspects of a  leading practice were met. When we were unable to assess whether all  aspects of a leading practice were met, we determined that the design  partially met the criteria. Finally, when we saw no evidence of a leading  practice, or if we identified a critical gap or shortcoming related to the  practice, we determined that the criteria were not met. In continuation of  our constructive engagement approach for working with agencies  implementing the DATA Act, we provided HHS and OMB with feedback  on the design of the grants and procurement portions of the pilot during  our review. These officials generally accepted our feedback as useful  and, in some instances, noted that they have or planned to make  changes to their design as a result of our input. We also met with OMB to  obtain information on the status of efforts to address our recommendation  related to the design of the pilot for reducing recipient reporting burden.
We conducted the work upon which this report is based from May 2016 to  December 2016 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
section_title: Appendix II: Status of GAO’s Recommendations Related to the DATA Act
section_title: Appendix III: Information Included in Chief Financial Officers Act Agencies’ Implementation Plan Updates
paragraphs: In May 2015, the Office of Management and Budget (OMB) directed  federal agencies to submit Digital Accountability and Transparency Act of  2014 (DATA Act) implementation plans to OMB concurrent with the  agencies’ fiscal year 2017 budget requests. In June 2015, the  Department of the Treasury (Treasury) issued guidance—DATA Act  Implementation Playbook (Version 1.0)—to help agencies prepare their  implementation plans. We reviewed these implementation plans and on  July 29, 2016, we issued a report on the results of our review.
In June 2016, OMB directed Chief Financial Officers Act of 1990 (CFO  Act) agencies to submit updates to their initial DATA Act implementation  plans by August 12, 2016. The updates were to (1) update timeline and  milestones and explain the agency’s progress to date and the remaining  actions it would take to implement the act in accordance with the  suggested steps in the DATA Act Implementation Playbook (Version 2.0)  (Playbook 2.0), (2) report costs to date and estimated total future costs,  and (3) explain any new challenges and mitigation strategies.
Treasury’s DATA Act Implementation Playbook (Version 1.0) contained  eight suggested steps and a timeline for agencies to use as they began to  develop their DATA Act implementation plans. Steps 1 through 4 were to  be completed by September 2015. However, as of October 2016, only 16  of the 24 CFO Act agencies reported that they had completed all steps 1  through 4. For example, there were four agencies reporting that they had  not completed their inventory of data and identified the gaps in systems  and processes for data elements (step 3).
DATA Act Implementation Playbook (Version 1.0) indicated that agencies  would be working on steps 5 through 8 throughout fiscal years 2016 and  2017. Playbook 2.0—issued June 24, 2016—includes, among other things, expanded guidance on actions to be included in steps 5 through 8.  Playbook 2.0 did not include expected timeframes for agencies to  complete each step, rather, it referred agencies to Treasury’s  implementation roadmap, which includes high level milestones for  Treasury’s technical deliverables. Playbook 2.0 states that agencies can  use the milestones in Treasury’s implementation roadmap to help  determine their own implementation milestones. Descriptions of steps 5  through 8 from Playbook 2.0 follow:
Step 5: Prepare Data for Submission to the Broker. This step  involves reviewing the schema version 1.0, extracting data from  source systems, mapping agency data to the schema version 1.0, and  implementing system changes as needed to collect and link data.
Step 6: Test Broker Outputs and Ensure Data are Valid. Agencies  may use the broker to verify the data files the agency plans to submit  to Treasury. The broker uses validation rules to test the completeness  and accuracy of the data elements and linkages between financial  and award data. The broker also tests whether the data passes basic  validations within the schema version 1.0.
Step 7: Update Data. This step involves updating information and  systems. If data does not pass validation (see Step 6), the broker will  provide error details to the agency. The agency should then reference  the authoritative data sources and address the discrepancies.
Step 8: Submit Data. Once the data is linked, validated, and  standardized, agencies are to submit the data to Treasury for posting  on USASpending.gov or a successor system. Agency senior  accountable officials (SAO) are to provide reasonable assurance that  their internal controls support the reliability and validity of the agency  account-level and award-level data they submit to Treasury. This  assurance is to be provided quarterly with data submissions beginning  with fiscal year 2017 second quarter data. The SAO assurance  means, at a minimum, that data reported are based on appropriate  internal controls and risk management strategies identified in OMB  Circular A-123.
Table 3 shows the information that OMB required CFO Act agencies to  include in their implementation plan updates, information on remaining  actions the agencies should take to implement suggested steps 5 through  8 in Playbook 2.0, and the number of CFO Act agencies that included the  information.
Table 4 describes the categories of challenges reported by 19 of the 24  CFO Act agencies in their implementation plan updates and the number  of agencies reporting challenges in each category. Five CFO Act  agencies did not identify any challenges in their implementation plan  updates.
Table 5 describes the mitigating strategies reported by 16 of the 24 CFO  Act agencies in their implementation plan updates and the number of  agencies reporting mitigating strategies in each category.
section_title: Appendix IV: Comments from Office of Management and Budget
section_title: Appendix V: Comments from the Department of the Treasury
section_title: Appendix VI: Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to the above contacts, Peter Del Toro (Assistant Director),  Michael LaForge (Assistant Director), Kathleen Drennan (analyst-in- charge), Diane Morris (analyst-in-charge), Michelle Sager, Shirley Hwang,  Aaron Colsher, Katherine Morris, Sophia Tan, Thomas Hackney, Charles  Jones, Laura Pacheco, Maria Belaval, Carrol Warfield, Jr., Mark Canter,  James Sweetman, Jr., Andrew J. Stephens, Carl Ramirez and Jenny  Chanley made major contributions to this report. Additional members of  GAO’s DATA Act Internal Working Group also contributed to the  development of this report.