Data Act: Reported Quality of Agencies' Spending Data Reviewed by OIGs Varied Because of Government-wide and Agency Issues
Released: 2018-07-23T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: The DATA Act was enacted to increase accountability and transparency and, among other things, expanded on the required federal spending information that agencies are to submit to Treasury for posting to a publicly available website. The act also includes provisions requiring a series of oversight reports by agencies' OIGs and GAO.

The objectives of this report are to describe (1) the reported scope of work covered and type of audit standards OIGs used in their reviews of agencies' DATA Act spending data; (2) any variations in the reported implementation and use of data standards and quality of agencies' data, and any common issues and recommendations reported by the OIGs; and (3) the actions, if any, OMB and Treasury have reported taking or planning to take to use the results of OIG reviews to help monitor agencies' implementation of the act.

To address these objectives, GAO reviewed 53 OIG reports issued on or before January 31, 2018, that assessed agencies' first submissions of spending data for the second quarter of fiscal year 2017 and surveyed the OIGs to obtain additional information.

section_title: What GAO Found
paragraphs: The Digital Accountability and Transparency Act of 2014 (DATA Act) requires agencies' Offices of Inspector General (OIG) to issue reports on their assessments of the quality of the agencies' spending data submissions and compliance with the DATA Act. The scope of all OIG reviews covered their agencies' second quarter fiscal year 2017 submissions. The files the OIGs used to select and review sample transactions varied based on data availability, and OIGs performed different types of reviews under generally accepted government auditing standards. Some OIGs reported testing a statistical sample of transactions that their agencies submitted and other OIGs reported testing the full population of submitted transactions. Because of these variations, the overall error rates reported by the OIGs are not fully comparable and a government-wide error rate cannot be projected.

According to the OIG reports, about half of the agencies met Office of Management and Budget (OMB) and Department of the Treasury (Treasury) requirements for the implementation and use of data standards. The OIGs also reported that most agencies' first data submissions were not complete, timely, accurate, or of quality.

OIG survey responses show that OIGs generally reported higher (projected) overall error rates for the accuracy of data than for completeness and timeliness. OIGs reported certain errors that involve inconsistencies in how the Treasury broker (system that collects and validates agency-submitted data) extracted data from certain federal award systems that resulted in government-wide issues outside the agencies' control, while other errors may have been caused by agency-specific control deficiencies. For example, OIGs reported deficiencies related to agencies' lack of effective procedures or controls and systems issues. Most OIGs made recommendations to agencies to address identified concerns.

OMB staff and Treasury officials told GAO that they reviewed the OIG reports to better understand issues identified by the OIGs. OMB issued new guidance in June 2018 requiring agencies to develop data quality plans intended to achieve the objectives of the DATA Act. Treasury officials told GAO that they are collaborating with OMB and the Chief Financial Officers Council DATA Act Audit Collaboration working group to identify and resolve government-wide issues.

section_title: What GAO Recommends
paragraphs: GAO is not making recommendations in this report. The Council of the Inspectors General on Integrity and Efficiency (CIGIE) noted that GAO's report provides useful information on OIG efforts to meet oversight and reporting responsibilities under the DATA Act. OMB, Treasury, and CIGIE also provided technical comments that GAO incorporated as appropriate.

section_title: Letter
paragraphs: The Digital Accountability and Transparency Act of 2014 (DATA Act) was  enacted, in part, to increase accountability and transparency of federal  spending, which totaled almost $4 trillion for fiscal year 2017. Among  other things, the DATA Act includes provisions requiring a series of Office  of Inspector General (OIG) and GAO oversight reports evaluating the  completeness, timeliness, quality, and accuracy of federal agencies’  spending data and the implementation and use of data standards. The  act also requires the Office of Management and Budget (OMB) and  Department of the Treasury (Treasury) to establish data standards to  generate uniform agency data that are consistent and comparable.

In accordance with the DATA Act and OMB and Treasury guidance,  federal agencies submitted their first round of spending data in May 2017  for the second quarter of fiscal year 2017, and the OIGs issued their first  mandated data quality oversight reports beginning in October 2017. This  report is part of our ongoing monitoring of DATA Act implementation in  response to provisions in the DATA Act that call for us to review OIG  reports and issue reports assessing and comparing the quality of agency  data submitted under the act and agencies’ implementation and use of  data standards.

The objectives of this report are to describe (1) the reported scope of  work covered and type of audit standards OIGs used in their reviews of  agencies’ DATA Act spending data; (2) any variations in the reported  implementation and use of data standards and quality of agencies’ data,  and any common issues and recommendations reported by the OIGs;  and (3) the actions, if any, that OMB and Treasury have reported taking  or planning to take to use the results of OIG DATA Act reviews to help  monitor agencies’ implementation of the act.

To address these objectives, we obtained and reviewed 53 OIG DATA  Act reports that were issued on or before January 31, 2018, from 24  Chief Financial Officers Act of 1990 (CFO Act) agency OIGs and 29 non- CFO Act agency OIGs. We identified the OIGs’ reported (1) scope of  work and type of audit standards OIGs used in their reviews,   (2) conclusions about agencies’ implementation and use of the data  standards and the quality of the agencies’ data, (3) government-wide and  agency-specific issues identified, and (4) recommendations to address  identified deficiencies. We also surveyed OIGs to obtain additional  information regarding error rates, sample sizes, control deficiencies  identified, and other items associated with the reviews they conducted.  We received and reviewed responses from the 53 OIGs that we obtained  reports from and followed up with OIGs for clarification and corroboration,  as necessary. Finally, we interviewed OMB staff and Treasury officials to  determine how they used or plan to use the results of the OIG reviews in  monitoring agencies’ implementation of the DATA Act. Appendix I  provides additional details on our scope and methodology.

We conducted this performance audit from September 2017 to July 2018  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The DATA Act was enacted May 9, 2014, for purposes that include  expanding on previous federal transparency legislation by requiring the  disclosure of federal agency expenditures and linking agency spending  information to federal program activities, so that both policymakers and  the public can more effectively track federal spending. The act also calls  for improving the quality of data submitted to USAspending.gov by  holding federal agencies accountable for the completeness and accuracy  of the data submitted. The Federal Funding Accountability and  Transparency Act of 2006 (FFATA), as amended by the DATA Act,  identifies OMB and Treasury as the two agencies responsible for leading  government-wide implementation. For example, the DATA Act requires  OMB and Treasury to establish government-wide financial data standards  that shall, to the extent reasonable and practicable, provide consistent,  reliable, and searchable spending data for any federal funds made  available to or expended by federal agencies. These standards specify  the data elements to be reported under the DATA Act and define and  describe what is to be included in each data element, with the aim of  ensuring that information will be consistent and comparable. The DATA  Act also requires OMB and Treasury to ensure that the standards are  applied to the data made available on USAspending.gov.
subsections: section_title: Sources of Data on USAspending.gov
paragraphs: USAspending.gov has many sources of data. For example, agencies  submit data from their financial management systems, and other data are  extracted from government-wide federal financial award reporting  systems populated by federal agencies and external award recipients. A  key component of the reporting framework is Treasury’s DATA Act broker  (broker)—a system that collects and validates agency-submitted data to  create linkages between the financial and award data prior to their  publication on the USAspending.gov website.

According to Treasury guidance documents, agencies are expected to  submit three data files with specific details and data elements to the  broker from their financial management systems.

File A: Appropriations account. This includes summary information  such as the fiscal year cumulative federal appropriations account  balances and includes data elements such as the agency identifier,  main account code, budget authority appropriated amount, gross  outlay amount, and unobligated balance.

File B: Object class and program activity. This includes summary data  such as the names of specific activities or projects as listed in the  program and financing schedules of the annual budget of the U.S.  government.

File C: Award financial. This includes award transaction data such as  the obligation amounts for each federal financial award made or  modified during the reporting quarter (e.g., January 1, 2017, through  March 31, 2017).

The broker also extracts spending information from government-wide  award reporting systems that supply award data (e.g., federal grants,  loans, and contracts) to USAspending.gov. These systems—including the  Federal Procurement Data System-Next Generation (FPDS-NG), System  for Award Management (SAM), Financial Assistance Broker Submission  (FABS), and the FFATA Subaward Reporting System (FSRS)—compile  information that agencies and external federal award recipients submit to  report, among other things, procurement and financial assistance award  information required under FFATA. The four files produced with  information extracted by the broker from the four systems are as follows:

File D1: Procurement. This includes award and awardee attribute  information (extracted from FPDS-NG) on procurement (contract)  awards and contains elements such as the total dollars obligated,  current total value of award, potential total value of award, period of  performance start date, and other information to identify the  procurement award.

File D2: Financial assistance. This includes award and awardee  attribute information (extracted from FABS) on financial assistance  awards and contains elements such as the federal award identification  number, the total funding amount, the amount of principal to be repaid  for the direct loan or loan guarantee, the funding agency name, and  other information to identify the financial assistance award.

File E: Additional awardee attributes. This includes additional  information (extracted from SAM) on the award recipients and  contains elements such as the awardee or recipient unique identifier;  the awardee or recipient legal entity name; and information on the  award recipient’s five most highly compensated officers, managing  partners, or other employees in management positions.

File F: Subaward attributes. This includes information (extracted from  FSRS) on awards made to subrecipients under a prime award and  contains elements such as the subaward number, the subcontract  award amount, total funding amount, the award description, and other  information to facilitate the tracking of subawards.

The key components of the broker and how the broker operated when the  agencies submitted their data for the second quarter fiscal year 2017 are  shown in figure 1.

After agencies submit the three files to the DATA Act broker, it runs a  series of validations and produces warnings and error reports for  agencies to review. After passing validations for these three files, the  agencies are to generate Files D1 and D2, the files containing details on  procurement and assistance awards. Before the data are displayed on  USAspending.gov, agency senior accountable officials are required to  certify the data submissions in accordance with OMB guidance.  Certification is intended to assure alignment among Files A, B, C, D1, D2,  E, and F, and to provide assurance that the data are valid and reliable.  According to Treasury officials, once the certification is submitted a  sequence of computer program instructions or scripts are issued to  transfer and map the data from broker data tables to tables set up in a  database used as a source for the information on the website. Certified  data are then displayed on USAspending.gov along with certain historical  information from other sources, including Monthly Treasury Statements.

section_title: OIG Methodology and Reporting Guidance for Assessing Agencies’ DATA Act Submissions
paragraphs: The DATA Act requires each OIG to issue three reports on its  assessment of the quality of the agency’s data submission and  compliance with the DATA Act. The first report was due November 8,  2016; however, agencies were not required to submit spending data in  compliance with the DATA Act until May 2017. Therefore, the Council of  the Inspectors General on Integrity and Efficiency (CIGIE) developed an  approach to address what it described as a reporting date anomaly;  encouraged interim OIG readiness reviews and related reports on  agencies’ implementation efforts; and delayed issuance of the mandated  reports to November 2017, with subsequent reports following a 2-year  cycle and due November 2019 and 2021.

CIGIE established the Federal Audit Executive Council (FAEC) to discuss  and coordinate issues affecting the federal audit community, with special  emphasis on audit policy and operations of common interest to FAEC  members. FAEC formed the FAEC DATA Act Working Group to assist the  OIG community in understanding and meeting its DATA Act oversight  requirements by (1) serving as a working-level liaison with Treasury,   (2) consulting with GAO, (3) developing a common approach and  methodology for conducting the readiness reviews and mandated  reviews, and (4) coordinating key communications with other  stakeholders. To assist the OIG community, the FAEC DATA Act Working  Group developed a common methodology and published the Inspectors  General Guide to Compliance Under the DATA Act (IG Guide) for use in  conducting mandated reviews.

The IG Guide includes procedures to test data in agencies’ Files A and B  by reconciling these data to the information that agencies report in their  quarterly SF 133, Report on Budget Execution and Budgetary  Resources. The IG Guide also instructs OIGs to select a statistically  valid sample of spending data from the agencies’ available award-level  transactions in File C, and among other procedures, to confirm whether  these data are also included in the agencies’ Files D1 and D2. The OIGs  are also to confirm whether the transactions in the sample were linked to  the award and awardee attributes in Files E and F. The data in Files E  and F are reported by award recipients in two external government-wide  systems, and are outside the direct control of the federal agencies, except  for the General Services Administration, which manages these external  systems. Based on additional guidance from the FAEC DATA Act  Working Group, OIGs are not required to assess the quality of the award  recipient-entered data that the broker extracted from the two external  government-wide systems used to create Files E and F.

According to the IG Guide, the sampled spending data and testing results  are to be evaluated using the following definitions for the requirements  being assessed:

Completeness is measured in two ways: (1) all transactions that  should have been recorded are recorded in the proper reporting  period, and (2) as the percentage of transactions containing all  applicable data elements required by the DATA Act.

Timeliness is measured as the percentage of transactions reported  within 30 days of the end of the quarter.

Accuracy is measured as the percentage of transactions that are  complete and agree with the systems of record or other authoritative  sources.

Quality is defined in OMB guidance as a combination of utility,  objectivity, and integrity. Utility refers to the usefulness of the  information to the intended users. Objectivity refers to whether the  disseminated information is being presented in an accurate, clear,  complete, and unbiased manner. Integrity refers to the protection of  information from unauthorized access or revision.

The IG Guide also states that OIGs should assess agencies’  implementation and use of the data standards, including evaluating each  agency’s process for reviewing the 57 required data elements and  associated definitions that OMB and Treasury established and  documenting any variances.

section_title: Prior GAO Reports Related to the DATA Act and Data Quality
paragraphs: In November 2017, we issued our first report on data quality as required  by the DATA Act, which identified issues with the completeness and  accuracy of the data that agencies submitted for the second quarter of  fiscal year 2017, use of data elements, and presentation of the data on  Beta.USAspending.gov. Among other things, we recommended that  Treasury disclose known data quality issues and limitations on the new  USAspending.gov website. Treasury agreed with that recommendation  and stated that it would develop a plan to better disclose known data  quality issues. Since the DATA Act’s enactment in 2014, we have issued  a series of interim reports on our ongoing monitoring of the  implementation of the DATA Act and made recommendations intended to  help ensure effective government-wide implementation. However, many  of those recommendations still remain open.

These reports identified a number of challenges related to OMB’s and  Treasury’s efforts to facilitate agency reporting of federal spending, as  well as internal control weaknesses and challenges related to agency  financial management systems that we and agency auditors reported that  present risks to agencies’ ability to submit quality data as required under  the act. For example, our prior work has identified issues with agency  source systems that could affect the quality of spending data made  available to the public. In April 2017, we reported a number of  weaknesses and issues previously identified by agencies’ auditors and  OIGs that affect agencies’ financial reporting and may affect the quality of  the information reported under the DATA Act. We also reported on  findings and recommendations from prior reports with issues on the four  key award systems—FPDS-NG, SAM, the Award Submission Portal  (ASP), and FSRS—which increase the risk that the data submitted to  USAspending.gov may not be complete, accurate, and timely.

section_title: OIG Reviews of Agencies’ DATA Act Submissions Varied in Scope and Type of Standards Used
paragraphs: Based on our review of the 53 OIG reports, the scope of all of the OIG  reviews covered their agencies’ submission of spending data for the  second quarter of fiscal year 2017 (i.e., January through March 2017).  However, the files that the OIGs included in their scope to select and  review sample transactions and the type of audit standards used—such  as attestation examination engagement or performance audit—varied  among the OIGs.

According to the IG Guide, the OIGs were to select and review a  statistically valid sample of transactions, preferably from the agencies’  File C certified data submissions; if File C was unavailable or did not  contain data, they were to select their sample test items from Files D1  and D2. Based on their survey responses, we found that most OIGs  tested data from File C, File D1, File D2, or some combination of these  agency file submissions. We also found that some OIGs tested a  statistical sample of transactions in these files, while others tested all the  transactions in the files because of the small population size. Further, we  found that some OIGs used different files when testing for completeness,  timeliness, or accuracy. For example, one OIG used File C when testing  for completeness, File D1 when testing for timeliness, and File D2 when  testing for accuracy. Overall, as shown in figure 2, the source files that 47  of the 53 OIGs used for testing accuracy were as follows.

Twenty-eight OIGs selected items for testing accuracy from File C.

Twelve OIGs selected items for testing accuracy from Files D1, D2, or  both.

Seven OIGs selected items for testing accuracy from a combination of  Files C, D1, and D2.

The IG Guide also states that OIGs should conduct either attestation  examination engagements or performance audits in accordance with  generally accepted government auditing standards (GAGAS).  Performance audits are audits that provide findings or conclusions based  on an evaluation of sufficient, appropriate evidence against criteria.  Attestation examination engagements involve obtaining sufficient,  appropriate evidence with which to express an opinion stating whether  the subject matter is in conformity with the identified criteria. In contrast to  these two types of engagements that provide conclusions or opinions,  agreed-upon procedures attestation engagements do not result in  opinions or conclusions, but instead involve auditors performing specific  procedures on the subject matter and issuing a report of findings.

All 53 OIGs reported that they performed their engagements in  accordance with GAGAS; 47 OIGs reported that they conducted a  performance audit, 5 reported that they performed an attestation  examination engagement, and 1 reported that it performed an agreed- upon procedures attestation engagement. Twenty-one CFO Act agency  OIGs and 26 non-CFO Act agency OIGs conducted performance audits,  3 CFO Act agency OIGs and 2 non-CFO Act agency OIGs conducted  attestation examination engagements, and 1 non-CFO Act agency OIG  conducted an agreed-upon procedures attestation engagement.

section_title: OIG Reports Show Variations in Agencies’ Use of Data Standards and Quality of Data, and Most OIGs Made Recommendations to Address Identified Deficiencies
paragraphs: According to the OIG reports, about half of the agencies met the OMB  and Treasury requirements for implementation and use of data standards.  However, almost three-fourths of OIGs determined that their respective  agencies’ submissions were not complete, timely, accurate, or of quality.  Based on their reports and survey responses, certain OIGs also found  data errors related to problems with how Treasury’s DATA Act broker  extracted information from external award reporting systems. The FAEC  DATA Act Working Group considered these data errors to be a  government-wide issue. Other errors that the OIGs identified may have  been caused by agency-specific internal control deficiencies. Most of the  OIGs made recommendations to agencies to help address the concerns  they identified in their reports.
subsections: section_title: OIG Reports Show About Half of the Agencies Met Requirements for Implementation and Use of Data Standards
paragraphs: Based on our review of the 53 OIG reports, we found that 27 OIGs  determined that their agencies met OMB and Treasury requirements for  implementation and use of the data standards, whereas 23 OIGs  determined that their agencies did not meet these requirements. In  addition, 3 CFO Act agency OIGs did not include an assessment of their  agencies’ implementation and use of the data standards in their reports.

The OIG reports described reasons why the 23 agencies did not meet the  implementation and use of data standards requirements, including data  submissions that did not include required data elements or included data  elements that did not conform with the established data standards. For  example, one OIG reported that 74 percent of transactions it tested did  not contain program activity names or codes aligned with the President’s  Budget, and as a result, 39 percent of total obligations and 57 percent of  total expenditures from that agency’s data submission could not be  aligned with established programs. Another OIG reported that because of  inconsistent application of data standards and definitions across award  systems, the agency’s spending data were not complete, timely, or  accurate.

In their survey responses, certain OIGs identified additional concerns  about their agencies’ implementation and use of data standards and  related data elements. Specifically, six OIGs identified differences  between their agencies’ definitions of the data standards and OMB  guidance. For example, two OIGs noted differences between definitions  in OMB guidance and their agencies’ definitions of “primary place of  performance address.” One of these OIGs noted that its agency  submitted the wrong data, providing the address of the legal entity  receiving the award instead of the address of the primary place where  performance of the award will be accomplished or take place. In our  November 2017 report, we also noted that OMB guidance for this data  element was unclear and recommended that OMB clarify and align  existing guidance regarding the appropriate definitions agencies should  use to collect and report on primary place of performance and establish  monitoring mechanisms to foster consistent application and compliance.

In addition, based on their survey responses, 21 OIGs reported error  rates over 50 percent for 25 data elements. This includes 10 data  elements that were reported by multiple OIGs and 15 data elements only  reported by one OIG, as shown in table 1. There were five other data  elements with error rates over 50 percent that the FAEC DATA Act  Working Group determined to be government-wide broker-related data  reporting issues, as discussed later in this report. The OIGs’ survey  responses did not indicate whether the data elements with errors were  the result of issues related to the agencies’ implementation or use of  required data standards.

section_title: OIG Reports and Survey Responses Show Most Agencies Did Not Submit Complete, Timely, Accurate, or Quality Data
paragraphs: Based on the OIG reports, we found that 15 of the 53 OIGs determined  that their agencies’ data were generally complete, timely, accurate, or of  quality, comprising 6 CFO Act agency OIGs and 9 non-CFO Act agency  OIGs (see fig. 3). Conversely, 38 of 53 OIGs determined that their  agencies’ data were not complete, timely, accurate, or of quality,  comprising 18 CFO Act agency OIGs and 20 non-CFO Act agency OIGs.  OIG reports did not always include separate assessments for  completeness, timeliness, and accuracy, but gave an overall assessment  of the quality of the data.

As part of our OIG survey, we requested the overall error rates, agency- specific error rates, and broker error rates for each requirement— completeness, timeliness, and accuracy—used to evaluate the quality of  data tested to help provide more insights on the nature and extent of  errors that the OIGs identified. For the purposes of our survey, based on  guidance from the FAEC DATA Act Working Group and in the IG Guide,  these error rates were defined as follows:

Overall error rate is the percentage of transactions tested that were  not in accordance with policy, and includes errors due to the agency,  broker, and external award reporting systems.

Agency error rate is the percentage of transactions tested that were  not in accordance with policy, and includes only errors that were  within the agency’s control.

Broker error rate is the percentage of transactions tested that were  not in accordance with policy, and includes only errors due to the  broker and external award reporting systems.

With regard to overall error rates and the tests conducted, 40 OIGs  reported that they tested a statistical sample of transactions, 9 OIGs  reported that they tested all transactions in the populations of data, and 4  OIGs reported that they did not test any transactions or were unable to  complete their testing. As shown in figure 4, our survey results show that  the 40 OIGs that tested a statistical sample of transactions generally  reported higher (projected) overall error rates for the accuracy and  completeness of data than for the timeliness of data. We found similar  results based on our tests to assess the completeness, timeliness, and  accuracy of government-wide spending data that we tested for the same  time period, as described in our November 2017 report. More than half  of the 40 OIGs reported projected overall error rates of 25 percent or  greater for accuracy, including 8 OIGs reporting projected accuracy error  rates of over 75 percent. In contrast, more than three-fourths of the OIGs  projected overall error rates of less than 25 percent for completeness and  timeliness of their agencies’ data.

See appendix II for more details on the 53 OIGs’ individual agency testing  results, including the actual overall error rates for those OIGs that tested  the full population of transactions included in their agencies’ data  submissions and the estimated range of projected overall error rates for  OIGs that conducted a statistical sample.

The OIG survey responses that included agency-specific error rates  showed that the agency-specific error rates were similar to the overall  error rates, with accuracy of data having higher error rates than those for  completeness and timeliness. Fourteen OIGs provided agency-specific  error rates for accuracy, 13 OIGs provided agency-specific error rates for  completeness, and 12 OIGs provided agency-specific error rates for  timeliness of the data sampled.

In addition, nine OIGs reported error rates for broker-related errors that,  similar to the overall and agency-specific error rates, had higher error  rates for accuracy of data than for completeness and timeliness. The  FAEC DATA Act Working Group determined that the broker-related errors  had a government-wide impact, as discussed further below. In October  2017—1 month before the mandated reports were to be issued—the  working group provided guidance to the OIGs suggesting that they  determine and report these additional broker error rates separately  because they were not within the agencies’ control. Some OIGs may not  have reported separate agency-specific and broker error rates as their  work was already substantially completed.

Of the nine OIGs that reported they tested all transactions in the  populations of their agencies’ data, five OIGs reported actual overall error  rates and found that overall error rates for accuracy were higher than the  error rates for completeness or timeliness. Of the four OIGs that reported  agency-specific error rates, only one OIG reported an error rate for  accuracy, and it was greater than 75 percent. One OIG reported a broker  error rate, and it was higher for accuracy than for completeness or  timeliness.

In addition to using different testing methodologies (e.g., statistical  sampling or testing the full population of transactions) and source files, as  previously discussed, the OIGs also used different assumptions and  sampling criteria to design and select sample items for testing. As a  result, the overall error rates are not comparable and a government-wide  error rate cannot be projected.

section_title: DATA Act Broker-Related Issues Caused Certain Government-wide Data Reporting Errors
paragraphs: Based on discussions with OIGs, the FAEC DATA Act Working Group  identified certain data errors caused by broker-related issues that it  determined to be government-wide data reporting issues. Also, because  the broker is maintained by Treasury, these issues were beyond the  control of the affected agencies. According to the working group, these  issues involve inconsistencies in data the broker extracted from  government-wide federal financial award reporting systems, as described  in table 2. To help provide consistency in reporting these issues, the  working group developed standard report language used by OIGs in their  reports to describe the errors caused by the broker. The standard  reporting language stated that because agencies do not have  responsibility for how the broker extracts data, the working group did not  expect agency OIGs to evaluate the reasonableness of Treasury’s  planned corrective actions.

In April 2018, a Treasury official told us that the issues causing these  problems have been resolved. To address these issues, the Treasury  official stated that, among other things, Treasury implemented the DATA  Act Information Model Schema version 1.1, loaded previously missing  historical procurement data to USAspending.gov, updated how  information from FPDS-NG is mapped to File D1, and replaced ASP with  FABS. However, we plan to follow up on these efforts as a part of our  ongoing monitoring efforts.

section_title: OIGs Identified Agency- Specific Control Deficiencies That May Have Contributed to Data Errors
paragraphs: In their survey responses and OIG reports, 43 OIGs reported agency- specific control deficiencies that may have contributed to or increased the  risk of data errors. Of these 43 OIGs, 37 OIGs identified deficiencies  affecting accuracy, 32 OIGs identified deficiencies affecting  completeness, and 14 OIGs identified deficiencies affecting timeliness. A  few OIGs reported that they leveraged their financial statement audit  results, which found deficiencies in certain financial reporting controls, in  conducting their DATA Act reviews. We categorized the OIGs’ reported  control deficiencies and found that the categories with the most frequently  reported deficiencies related to their agencies’ lack of effective  procedures or controls, such as conducting reviews and reconciliations of  data submissions to source systems, and information technology system  deficiencies, as shown in figure 5. In their survey responses, OIGs  provided additional information about whether their agencies’ controls  over agency source systems and controls over the DATA Act submission  processes were properly designed, implemented, and operating  effectively to achieve their objectives. For both CFO Act and non-CFO Act  agencies, OIGs generally reported that agencies’ internal controls over  source systems and the DATA Act submission process were designed  effectively but were not implemented or operating effectively as designed.

Some examples of agency-specific control deficiencies reported by the  OIGs are as follows.

Lack of effective procedures or controls. Deficiencies where agency  procedures for reviewing and reconciling data and files to different  sources were not performed, or were performed ineffectively, or standard  operating procedures for data submissions had not been designed and  implemented. For example, some of these deficiencies related to  agencies’ lack of review or reconciliation of data in Files A and B to data  in Files D1 and D2. Further, two OIGs found that their agencies did not  perform any sort of quality review of their data until after they were  submitted to the broker. Another OIG found that its agency did not ensure  that its components developed objectives for accomplishing its data  submissions, assessed the risks to achieving those objectives, or  established corresponding controls to address them. As a result, the  agency’s DATA Act submissions included errors.

Information technology system deficiencies. Deficiencies related to  the lack of effective automated systems controls necessary to ensure  proper system user access or automated quality control procedures and  the accuracy and completeness of data, as well as systems that are not  compliant with federal financial management system requirements. For  example, one OIG noted that its agency experienced issues related to  segregation of duties and access controls that affected the agency’s  ability to ensure completeness and accuracy of data in its financial,  procurement, and grant processing systems. Another OIG found that its  agency did not complete necessary system updates to ensure that all  data were certified prior to submission. Further, an OIG reported that its  agency’s information system was unable to combine transactions with the  same unique identifiers, resulting in over 12,000 transactions being  removed because of broker warnings.

Insufficient documentation. Deficiencies related to agencies’ production  and retention of documentary evidence supporting their DATA Act  submissions. For example, three OIGs found that their agencies were  unable to provide supporting documentation for various portions of their  DATA Act submissions. Another OIG reported that one of its agency’s  components did not take effective steps to ensure that procurement and  grant personnel understood the specific documentation that should be  maintained to support data entered in grant and contract files. Further,  another OIG found that its agency did not document the process for  compiling the agency’s DATA Act submission files.

Inappropriate application of data standards and data elements.  Deficiencies related to the inappropriate use of data definition standards  or the misapplication of data elements. For example, one OIG found that  its agency did not identify the prior year funding activity names or codes  for all transactions included in its spending data submission. Another OIG  found that its agency did not consistently apply standardized object class  codes in compliance with OMB guidance, as well as standardized U.S.  Standard General Ledger account codes as outlined in Treasury  guidance. Similarly, an OIG reported instances where agency users of  certain award systems were not knowledgeable about how required  DATA Act elements were reported in their procurement system.

Data entry errors or incomplete data. Deficiencies related to controls  over data entry and errors or incomplete data in agency or government- wide external systems. For example, an OIG found that its agency did not  include purchase card transactions greater than $3,500, which  represented about 1 percent of the agency’s data submission. Another  OIG reported that its agency’s service provider did not enter  miscellaneous obligations in the data submission file because it expected  the agency to enter such transactions in the federal procurement data  system.

Timing errors. Deficiencies related to delays in reporting information to  external government-wide systems that result in errors in the data  submitted. For example, one OIG reported that its agency did not take  effective steps to ensure that contracting officers timely report required  DATA Act award attribute information in FPDS-NG. Another OIG reported  that a bureau in its agency consistently submitted certain payment files 2  months late, resulting in incomplete Files C and D2 in the agency’s data  submission.

Inaccurate broker uploads. Deficiencies related to agencies uploading  data to the broker. For example, one OIG found a lack of effective internal  controls over data reporting from its agency’s source systems to the  DATA Act broker for ensuring that the data reported are complete, timely,  accurate, and of quality. Specifically, certain components were not able to  consolidate data from multiple source systems and upload accurate data  to the broker for File C. Another OIG reported that the broker could not  identify and separate an individual component’s award data from agency- wide award data. Specifically, the broker recognized only agency-wide  award data and did not include award data from its agency’s individual  components. As a result, the OIG reported that the component did not  comply with the DATA Act requirements because its submission did not  include all of the agency’s required award data.

Reliance on manual processes. Deficiencies that cause agencies to  rely on manual processes and work-arounds. For example, one OIG  found that in the absence of system patches to map data elements  directly from feeder award systems to financial systems, its agency  developed an interim solution that relied heavily on manual processes to  collect data from multiple owners and systems and increased the risk for  data quality to be compromised. Another OIG reported that its agency’s  financial management systems are outdated and unable to meet DATA  Act requirements without extensive manual efforts, resulting in  inefficiencies in preparing data submissions.

Other. Other deficiencies including, among other things, instances where  an agency’s senior accountable official did not submit a statement of  assurance certifying the reliability and validity of the agency account-level  and award-level data submitted to the DATA Act broker, an agency did  not provide adequate training and cross-training of personnel on the  various DATA Act roles, and certain components of one agency were not  included in the agency’s DATA Act executive governance structure.

section_title: Most OIGs Made Recommendations to Agencies to Improve Data Quality and Controls
paragraphs: To help address control deficiencies and other issues that resulted in data  errors, 48 of the 53 OIGs (23 CFO Act agency OIGs and 25 non-CFO Act  agency OIGs) included recommendations in their reports. As shown in  figure 6, the most common recommendations OIGs made to their  agencies related to the need for agencies to develop controls over their  data submissions, develop procedures to address errors, and finalize or  implement procedures or guidance.

Some examples of OIG recommendations made to agencies to improve  data quality and controls are as follows.

Develop controls over submission process. Recommendations related  to controls or processes to resolve issues in submitting agency financial  system data to the broker. For example, one OIG recommended that its  agency develop and implement a formal process to appropriately address  significant items on broker warning reports, which could indicate systemic  issues.

Develop procedures to address errors. Recommendations related to  procedures to address data errors in the agency’s internal systems. For  example, one OIG recommended that its agency correct queries to  extract the correct information and ensure that all reportable  procurements are included in its DATA Act submissions.

Finalize or implement procedures or guidance. Recommendations  related to establishing and documenting an agency’s DATA Act-related  standard operating procedures or agency guidance, including the roles  and responsibilities of agency stakeholders. For example, one OIG  recommended that its agency update its guidance on what address to use  for primary place of performance to be consistent with OMB and Treasury  guidance.

Maintain documentation. Recommendations related to establishing or  maintaining documentation of the agency’s procedures, controls, and  related roles and responsibilities for performing them. For example, one  OIG recommended that its agency develop a central repository for grant  award documentation and maintain documentation to support its DATA  Act submissions.

Provide training. Recommendations related to developing,  implementing, and documenting training for an agency’s DATA Act  stakeholders. For example, one OIG recommended that its agency  provide mandatory training to all contracting officers and grant program  staff to ensure their understanding of DATA Act requirements.

Work with Treasury, OMB, and other external stakeholders.  Recommendations for the agency to work with Treasury, OMB, or other  stakeholders external to the agency to resolve government-wide issues.  For example, one OIG recommended that its agency work closely with its  federal shared service provider to address timing and coding errors that  the service provider caused for future DATA Act submissions.

Implement systems controls or modify systems. Recommendations  related to developing and implementing automated systems and controls.  For example, one OIG recommended that its agency complete the  implementation of system interfaces and new procedures that are  designed to improve collection of certain data that were not reported  timely to FPDS-NG and improve linkages of certain financial transactions  and procurement awards using a unique procurement instrument  identifier.

Increase resources. Recommendations related to increasing the staff,  resources, or both necessary to fully implement DATA Act requirements.  For example, one OIG recommended that its agency allocate the  resources to ensure that reconciliations are performed when  consolidating source system data to the DATA Act submission files.

Management for 36 agencies stated that they concurred or generally  concurred with the recommendations of their OIGs (see fig. 7).  Management at many of these agencies stated that they continued to  improve their processes and controls for subsequent data submissions. In  addition, management for seven agencies stated that they partially  concurred with the recommendations that their OIGs made. Management  for two agencies did not concur with their OIGs’ recommendations.  Management for one agency that did not concur with the  recommendations stated that they should not be held responsible for data  discrepancies that other agencies caused, and management for the other  agency stated that they followed authoritative guidance that OMB and  Treasury issued related to warnings and error messages.

section_title: OMB Staff and Treasury Officials Said They Use OIG Reports to Identify and Resolve Issues and Determine the Need for Additional Guidance
paragraphs: OMB staff told us that they reviewed the OIG reports—focusing on the 24  CFO Act agencies—to better understand issues that the OIGs identified  and to determine whether additional guidance is needed to help agencies  improve the completeness, timeliness, accuracy, and quality of their  DATA Act submissions. OMB staff explained to us how they have or are  planning to address OIG-identified issues. OMB staff told us that in April  2017 the CFO Council’s DATA Act Audit Collaboration working group was  formed, which includes officials from OMB, Treasury, and the Chief  Financial Officers (CFO) Council to foster collaboration and  understanding of the risks that were being identified as agencies  prepared and submitted their data. The working group also consults with  CIGIE, which is not a member of the working group, but its  representatives attend meetings to help the group members better  understand issues involving the OIG reviews and the IG guide. According  to OMB staff, the working group is the focal point to identify government- wide issues and identify guidance that can be clarified. They also told us  that OMB continues to meet with this working group to determine what  new guidance is needed to meet the DATA Act requirement to ensure  that the standards are applied to the data available on the website. In  June 2018, OMB issued new guidance requiring agencies to develop data  quality plans intended to achieve the objectives of the DATA Act.  According to OMB staff, OMB is committed to ensuring integrity and  providing technical assistance to ensure data quality.

Treasury officials told us that they reviewed OIG reports that were publicly  available on Oversight.gov and are collaborating with OMB and the CFO  Council to identify and resolve government-wide issues, including issues  related to the broker, so that agencies can focus on resolving their  agency-specific issues. In February 2018, the working group documented  certain topics identified for improving data quality and value.

OMB staff and Treasury officials also told us that OMB and Treasury have  taken steps to address issues we previously reported related to their  oversight of agencies’ implementation of the DATA Act. For example, we  recommended in April 2017 that OMB and Treasury take appropriate  actions to establish mechanisms to assess the results of independent  audits and reviews of agencies’ compliance with the DATA Act  requirements. The DATA Act Audit Collaboration working group is one  of the mechanisms OMB and Treasury use to assess and discuss the  results of independent audits and to address identified issues.

In November 2017, we also recommended, among other things, that  Treasury (1) reasonably assure that ongoing monitoring controls to help  ensure the completeness and accuracy of agency submissions are  designed, implemented, and operating as designed, and (2) disclose  known data quality issues and limitations on the new USAspending.gov.  Treasury has taken some steps and is continuing to take steps to address  these recommendations. For example, under the data quality section of  the About page on USAspending.gov, Treasury disclosed the requirement  for each agency OIG to report on its agency’s compliance with the DATA  Act and noted the availability of the reports at Oversight.gov.

section_title: Agency Comments
paragraphs: We provided a draft of this report to OMB, Treasury, and CIGIE for  comment. We received written comments from CIGIE that are reproduced  in appendix III and summarized below. In addition, OMB, Treasury, and  CIGIE provided technical comments, which we incorporated as  appropriate.

In its written comments, CIGIE noted that the report provides useful  information on OIG efforts to meet oversight and reporting responsibilities  under the DATA Act. CIGIE further stated that it believes that the report  will contribute to a greater understanding of the oversight work that the  OIG community performs and of agency efforts to report and track  government-wide spending more effectively.

We are sending copies of this report to the Director of the Office of  Management and Budget, the Secretary of the Treasury, the Chairperson  and Vice Chairperson of the Council of the Inspectors General on  Integrity and Efficiency, as well as interested congressional committees  and other interested parties. In addition, the report is available at no  charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact  me at (202) 512-9816 or rasconap@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix IV.

section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The Digital Accountability and Transparency Act of 2014 (DATA Act)  includes provisions requiring us to review the Offices of Inspector  Generals’ (OIG) mandated reports and issue our own reports assessing  and comparing the completeness, timeliness, accuracy, and quality of the  data that federal agencies submit under the act and the federal agencies’  implementation and use of data standards. We issued our first report on  data quality in November 2017, as required. This report includes our  review of the OIGs’ mandated reports, which were also issued primarily in  November 2017. Our reporting objectives were to describe  1.  the reported scope of work covered and type of audit standards OIGs  used in their reviews of agencies’ DATA Act spending data;  2.  any variations in the reported implementation and use of data  standards and quality of agencies’ data, and any common issues and  recommendations reported by the OIGs; and  3.  the actions, if any, that the Office of Management and Budget (OMB)  and the Department of the Treasury (Treasury) have reported taking  or planning to take to use the results of OIG reviews to help monitor  agencies’ implementation of the act.

To address our first and second objectives, we obtained and reviewed 53  OIG reports that were issued on or before January 31, 2018, including  reports related to 24 Chief Financial Officers Act of 1990 (CFO Act)  agencies and 29 non-CFO Act agencies. Of 91 entities for which  second quarter fiscal year 2017 spending data were submitted, we did not  obtain and review OIG DATA Act reports for 38 entities with obligations  totaling at least $1.2 billion (as displayed on USAspending.gov on May  23, 2018) because no reports for those entities were publicly available by  our January 31, 2018, cutoff date.

Table 3 lists the 53 agencies for which we obtained and reviewed the OIG  reports on the quality of data that agencies submitted in accordance with  DATA Act requirements.

We also developed and conducted a survey of OIGs to provide further  details on the design and results of their efforts to conduct statistical  samples to select and test agencies’ data submissions and reviews of  internal controls. In November 2017, we sent the survey to those OIGs  whose agencies originally submitted DATA Act data to Treasury’s DATA  Act broker. We received and reviewed responses from the 53 OIGs that  we obtained reports from, with 9 OIGs including the completed surveys in  their published reports and the others providing us their completed survey  responses separately. We analyzed 53 OIG reports and survey  responses, following up with OIGs for clarification when necessary.

We reviewed each of the 53 OIG reports we obtained and identified the  reported scope of work covered (e.g., the quarter of data reviewed) and  the type of audit standards OIGs used to conduct their reviews (e.g.,  performance audit or attestation examination engagement). We also  developed and used a data collection instrument to compile and  summarize the conclusions and opinions included in the OIG reports on  the completeness, timeliness, accuracy, and quality of agencies’ data  submissions and their implementation and use of data standards. During  this process, GAO analysts worked in teams of three to reach a  consensus on how these OIG conclusions and opinions were  categorized. For OIG reports that did not specifically state whether the  agencies met the DATA Act requirements, we considered the reported  results in conjunction with the more detailed information provided in the  OIG responses to our survey and made conclusions about the OIGs’  assessments based on our professional judgment.

We also reviewed the OIG reports and survey responses and used two  data collection instruments to compile, analyze, and categorize common  issues or agency-specific control deficiencies the OIGs identified in their  reviews and recommendations they made to address them. During this  process, GAO analysts worked in teams of three to obtain a consensus in  how these issues and deficiencies were categorized.

To address our third objective, we interviewed OMB staff and Treasury  officials about how they used or planned to use the results of the OIG  DATA Act reviews to assist them in their monitoring of agencies’  implementation of the act.

We conducted this performance audit from September 2017 to July 2018  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

section_title: Appendix II: Offices of Inspector General Digital Accountability and Transparency Act of 2014 Testing Results
paragraphs: In their survey responses, Offices of Inspector General (OIG) for 45  agencies reported actual overall error rates or estimated error rates and  estimated ranges of errors associated with the spending data transactions  they tested for accuracy, completeness, or timeliness (see table 4).  These results include OIGs that tested a statistical sample of  transactions, tested the full population, and conducted an assessment of  internal controls without additional substantive testing. OIGs that tested a  sample responded that they used different sampling criteria, and the  sources of files they used to select their statistical samples varied based  on the files that were available. Regardless of whether the OIG tested a  sample or the full population, some of the OIGs selected items for testing  from File C, File D1, File D2, or some combination thereof. As a result,  the overall error rates the OIGs reported are not from the same data  submission files and are not fully comparable, but are intended to provide  additional information on the individual results of the completeness,  timeliness, and accuracy of the data each agency OIG tested.

section_title: Appendix III: Comments from the Council of the Inspectors General on Integrity and Efficiency

section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Michael LaForge (Assistant  Director), Diane Morris (Auditor in Charge), Umesh Basnet, Thomas  Hackney, and Laura Pacheco made major contributions to this report.  Other key contributors include Dave Ballard, Carl Barden, Maria Belaval,  Jenny Chanley, Patrick Frey, Ricky Harrison, Jason Kelly, Jason Kirwan,  Quang Nguyen, Samuel Portnow, Carl Ramirez, Anne Rhodes-Kline, and  Dacia Stewart.

section_title: Related GAO Products
paragraphs: DATA Act: OMB, Treasury, and Agencies Need to Improve Completeness  and Accuracy of Spending Data and Disclose Limitations. GAO-18-138.  Washington, D.C.: November 8, 2017.

DATA Act: As Reporting Deadline Nears, Challenges Remain That Will  Affect Data Quality. GAO-17-496. Washington, D.C.: April 28, 2017.

DATA Act: Office of Inspector General Reports Help Identify Agencies’  Implementation Challenges. GAO-17-460. Washington, D.C.: April 26,  2017.

DATA Act: Implementation Progresses but Challenges Remain. GAO-17- 282T. Washington, D.C.: December 8, 2016.

DATA Act: OMB and Treasury Have Issued Additional Guidance and  Have Improved Pilot Design but Implementation Challenges Remain.  GAO-17-156. Washington, D.C.: December 8, 2016.

DATA Act: Initial Observations on Technical Implementation. GAO-16- 824R. Washington, D.C.: August 3, 2016.

DATA Act: Improvements Needed in Reviewing Agency Implementation  Plans and Monitoring Progress. GAO-16-698. Washington, D.C.: July 29,  2016.

DATA Act: Progress Made but Significant Challenges Must Be Addressed  to Ensure Full and Effective Implementation. GAO-16-556T. Washington,  D.C.: April 19, 2016.

DATA Act: Data Standards Established, but More Complete and Timely  Guidance Is Needed to Ensure Effective Implementation. GAO-16-261.  Washington, D.C.: January 29, 2016.

DATA Act: Progress Made in Initial Implementation but Challenges Must  be Addressed as Efforts Proceed. GAO-15-752T. Washington, D.C.:   July 29, 2015.


Question-summary annotations: questions: question: How does the DATA Act affect OIGs?
answer: The Digital Accountability and Transparency Act of 2014 (DATA Act) requires agencies' Offices of Inspector General (OIG) to issue reports on their assessments of the quality of the agencies' spending data submissions and compliance with the DATA Act.
child_questions: question: How was the scope of the OIG reviews determined?
answer: The scope of all OIG reviews covered their agencies' second quarter fiscal year 2017 submissions. The files the OIGs used to select and review sample transactions varied based on data availability, and OIGs performed different types of reviews under generally accepted government auditing standards.

question: How did OIG reports vary?
answer: Some OIGs reported testing a statistical sample of transactions that their agencies submitted and other OIGs reported testing the full population of submitted transactions.
child_questions: question: How did this variation affect reported error rates?
answer: Because of these variations, the overall error rates reported by the OIGs are not fully comparable and a government-wide error rate cannot be projected.

questions: question: What were the results of the OIG survey?
answer: OIG survey responses show that OIGs generally reported higher (projected) overall error rates for the accuracy of data than for completeness and timeliness.
child_questions: question: What types of errors were reported?
answer: OIGs reported certain errors that involve inconsistencies in how the Treasury broker (system that collects and validates agency-submitted data) extracted data from certain federal award systems that resulted in government-wide issues outside the agencies' control, while other errors may have been caused by agency-specific control deficiencies. For example, OIGs reported deficiencies related to agencies' lack of effective procedures or controls and systems issues.

question: How did OIGs respond to the survey results?
answer: Most OIGs made recommendations to agencies to address identified concerns.

questions: question: Why did OMB staff and Treasury officials review the OIG reports?
answer: OMB staff and Treasury officials told GAO that they reviewed the OIG reports to better understand issues identified by the OIGs.
child_questions: question: What guidance did OMB issue?
answer: OMB issued new guidance in June 2018 requiring agencies to develop data quality plans intended to achieve the objectives of the DATA Act.

question: What actions did Treasury take?
answer: Treasury officials told GAO that they are collaborating with OMB and the Chief Financial Officers Council DATA Act Audit Collaboration working group to identify and resolve government-wide issues.