Data Act: Reported Quality of Agencies' Spending Data Reviewed by OIGs Varied Because of Government-wide and Agency Issues
Released: 2018-07-23T00:00:00

Letter

Background

The DATA Act was enacted May 9, 2014, for purposes that include expanding on previous federal transparency legislation by requiring the disclosure of federal agency expenditures and linking agency spending information to federal program activities, so that both policymakers and the public can more effectively track federal spending. The act also calls for improving the quality of data submitted to USAspending.gov by holding federal agencies accountable for the completeness and accuracy of the data submitted. The Federal Funding Accountability and Transparency Act of 2006 (FFATA), as amended by the DATA Act, identifies OMB and Treasury as the two agencies responsible for leading government-wide implementation. For example, the DATA Act requires OMB and Treasury to establish government-wide financial data standards that shall, to the extent reasonable and practicable, provide consistent, reliable, and searchable spending data for any federal funds made available to or expended by federal agencies. These standards specify the data elements to be reported under the DATA Act and define and describe what is to be included in each data element, with the aim of ensuring that information will be consistent and comparable. The DATA Act also requires OMB and Treasury to ensure that the standards are applied to the data made available on USAspending.gov.

Sources of Data on USAspending.gov

USAspending.gov has many sources of data. For example, agencies submit data from their financial management systems, and other data are extracted from government-wide federal financial award reporting systems populated by federal agencies and external award recipients. A key component of the reporting framework is Treasury’s DATA Act broker (broker)—a system that collects and validates agency-submitted data to create linkages between the financial and award data prior to their publication on the USAspending.gov website.

According to Treasury guidance documents, agencies are expected to submit three data files with specific details and data elements to the broker from their financial management systems.

File A: Appropriations account. This includes summary information such as the fiscal year cumulative federal appropriations account balances and includes data elements such as the agency identifier, main account code, budget authority appropriated amount, gross outlay amount, and unobligated balance.

File B: Object class and program activity. This includes summary data such as the names of specific activities or projects as listed in the program and financing schedules of the annual budget of the U.S. government.

File C: Award financial. This includes award transaction data such as the obligation amounts for each federal financial award made or modified during the reporting quarter (e.g., January 1, 2017, through March 31, 2017).

The broker also extracts spending information from government-wide award reporting systems that supply award data (e.g., federal grants, loans, and contracts) to USAspending.gov. These systems—including the Federal Procurement Data System-Next Generation (FPDS-NG), System for Award Management (SAM), Financial Assistance Broker Submission (FABS), and the FFATA Subaward Reporting System (FSRS)—compile information that agencies and external federal award recipients submit to report, among other things, procurement and financial assistance award information required under FFATA. The four files produced with information extracted by the broker from the four systems are as follows:

File D1: Procurement. This includes award and awardee attribute information (extracted from FPDS-NG) on procurement (contract) awards and contains elements such as the total dollars obligated, current total value of award, potential total value of award, period of performance start date, and other information to identify the procurement award.

File D2: Financial assistance. This includes award and awardee attribute information (extracted from FABS) on financial assistance awards and contains elements such as the federal award identification number, the total funding amount, the amount of principal to be repaid for the direct loan or loan guarantee, the funding agency name, and other information to identify the financial assistance award.

File E: Additional awardee attributes. This includes additional information (extracted from SAM) on the award recipients and contains elements such as the awardee or recipient unique identifier; the awardee or recipient legal entity name; and information on the award recipient’s five most highly compensated officers, managing partners, or other employees in management positions.

File F: Subaward attributes. This includes information (extracted from FSRS) on awards made to subrecipients under a prime award and contains elements such as the subaward number, the subcontract award amount, total funding amount, the award description, and other information to facilitate the tracking of subawards.

The key components of the broker and how the broker operated when the agencies submitted their data for the second quarter fiscal year 2017 are shown in figure 1.

After agencies submit the three files to the DATA Act broker, it runs a series of validations and produces warnings and error reports for agencies to review. After passing validations for these three files, the agencies are to generate Files D1 and D2, the files containing details on procurement and assistance awards. Before the data are displayed on USAspending.gov, agency senior accountable officials are required to certify the data submissions in accordance with OMB guidance. Certification is intended to assure alignment among Files A, B, C, D1, D2, E, and F, and to provide assurance that the data are valid and reliable. According to Treasury officials, once the certification is submitted a sequence of computer program instructions or scripts are issued to transfer and map the data from broker data tables to tables set up in a database used as a source for the information on the website. Certified data are then displayed on USAspending.gov along with certain historical information from other sources, including Monthly Treasury Statements.

OIG Methodology and Reporting Guidance for Assessing Agencies’ DATA Act Submissions

The DATA Act requires each OIG to issue three reports on its assessment of the quality of the agency’s data submission and compliance with the DATA Act. The first report was due November 8, 2016; however, agencies were not required to submit spending data in compliance with the DATA Act until May 2017. Therefore, the Council of the Inspectors General on Integrity and Efficiency (CIGIE) developed an approach to address what it described as a reporting date anomaly; encouraged interim OIG readiness reviews and related reports on agencies’ implementation efforts; and delayed issuance of the mandated reports to November 2017, with subsequent reports following a 2-year cycle and due November 2019 and 2021.

CIGIE established the Federal Audit Executive Council (FAEC) to discuss and coordinate issues affecting the federal audit community, with special emphasis on audit policy and operations of common interest to FAEC members. FAEC formed the FAEC DATA Act Working Group to assist the OIG community in understanding and meeting its DATA Act oversight requirements by (1) serving as a working-level liaison with Treasury, (2) consulting with GAO, (3) developing a common approach and methodology for conducting the readiness reviews and mandated reviews, and (4) coordinating key communications with other stakeholders. To assist the OIG community, the FAEC DATA Act Working Group developed a common methodology and published the Inspectors General Guide to Compliance Under the DATA Act (IG Guide) for use in conducting mandated reviews.

The IG Guide includes procedures to test data in agencies’ Files A and B by reconciling these data to the information that agencies report in their quarterly SF 133, Report on Budget Execution and Budgetary Resources. The IG Guide also instructs OIGs to select a statistically valid sample of spending data from the agencies’ available award-level transactions in File C, and among other procedures, to confirm whether these data are also included in the agencies’ Files D1 and D2. The OIGs are also to confirm whether the transactions in the sample were linked to the award and awardee attributes in Files E and F. The data in Files E and F are reported by award recipients in two external government-wide systems, and are outside the direct control of the federal agencies, except for the General Services Administration, which manages these external systems. Based on additional guidance from the FAEC DATA Act Working Group, OIGs are not required to assess the quality of the award recipient-entered data that the broker extracted from the two external government-wide systems used to create Files E and F.

According to the IG Guide, the sampled spending data and testing results are to be evaluated using the following definitions for the requirements being assessed:

Completeness is measured in two ways: (1) all transactions that should have been recorded are recorded in the proper reporting period, and (2) as the percentage of transactions containing all applicable data elements required by the DATA Act.

Timeliness is measured as the percentage of transactions reported within 30 days of the end of the quarter.

Accuracy is measured as the percentage of transactions that are complete and agree with the systems of record or other authoritative sources.

Quality is defined in OMB guidance as a combination of utility, objectivity, and integrity. Utility refers to the usefulness of the information to the intended users. Objectivity refers to whether the disseminated information is being presented in an accurate, clear, complete, and unbiased manner. Integrity refers to the protection of information from unauthorized access or revision.

The IG Guide also states that OIGs should assess agencies’ implementation and use of the data standards, including evaluating each agency’s process for reviewing the 57 required data elements and associated definitions that OMB and Treasury established and documenting any variances.

Prior GAO Reports Related to the DATA Act and Data Quality

In November 2017, we issued our first report on data quality as required by the DATA Act, which identified issues with the completeness and accuracy of the data that agencies submitted for the second quarter of fiscal year 2017, use of data elements, and presentation of the data on Beta.USAspending.gov. Among other things, we recommended that Treasury disclose known data quality issues and limitations on the new USAspending.gov website. Treasury agreed with that recommendation and stated that it would develop a plan to better disclose known data quality issues. Since the DATA Act’s enactment in 2014, we have issued a series of interim reports on our ongoing monitoring of the implementation of the DATA Act and made recommendations intended to help ensure effective government-wide implementation. However, many of those recommendations still remain open.

These reports identified a number of challenges related to OMB’s and Treasury’s efforts to facilitate agency reporting of federal spending, as well as internal control weaknesses and challenges related to agency financial management systems that we and agency auditors reported that present risks to agencies’ ability to submit quality data as required under the act. For example, our prior work has identified issues with agency source systems that could affect the quality of spending data made available to the public. In April 2017, we reported a number of weaknesses and issues previously identified by agencies’ auditors and OIGs that affect agencies’ financial reporting and may affect the quality of the information reported under the DATA Act. We also reported on findings and recommendations from prior reports with issues on the four key award systems—FPDS-NG, SAM, the Award Submission Portal (ASP), and FSRS—which increase the risk that the data submitted to USAspending.gov may not be complete, accurate, and timely.

OIG Reviews of Agencies’ DATA Act Submissions Varied in Scope and Type of Standards Used

Based on our review of the 53 OIG reports, the scope of all of the OIG reviews covered their agencies’ submission of spending data for the second quarter of fiscal year 2017 (i.e., January through March 2017). However, the files that the OIGs included in their scope to select and review sample transactions and the type of audit standards used—such as attestation examination engagement or performance audit—varied among the OIGs.

According to the IG Guide, the OIGs were to select and review a statistically valid sample of transactions, preferably from the agencies’ File C certified data submissions; if File C was unavailable or did not contain data, they were to select their sample test items from Files D1 and D2. Based on their survey responses, we found that most OIGs tested data from File C, File D1, File D2, or some combination of these agency file submissions. We also found that some OIGs tested a statistical sample of transactions in these files, while others tested all the transactions in the files because of the small population size. Further, we found that some OIGs used different files when testing for completeness, timeliness, or accuracy. For example, one OIG used File C when testing for completeness, File D1 when testing for timeliness, and File D2 when testing for accuracy. Overall, as shown in figure 2, the source files that 47 of the 53 OIGs used for testing accuracy were as follows.

Twenty-eight OIGs selected items for testing accuracy from File C.

Twelve OIGs selected items for testing accuracy from Files D1, D2, or both.

Seven OIGs selected items for testing accuracy from a combination of Files C, D1, and D2.

The IG Guide also states that OIGs should conduct either attestation examination engagements or performance audits in accordance with generally accepted government auditing standards (GAGAS). Performance audits are audits that provide findings or conclusions based on an evaluation of sufficient, appropriate evidence against criteria. Attestation examination engagements involve obtaining sufficient, appropriate evidence with which to express an opinion stating whether the subject matter is in conformity with the identified criteria. In contrast to these two types of engagements that provide conclusions or opinions, agreed-upon procedures attestation engagements do not result in opinions or conclusions, but instead involve auditors performing specific procedures on the subject matter and issuing a report of findings.

All 53 OIGs reported that they performed their engagements in accordance with GAGAS; 47 OIGs reported that they conducted a performance audit, 5 reported that they performed an attestation examination engagement, and 1 reported that it performed an agreed- upon procedures attestation engagement. Twenty-one CFO Act agency OIGs and 26 non-CFO Act agency OIGs conducted performance audits, 3 CFO Act agency OIGs and 2 non-CFO Act agency OIGs conducted attestation examination engagements, and 1 non-CFO Act agency OIG conducted an agreed-upon procedures attestation engagement.

OIG Reports Show Variations in Agencies’ Use of Data Standards and Quality of Data, and Most OIGs Made Recommendations to Address Identified Deficiencies

According to the OIG reports, about half of the agencies met the OMB and Treasury requirements for implementation and use of data standards. However, almost three-fourths of OIGs determined that their respective agencies’ submissions were not complete, timely, accurate, or of quality. Based on their reports and survey responses, certain OIGs also found data errors related to problems with how Treasury’s DATA Act broker extracted information from external award reporting systems. The FAEC DATA Act Working Group considered these data errors to be a government-wide issue. Other errors that the OIGs identified may have been caused by agency-specific internal control deficiencies. Most of the OIGs made recommendations to agencies to help address the concerns they identified in their reports.

OIG Reports Show About Half of the Agencies Met Requirements for Implementation and Use of Data Standards

Based on our review of the 53 OIG reports, we found that 27 OIGs determined that their agencies met OMB and Treasury requirements for implementation and use of the data standards, whereas 23 OIGs determined that their agencies did not meet these requirements. In addition, 3 CFO Act agency OIGs did not include an assessment of their agencies’ implementation and use of the data standards in their reports.

The OIG reports described reasons why the 23 agencies did not meet the implementation and use of data standards requirements, including data submissions that did not include required data elements or included data elements that did not conform with the established data standards. For example, one OIG reported that 74 percent of transactions it tested did not contain program activity names or codes aligned with the President’s Budget, and as a result, 39 percent of total obligations and 57 percent of total expenditures from that agency’s data submission could not be aligned with established programs. Another OIG reported that because of inconsistent application of data standards and definitions across award systems, the agency’s spending data were not complete, timely, or accurate.

In their survey responses, certain OIGs identified additional concerns about their agencies’ implementation and use of data standards and related data elements. Specifically, six OIGs identified differences between their agencies’ definitions of the data standards and OMB guidance. For example, two OIGs noted differences between definitions in OMB guidance and their agencies’ definitions of “primary place of performance address.” One of these OIGs noted that its agency submitted the wrong data, providing the address of the legal entity receiving the award instead of the address of the primary place where performance of the award will be accomplished or take place. In our November 2017 report, we also noted that OMB guidance for this data element was unclear and recommended that OMB clarify and align existing guidance regarding the appropriate definitions agencies should use to collect and report on primary place of performance and establish monitoring mechanisms to foster consistent application and compliance.

In addition, based on their survey responses, 21 OIGs reported error rates over 50 percent for 25 data elements. This includes 10 data elements that were reported by multiple OIGs and 15 data elements only reported by one OIG, as shown in table 1. There were five other data elements with error rates over 50 percent that the FAEC DATA Act Working Group determined to be government-wide broker-related data reporting issues, as discussed later in this report. The OIGs’ survey responses did not indicate whether the data elements with errors were the result of issues related to the agencies’ implementation or use of required data standards.

OIG Reports and Survey Responses Show Most Agencies Did Not Submit Complete, Timely, Accurate, or Quality Data

Based on the OIG reports, we found that 15 of the 53 OIGs determined that their agencies’ data were generally complete, timely, accurate, or of quality, comprising 6 CFO Act agency OIGs and 9 non-CFO Act agency OIGs (see fig. 3). Conversely, 38 of 53 OIGs determined that their agencies’ data were not complete, timely, accurate, or of quality, comprising 18 CFO Act agency OIGs and 20 non-CFO Act agency OIGs. OIG reports did not always include separate assessments for completeness, timeliness, and accuracy, but gave an overall assessment of the quality of the data.

As part of our OIG survey, we requested the overall error rates, agency- specific error rates, and broker error rates for each requirement— completeness, timeliness, and accuracy—used to evaluate the quality of data tested to help provide more insights on the nature and extent of errors that the OIGs identified. For the purposes of our survey, based on guidance from the FAEC DATA Act Working Group and in the IG Guide, these error rates were defined as follows:

Overall error rate is the percentage of transactions tested that were not in accordance with policy, and includes errors due to the agency, broker, and external award reporting systems.

Agency error rate is the percentage of transactions tested that were not in accordance with policy, and includes only errors that were within the agency’s control.

Broker error rate is the percentage of transactions tested that were not in accordance with policy, and includes only errors due to the broker and external award reporting systems.

With regard to overall error rates and the tests conducted, 40 OIGs reported that they tested a statistical sample of transactions, 9 OIGs reported that they tested all transactions in the populations of data, and 4 OIGs reported that they did not test any transactions or were unable to complete their testing. As shown in figure 4, our survey results show that the 40 OIGs that tested a statistical sample of transactions generally reported higher (projected) overall error rates for the accuracy and completeness of data than for the timeliness of data. We found similar results based on our tests to assess the completeness, timeliness, and accuracy of government-wide spending data that we tested for the same time period, as described in our November 2017 report. More than half of the 40 OIGs reported projected overall error rates of 25 percent or greater for accuracy, including 8 OIGs reporting projected accuracy error rates of over 75 percent. In contrast, more than three-fourths of the OIGs projected overall error rates of less than 25 percent for completeness and timeliness of their agencies’ data.

See appendix II for more details on the 53 OIGs’ individual agency testing results, including the actual overall error rates for those OIGs that tested the full population of transactions included in their agencies’ data submissions and the estimated range of projected overall error rates for OIGs that conducted a statistical sample.

The OIG survey responses that included agency-specific error rates showed that the agency-specific error rates were similar to the overall error rates, with accuracy of data having higher error rates than those for completeness and timeliness. Fourteen OIGs provided agency-specific error rates for accuracy, 13 OIGs provided agency-specific error rates for completeness, and 12 OIGs provided agency-specific error rates for timeliness of the data sampled.

In addition, nine OIGs reported error rates for broker-related errors that, similar to the overall and agency-specific error rates, had higher error rates for accuracy of data than for completeness and timeliness. The FAEC DATA Act Working Group determined that the broker-related errors had a government-wide impact, as discussed further below. In October 2017—1 month before the mandated reports were to be issued—the working group provided guidance to the OIGs suggesting that they determine and report these additional broker error rates separately because they were not within the agencies’ control. Some OIGs may not have reported separate agency-specific and broker error rates as their work was already substantially completed.

Of the nine OIGs that reported they tested all transactions in the populations of their agencies’ data, five OIGs reported actual overall error rates and found that overall error rates for accuracy were higher than the error rates for completeness or timeliness. Of the four OIGs that reported agency-specific error rates, only one OIG reported an error rate for accuracy, and it was greater than 75 percent. One OIG reported a broker error rate, and it was higher for accuracy than for completeness or timeliness.

In addition to using different testing methodologies (e.g., statistical sampling or testing the full population of transactions) and source files, as previously discussed, the OIGs also used different assumptions and sampling criteria to design and select sample items for testing. As a result, the overall error rates are not comparable and a government-wide error rate cannot be projected.

DATA Act Broker-Related Issues Caused Certain Government-wide Data Reporting Errors

Based on discussions with OIGs, the FAEC DATA Act Working Group identified certain data errors caused by broker-related issues that it determined to be government-wide data reporting issues. Also, because the broker is maintained by Treasury, these issues were beyond the control of the affected agencies. According to the working group, these issues involve inconsistencies in data the broker extracted from government-wide federal financial award reporting systems, as described in table 2. To help provide consistency in reporting these issues, the working group developed standard report language used by OIGs in their reports to describe the errors caused by the broker. The standard reporting language stated that because agencies do not have responsibility for how the broker extracts data, the working group did not expect agency OIGs to evaluate the reasonableness of Treasury’s planned corrective actions.

In April 2018, a Treasury official told us that the issues causing these problems have been resolved. To address these issues, the Treasury official stated that, among other things, Treasury implemented the DATA Act Information Model Schema version 1.1, loaded previously missing historical procurement data to USAspending.gov, updated how information from FPDS-NG is mapped to File D1, and replaced ASP with FABS. However, we plan to follow up on these efforts as a part of our ongoing monitoring efforts.

OIGs Identified Agency- Specific Control Deficiencies That May Have Contributed to Data Errors

In their survey responses and OIG reports, 43 OIGs reported agency- specific control deficiencies that may have contributed to or increased the risk of data errors. Of these 43 OIGs, 37 OIGs identified deficiencies affecting accuracy, 32 OIGs identified deficiencies affecting completeness, and 14 OIGs identified deficiencies affecting timeliness. A few OIGs reported that they leveraged their financial statement audit results, which found deficiencies in certain financial reporting controls, in conducting their DATA Act reviews. We categorized the OIGs’ reported control deficiencies and found that the categories with the most frequently reported deficiencies related to their agencies’ lack of effective procedures or controls, such as conducting reviews and reconciliations of data submissions to source systems, and information technology system deficiencies, as shown in figure 5. In their survey responses, OIGs provided additional information about whether their agencies’ controls over agency source systems and controls over the DATA Act submission processes were properly designed, implemented, and operating effectively to achieve their objectives. For both CFO Act and non-CFO Act agencies, OIGs generally reported that agencies’ internal controls over source systems and the DATA Act submission process were designed effectively but were not implemented or operating effectively as designed.

Some examples of agency-specific control deficiencies reported by the OIGs are as follows.

Lack of effective procedures or controls. Deficiencies where agency procedures for reviewing and reconciling data and files to different sources were not performed, or were performed ineffectively, or standard operating procedures for data submissions had not been designed and implemented. For example, some of these deficiencies related to agencies’ lack of review or reconciliation of data in Files A and B to data in Files D1 and D2. Further, two OIGs found that their agencies did not perform any sort of quality review of their data until after they were submitted to the broker. Another OIG found that its agency did not ensure that its components developed objectives for accomplishing its data submissions, assessed the risks to achieving those objectives, or established corresponding controls to address them. As a result, the agency’s DATA Act submissions included errors.

Information technology system deficiencies. Deficiencies related to the lack of effective automated systems controls necessary to ensure proper system user access or automated quality control procedures and the accuracy and completeness of data, as well as systems that are not compliant with federal financial management system requirements. For example, one OIG noted that its agency experienced issues related to segregation of duties and access controls that affected the agency’s ability to ensure completeness and accuracy of data in its financial, procurement, and grant processing systems. Another OIG found that its agency did not complete necessary system updates to ensure that all data were certified prior to submission. Further, an OIG reported that its agency’s information system was unable to combine transactions with the same unique identifiers, resulting in over 12,000 transactions being removed because of broker warnings.

Insufficient documentation. Deficiencies related to agencies’ production and retention of documentary evidence supporting their DATA Act submissions. For example, three OIGs found that their agencies were unable to provide supporting documentation for various portions of their DATA Act submissions. Another OIG reported that one of its agency’s components did not take effective steps to ensure that procurement and grant personnel understood the specific documentation that should be maintained to support data entered in grant and contract files. Further, another OIG found that its agency did not document the process for compiling the agency’s DATA Act submission files.

Inappropriate application of data standards and data elements. Deficiencies related to the inappropriate use of data definition standards or the misapplication of data elements. For example, one OIG found that its agency did not identify the prior year funding activity names or codes for all transactions included in its spending data submission. Another OIG found that its agency did not consistently apply standardized object class codes in compliance with OMB guidance, as well as standardized U.S. Standard General Ledger account codes as outlined in Treasury guidance. Similarly, an OIG reported instances where agency users of certain award systems were not knowledgeable about how required DATA Act elements were reported in their procurement system.

Data entry errors or incomplete data. Deficiencies related to controls over data entry and errors or incomplete data in agency or government- wide external systems. For example, an OIG found that its agency did not include purchase card transactions greater than $3,500, which represented about 1 percent of the agency’s data submission. Another OIG reported that its agency’s service provider did not enter miscellaneous obligations in the data submission file because it expected the agency to enter such transactions in the federal procurement data system.

Timing errors. Deficiencies related to delays in reporting information to external government-wide systems that result in errors in the data submitted. For example, one OIG reported that its agency did not take effective steps to ensure that contracting officers timely report required DATA Act award attribute information in FPDS-NG. Another OIG reported that a bureau in its agency consistently submitted certain payment files 2 months late, resulting in incomplete Files C and D2 in the agency’s data submission.

Inaccurate broker uploads. Deficiencies related to agencies uploading data to the broker. For example, one OIG found a lack of effective internal controls over data reporting from its agency’s source systems to the DATA Act broker for ensuring that the data reported are complete, timely, accurate, and of quality. Specifically, certain components were not able to consolidate data from multiple source systems and upload accurate data to the broker for File C. Another OIG reported that the broker could not identify and separate an individual component’s award data from agency- wide award data. Specifically, the broker recognized only agency-wide award data and did not include award data from its agency’s individual components. As a result, the OIG reported that the component did not comply with the DATA Act requirements because its submission did not include all of the agency’s required award data.

Reliance on manual processes. Deficiencies that cause agencies to rely on manual processes and work-arounds. For example, one OIG found that in the absence of system patches to map data elements directly from feeder award systems to financial systems, its agency developed an interim solution that relied heavily on manual processes to collect data from multiple owners and systems and increased the risk for data quality to be compromised. Another OIG reported that its agency’s financial management systems are outdated and unable to meet DATA Act requirements without extensive manual efforts, resulting in inefficiencies in preparing data submissions.

Other. Other deficiencies including, among other things, instances where an agency’s senior accountable official did not submit a statement of assurance certifying the reliability and validity of the agency account-level and award-level data submitted to the DATA Act broker, an agency did not provide adequate training and cross-training of personnel on the various DATA Act roles, and certain components of one agency were not included in the agency’s DATA Act executive governance structure.

Most OIGs Made Recommendations to Agencies to Improve Data Quality and Controls

To help address control deficiencies and other issues that resulted in data errors, 48 of the 53 OIGs (23 CFO Act agency OIGs and 25 non-CFO Act agency OIGs) included recommendations in their reports. As shown in figure 6, the most common recommendations OIGs made to their agencies related to the need for agencies to develop controls over their data submissions, develop procedures to address errors, and finalize or implement procedures or guidance.

Some examples of OIG recommendations made to agencies to improve data quality and controls are as follows.

Develop controls over submission process. Recommendations related to controls or processes to resolve issues in submitting agency financial system data to the broker. For example, one OIG recommended that its agency develop and implement a formal process to appropriately address significant items on broker warning reports, which could indicate systemic issues.

Develop procedures to address errors. Recommendations related to procedures to address data errors in the agency’s internal systems. For example, one OIG recommended that its agency correct queries to extract the correct information and ensure that all reportable procurements are included in its DATA Act submissions.

Finalize or implement procedures or guidance. Recommendations related to establishing and documenting an agency’s DATA Act-related standard operating procedures or agency guidance, including the roles and responsibilities of agency stakeholders. For example, one OIG recommended that its agency update its guidance on what address to use for primary place of performance to be consistent with OMB and Treasury guidance.

Maintain documentation. Recommendations related to establishing or maintaining documentation of the agency’s procedures, controls, and related roles and responsibilities for performing them. For example, one OIG recommended that its agency develop a central repository for grant award documentation and maintain documentation to support its DATA Act submissions.

Provide training. Recommendations related to developing, implementing, and documenting training for an agency’s DATA Act stakeholders. For example, one OIG recommended that its agency provide mandatory training to all contracting officers and grant program staff to ensure their understanding of DATA Act requirements.

Work with Treasury, OMB, and other external stakeholders. Recommendations for the agency to work with Treasury, OMB, or other stakeholders external to the agency to resolve government-wide issues. For example, one OIG recommended that its agency work closely with its federal shared service provider to address timing and coding errors that the service provider caused for future DATA Act submissions.

Implement systems controls or modify systems. Recommendations related to developing and implementing automated systems and controls. For example, one OIG recommended that its agency complete the implementation of system interfaces and new procedures that are designed to improve collection of certain data that were not reported timely to FPDS-NG and improve linkages of certain financial transactions and procurement awards using a unique procurement instrument identifier.

Increase resources. Recommendations related to increasing the staff, resources, or both necessary to fully implement DATA Act requirements. For example, one OIG recommended that its agency allocate the resources to ensure that reconciliations are performed when consolidating source system data to the DATA Act submission files.

Management for 36 agencies stated that they concurred or generally concurred with the recommendations of their OIGs (see fig. 7). Management at many of these agencies stated that they continued to improve their processes and controls for subsequent data submissions. In addition, management for seven agencies stated that they partially concurred with the recommendations that their OIGs made. Management for two agencies did not concur with their OIGs’ recommendations. Management for one agency that did not concur with the recommendations stated that they should not be held responsible for data discrepancies that other agencies caused, and management for the other agency stated that they followed authoritative guidance that OMB and Treasury issued related to warnings and error messages.

OMB Staff and Treasury Officials Said They Use OIG Reports to Identify and Resolve Issues and Determine the Need for Additional Guidance

OMB staff told us that they reviewed the OIG reports—focusing on the 24 CFO Act agencies—to better understand issues that the OIGs identified and to determine whether additional guidance is needed to help agencies improve the completeness, timeliness, accuracy, and quality of their DATA Act submissions. OMB staff explained to us how they have or are planning to address OIG-identified issues. OMB staff told us that in April 2017 the CFO Council’s DATA Act Audit Collaboration working group was formed, which includes officials from OMB, Treasury, and the Chief Financial Officers (CFO) Council to foster collaboration and understanding of the risks that were being identified as agencies prepared and submitted their data. The working group also consults with CIGIE, which is not a member of the working group, but its representatives attend meetings to help the group members better understand issues involving the OIG reviews and the IG guide. According to OMB staff, the working group is the focal point to identify government- wide issues and identify guidance that can be clarified. They also told us that OMB continues to meet with this working group to determine what new guidance is needed to meet the DATA Act requirement to ensure that the standards are applied to the data available on the website. In June 2018, OMB issued new guidance requiring agencies to develop data quality plans intended to achieve the objectives of the DATA Act. According to OMB staff, OMB is committed to ensuring integrity and providing technical assistance to ensure data quality.

Treasury officials told us that they reviewed OIG reports that were publicly available on Oversight.gov and are collaborating with OMB and the CFO Council to identify and resolve government-wide issues, including issues related to the broker, so that agencies can focus on resolving their agency-specific issues. In February 2018, the working group documented certain topics identified for improving data quality and value.

OMB staff and Treasury officials also told us that OMB and Treasury have taken steps to address issues we previously reported related to their oversight of agencies’ implementation of the DATA Act. For example, we recommended in April 2017 that OMB and Treasury take appropriate actions to establish mechanisms to assess the results of independent audits and reviews of agencies’ compliance with the DATA Act requirements. The DATA Act Audit Collaboration working group is one of the mechanisms OMB and Treasury use to assess and discuss the results of independent audits and to address identified issues.

In November 2017, we also recommended, among other things, that Treasury (1) reasonably assure that ongoing monitoring controls to help ensure the completeness and accuracy of agency submissions are designed, implemented, and operating as designed, and (2) disclose known data quality issues and limitations on the new USAspending.gov. Treasury has taken some steps and is continuing to take steps to address these recommendations. For example, under the data quality section of the About page on USAspending.gov, Treasury disclosed the requirement for each agency OIG to report on its agency’s compliance with the DATA Act and noted the availability of the reports at Oversight.gov.

Agency Comments

We provided a draft of this report to OMB, Treasury, and CIGIE for comment. We received written comments from CIGIE that are reproduced in appendix III and summarized below. In addition, OMB, Treasury, and CIGIE provided technical comments, which we incorporated as appropriate.

In its written comments, CIGIE noted that the report provides useful information on OIG efforts to meet oversight and reporting responsibilities under the DATA Act. CIGIE further stated that it believes that the report will contribute to a greater understanding of the oversight work that the OIG community performs and of agency efforts to report and track government-wide spending more effectively.

We are sending copies of this report to the Director of the Office of Management and Budget, the Secretary of the Treasury, the Chairperson and Vice Chairperson of the Council of the Inspectors General on Integrity and Efficiency, as well as interested congressional committees and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-9816 or rasconap@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

The Digital Accountability and Transparency Act of 2014 (DATA Act) includes provisions requiring us to review the Offices of Inspector Generals’ (OIG) mandated reports and issue our own reports assessing and comparing the completeness, timeliness, accuracy, and quality of the data that federal agencies submit under the act and the federal agencies’ implementation and use of data standards. We issued our first report on data quality in November 2017, as required. This report includes our review of the OIGs’ mandated reports, which were also issued primarily in November 2017. Our reporting objectives were to describe 1. the reported scope of work covered and type of audit standards OIGs used in their reviews of agencies’ DATA Act spending data; 2. any variations in the reported implementation and use of data standards and quality of agencies’ data, and any common issues and recommendations reported by the OIGs; and 3. the actions, if any, that the Office of Management and Budget (OMB) and the Department of the Treasury (Treasury) have reported taking or planning to take to use the results of OIG reviews to help monitor agencies’ implementation of the act.

To address our first and second objectives, we obtained and reviewed 53 OIG reports that were issued on or before January 31, 2018, including reports related to 24 Chief Financial Officers Act of 1990 (CFO Act) agencies and 29 non-CFO Act agencies. Of 91 entities for which second quarter fiscal year 2017 spending data were submitted, we did not obtain and review OIG DATA Act reports for 38 entities with obligations totaling at least $1.2 billion (as displayed on USAspending.gov on May 23, 2018) because no reports for those entities were publicly available by our January 31, 2018, cutoff date.

Table 3 lists the 53 agencies for which we obtained and reviewed the OIG reports on the quality of data that agencies submitted in accordance with DATA Act requirements.

We also developed and conducted a survey of OIGs to provide further details on the design and results of their efforts to conduct statistical samples to select and test agencies’ data submissions and reviews of internal controls. In November 2017, we sent the survey to those OIGs whose agencies originally submitted DATA Act data to Treasury’s DATA Act broker. We received and reviewed responses from the 53 OIGs that we obtained reports from, with 9 OIGs including the completed surveys in their published reports and the others providing us their completed survey responses separately. We analyzed 53 OIG reports and survey responses, following up with OIGs for clarification when necessary.

We reviewed each of the 53 OIG reports we obtained and identified the reported scope of work covered (e.g., the quarter of data reviewed) and the type of audit standards OIGs used to conduct their reviews (e.g., performance audit or attestation examination engagement). We also developed and used a data collection instrument to compile and summarize the conclusions and opinions included in the OIG reports on the completeness, timeliness, accuracy, and quality of agencies’ data submissions and their implementation and use of data standards. During this process, GAO analysts worked in teams of three to reach a consensus on how these OIG conclusions and opinions were categorized. For OIG reports that did not specifically state whether the agencies met the DATA Act requirements, we considered the reported results in conjunction with the more detailed information provided in the OIG responses to our survey and made conclusions about the OIGs’ assessments based on our professional judgment.

We also reviewed the OIG reports and survey responses and used two data collection instruments to compile, analyze, and categorize common issues or agency-specific control deficiencies the OIGs identified in their reviews and recommendations they made to address them. During this process, GAO analysts worked in teams of three to obtain a consensus in how these issues and deficiencies were categorized.

To address our third objective, we interviewed OMB staff and Treasury officials about how they used or planned to use the results of the OIG DATA Act reviews to assist them in their monitoring of agencies’ implementation of the act.

We conducted this performance audit from September 2017 to July 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Offices of Inspector General Digital Accountability and Transparency Act of 2014 Testing Results

In their survey responses, Offices of Inspector General (OIG) for 45 agencies reported actual overall error rates or estimated error rates and estimated ranges of errors associated with the spending data transactions they tested for accuracy, completeness, or timeliness (see table 4). These results include OIGs that tested a statistical sample of transactions, tested the full population, and conducted an assessment of internal controls without additional substantive testing. OIGs that tested a sample responded that they used different sampling criteria, and the sources of files they used to select their statistical samples varied based on the files that were available. Regardless of whether the OIG tested a sample or the full population, some of the OIGs selected items for testing from File C, File D1, File D2, or some combination thereof. As a result, the overall error rates the OIGs reported are not from the same data submission files and are not fully comparable, but are intended to provide additional information on the individual results of the completeness, timeliness, and accuracy of the data each agency OIG tested.

Appendix III: Comments from the Council of the Inspectors General on Integrity and Efficiency

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Michael LaForge (Assistant Director), Diane Morris (Auditor in Charge), Umesh Basnet, Thomas Hackney, and Laura Pacheco made major contributions to this report. Other key contributors include Dave Ballard, Carl Barden, Maria Belaval, Jenny Chanley, Patrick Frey, Ricky Harrison, Jason Kelly, Jason Kirwan, Quang Nguyen, Samuel Portnow, Carl Ramirez, Anne Rhodes-Kline, and Dacia Stewart.

Related GAO Products

DATA Act: OMB, Treasury, and Agencies Need to Improve Completeness and Accuracy of Spending Data and Disclose Limitations. GAO-18-138. Washington, D.C.: November 8, 2017.

DATA Act: As Reporting Deadline Nears, Challenges Remain That Will Affect Data Quality. GAO-17-496. Washington, D.C.: April 28, 2017.

DATA Act: Office of Inspector General Reports Help Identify Agencies’ Implementation Challenges. GAO-17-460. Washington, D.C.: April 26, 2017.

DATA Act: Implementation Progresses but Challenges Remain. GAO-17- 282T. Washington, D.C.: December 8, 2016.

DATA Act: OMB and Treasury Have Issued Additional Guidance and Have Improved Pilot Design but Implementation Challenges Remain. GAO-17-156. Washington, D.C.: December 8, 2016.

DATA Act: Initial Observations on Technical Implementation. GAO-16- 824R. Washington, D.C.: August 3, 2016.

DATA Act: Improvements Needed in Reviewing Agency Implementation Plans and Monitoring Progress. GAO-16-698. Washington, D.C.: July 29, 2016.

DATA Act: Progress Made but Significant Challenges Must Be Addressed to Ensure Full and Effective Implementation. GAO-16-556T. Washington, D.C.: April 19, 2016.

DATA Act: Data Standards Established, but More Complete and Timely Guidance Is Needed to Ensure Effective Implementation. GAO-16-261. Washington, D.C.: January 29, 2016.

DATA Act: Progress Made in Initial Implementation but Challenges Must be Addressed as Efforts Proceed. GAO-15-752T. Washington, D.C.: July 29, 2015.