Data Act: Pilot Effectively Tested Approaches for Reducing Reporting Burden for Grants but Not for Contracts
Released: 2019-04-30T00:00:00

Letter

Background

Signed into law on May 9, 2014, the DATA Act required OMB, or an agency it designated, to establish a pilot program to facilitate the development of recommendations for (1) standardized reporting elements across the federal government, (2) elimination of unnecessary duplication in financial reporting, and (3) reduction of compliance costs for recipients of federal awards. To meet these requirements, OMB established a pilot program with two components—one that focused on federal grants and another on federal contracts (procurement). OMB designated HHS as the executing agency of the grants portion of the Section 5 Pilot with oversight from OFFM. OFPP was responsible for designing and leading the procurement portion of the pilot focusing on reporting of Federal Acquisition Regulation (FAR) procurement requirements. OFPP collaborated with the Chief Acquisitions Officers’ Council and GSA on specific aspects of implementation including the development of the Central Reporting Portal, a reporting tool which is intended to centralize FAR reporting. See figure 1 for a timeline of the activities undertaken by the grants and procurement portions of the pilot as well as deadlines required by the act.

As part of our ongoing oversight of the DATA Act’s implementation, we have monitored OMB’s efforts to meet its statutory requirements related to the Section 5 Pilot. In April 2016, we reported on the design plans for the Section 5 Pilot. We found that HHS’s design for the grants portion of the pilot was generally on track to meet statutory requirements and partially adhered to leading pilot design practices. However, we also reported that the procurement portion was not on track to meet requirements, and that its plans did not follow leading pilot design practices. In response to a recommendation in our report, OMB revised its plan for the procurement portion to better reflect leading practices for pilot design identified in our April 2016 report. These changes included more fully documenting its data collection plans and including a sampling plan to meet diversity requirements for pilot participants.

According to OMB staff, the ongoing work and related grants guidance resulting from the Section 5 Pilot reflects a broader strategy for reducing federal recipient reporting burden that is outlined in the President’s Management Agenda (PMA). Released in March of 2018, and led by the Executive Office of the President and the President’s Management Council, PMA is a strategy to modernize how federal agencies deliver mission outcomes and provide services in three key areas: (1) modern information technology; (2) data, accountability, and transparency; and (3) the workforce for the 21st Century.

Several Cross-Agency Priority (CAP) goals include PMA’s milestones and activities. These CAP goals identify opportunities for multiple agencies to collaborate on government-wide efforts and report on goal progress quarterly. Two of these, CAP Goals 5 and 8, include strategies for reducing federal award recipient reporting burden. OMB staff told us that some of the findings from the Section 5 Pilot and recommendations from their subsequent report to Congress informed the focus of these CAP goals. For example, according to OMB staff, the grants portion of the Section 5 Pilot focused on identifying how changes in grants data collection and grant management may reduce federal recipient reporting burden. PMA CAP Goal 8 is described as building on these efforts by shifting the focus toward the life cycle of grants management and standardizing grants management activities using agile technology.

Section 5 Pilot Met Many but Not All Statutory Requirements

We determined that the Section 5 Pilot fully met three of the DATA Act’s statutory requirements, substantively met one, and partially met two others. The Section 5 Pilot fully met the following statutory requirements: (1) that pilot data collection cover a 12-month reporting cycle; (2) timely issuance of OMB’s report to Congress in August of 2017 to select congressional committees; and (3) that the report to Congress contain a discussion of any needed legislative actions as well as recommendations related to automating and streamlining aspects of federal financial reporting to reduce the reporting burden of federal award recipients.

We found that the pilot also substantively met the requirement that the pilot program include a combination of federal award recipients and an aggregate value of awards of not less than $1 billion but not more than $2 billion. Although the $122 billion in grants included in the pilot greatly exceeded the upper bound, this was principally a result of the decisions by OFFM and HHS to pilot different test models for reducing reporting burden, and to include a wide range of different types of grants. The total value of grant awards exceeded the amount envisioned by the act.

OMB’s August 2017 report stated that the decision to go beyond the minimum requirement of testing one approach was made in the interest of achieving the DATA Act’s objective to identify ways to reduce reporting burden as well as the effect this decision would have on the aggregate value of grants sampled. We believe that the pilot substantively met this requirement and did not identify any negative effects related to the larger aggregate value of grants, contracts, and subawards included in the grants portion of the pilot. We found that the approach followed by OMB and HHS furthered the broader objective identified by this section of the act.

In addition, we determined that the pilot partially met two of the act’s requirements. The first of these requirements concerns the act’s requirement that OMB’s report to Congress include a description of the data collected, the usefulness of the data provided, and the cost to collect pilot data from participants. The report that OMB issued to Congress in August 2017 included information on the first two of these but only partly addressed the third. Specifically, it contained cost information for only the grants portion of the pilot, stating that the cost associated with executing this portion during fiscal years 2015 through 2017 was more than $5.5 million. The report did not contain any cost information on the procurement portion of the pilot.

The DATA Act also required that OMB issue guidance to agencies for reducing reporting burden for federal award recipients—including both grantees and contractors—but the guidance subsequently issued only pertained to the grants community. We determined that OMB only partially met this requirement. On September 5, 2018, OMB issued M-18- 24: Strategies to Reduce Grant Recipient Reporting Burden. Among other things, this memorandum contained guidance to federal agencies making the SF-424B form optional based on findings from the grants portion of the pilot. Form SF-424B is used by grantees to document assurances regarding their compliance with a wide range of rules and regulations. Figure 2 summarizes our assessment.

The Grants Portion of the Pilot Identified Several Ways to Reduce Reporting Burden and Provided Support for Government-Wide Streamlining Efforts All Six Grant Test Models Reported Evidence of Reducing Burden, Increasing Accuracy, or Both

As the agency designated by OMB to execute the grants portion of the Section 5 Pilot, HHS developed and analyzed six “test models” to determine if adopting the proposed changes would contribute to the pilot program’s objectives of reducing reporting burden and duplication. These test models examined a variety of grant reporting issues that HHS had identified as presenting challenges. All but one of the test models, the Common Data Element Repository (CDER) Library 2, based their findings on data collected from grantees.

The text box below provides high-level summaries of each of the six models. Additional details on the approach followed for each model, as well as reported results, can be found in appendix II.

OMB Used Findings from the Grants Portion of the Pilot to Support Recommendations and Government-wide Guidance for Reducing Grantee Reporting Burden

OMB’s August 2017 report to Congress on the findings of the Section 5 Pilot contained three broad recommendations and stated that OMB plans to take action on these recommendations. These recommendations covered (1) standardizing core data elements, (2) eliminating duplication through auto-population of data, and (3) leveraging information technology open data standards to develop new tools across the federal government. We found that evidence from the grant test models supported all three recommendations for streamlining federal reporting discussed in the report. For example, OMB recommended that its staff standardize core data elements used for managing federal financial assistance awards based on reductions in administrative burden experienced in the CDER Library 1 test model. In another example, four test models supported OMB’s recommendation for increased use of data auto-population from existing federal data sources as a way to reduce duplication in reporting.

Findings from the grants portion of the Section 5 Pilot also provided support for government-wide efforts to streamline reporting and reduce recipient reporting burden. These include OMB’s memorandum M-18-24: Strategies to Reduce Grant Recipient Reporting Burden, which discusses efforts to automate and centralize grant management processes. Among other things, M-18-24 requires that federal agencies evaluate the systems and methods currently used to collect information from grant recipients to eliminate duplicative data requests. OMB staff confirmed that M-18-24 incorporates findings from some of the test models of the grants portion of the pilot such as the Single Audit test model, which examined reducing duplicative reporting of grant recipients’ data. The efforts to reduce duplicative reporting in M-18-24 also align with OMB’s recommendation in its August 2017 report to Congress to eliminate unnecessary duplication in reporting by leveraging information technology that can auto-populate from existing data sources.

In addition, OMB staff told us that findings from the grants portion of the pilot contributed to broader, government-wide initiatives related to federal reporting. For example, according to OMB staff, the three recommendations from the August 2017 report to Congress are reflected in CAP Goal 8 of the President’s Management Agenda, which focuses on results-oriented accountability for grants. These OMB staff also told us that findings from the grants portion of the pilot informed two CAP Goal 8 strategies. For example, the CAP Goal 8 grants management strategy focuses on standardizing grants management business processes and data. OMB developed a comprehensive taxonomy for core grants management data standards that is currently available for public comment. In addition, a second strategy focuses on incorporating a risk- based performance management approach to metrics in grant award operations to determine low-risk and high-value federal awards. CAP Goal 8 also states plans to streamline the 2019 Single Audit Compliance Supplement to focus on requirements that inform grant award performance.

Procurement Portion of Pilot Did Not Result in Sufficient or Appropriate Data to Assess Changes in Contractors’ Burden Reduction Lack of Contractor Participation and the Absence of Iterative and Ongoing Stakeholder Engagement Limited the Ability of Procurement Pilot to Achieve its Objectives

Unlike the grants portion of the pilot, the procurement portion did not result in data collection that could be used for an evidence-based assessment of ways to reduce reporting burden. OMB’s Office of Federal Procurement Policy (OFPP) sought to assess five test models that, according to the report to Congress, were essential to centralized procurement reporting. However, the pilot did not fully test any of the hypotheses associated with those test models. The reasons for not testing the hypotheses included a lack of contractor participation and a lack of iterative and ongoing stakeholder participation and engagement throughout the course of the pilot. See appendix III for additional information regarding the various procurement test models, associated hypotheses, and additional details regarding our assessment.

The procurement portion of the pilot focused entirely on the development and testing of a central reporting portal to consolidate FAR reporting requirements. According to OFPP staff, the pilot intended to eventually identify ways to centralize a wide range of reporting requirements that contractors currently meet through decentralized methods. Contractors must report many types of information depending on the contract. Toward that end, OFPP, with the assistance of GSA, created a procurement reporting website called the Central Reporting Portal. To test the efficacy of this portal for reducing burden, OFPP initially decided to examine how well it handled a specific FAR reporting requirement—the reporting of payroll data in accordance with the Davis-Bacon Act. According to pilot plans, Davis-Bacon reporting requirements were selected because they were identified by contractors as “pain points” during initial stakeholder outreach conducted in 2014 and 2015. OFPP planned to collect and analyze 1 year of weekly Davis-Bacon wage reporting data from at least 180 contractors through the Central Reporting Portal to identify how centralized reporting might reduce contractor reporting burden.

However, during the 12-month procurement data collection period, no contractors agreed to submit their Davis-Bacon data as part of the pilot. Consequently, OFPP did not collect any wage data. Despite OFPP stating in its plans and reiterating to us as late as September 2017 that it expected to be able to secure at least 180 pilot participants, only one contractor expressed interest in reporting its Davis-Bacon information using the portal. This contractor withdrew from the pilot before submitting any data through the Central Reporting Portal. OFPP staff told us they were aware of the potential for low pilot participation for Davis- Bacon reporting when pilot testing began in February 2017 because contractors already had established processes for fulfilling the highly complex Davis-Bacon reporting requirements, and pilot participation was optional. According to GSA contracting staff, the one contractor who initially expressed interest ultimately decided not to participate because the format in which the contractor tracked and reported payroll data was incompatible with that used by the pilot portal, resulting in additional burden.

However, it was not until August 2017—approximately 7 months into its year-long data collection period—that specific steps were taken to address the fact that the procurement portion of the pilot had not collected any data from Davis Bacon contractors. During this period OFPP did not conduct pilot outreach activities with the contractors, who were key to successful implementation of the pilot. OFPP staff told us that at the time of the pilot launch they learned that contractors were interested in having the Central Reporting Portal be able to communicate with third-party payroll reporting systems to automate reporting. OFPP staff said that although they are exploring this possibility, it was not a capability that was included as part of the pilot. Had this type of feedback on stakeholder needs been obtained sooner, OMB could have explored the feasibility of adding this capability to the portal or engaged in communication with stakeholders to develop alternate approaches that might have persuaded more contractors to participate.

The usefulness of iterative and ongoing communication is recognized by the Standards for Internal Control in the Federal Government. Those standards state that management should use quality information to achieve its objectives, and that management should collect quality information by engaging with stakeholders through iterative and ongoing processes and in a timely manner. In this case, key stakeholders include relevant agencies, contracting officials, and contractors using the system. OFPP’s plan for the procurement portion of the pilot recognized the importance of stakeholder engagement stating that, to include a diverse group of recipients in the pilot, they should identify eligible participants for the pilot, conduct outreach to identify participants, and repeat this process as necessary until they achieved the sample necessary to test the Central Reporting Portal. However, as previously stated, no contractors agreed to submit their Davis-Bacon data as part of the pilot. Therefore, OFPP did not repeat this process until the pilot obtained the necessary sample size. Such interactions could have provided important information on contractors’ needs and concerns that OFPP could have used to inform their decisions regarding the pilot’s implementation.

Expansion of Procurement Pilot to Include Hydrofluorocarbon Reporting Had Limitations

In November 2017, OFPP expanded the type of data accepted by the pilot to include hydrofluorocarbon (HFC) reporting, a new FAR reporting requirement. However, this choice had limitations in its suitability for providing useful data for testing the hypotheses of the five procurement test models. Unlike Davis-Bacon reporting, where contractors submit weekly reports, HFC is an annual reporting requirement for contractors that emit HFC gases over a certain threshold. The Central Reporting Portal is the only location where contractors can submit HFC reporting. For the purposes of the pilot, the Central Reporting Portal accepted HFC submissions from November 2017 through February 2018.

During the pilot, 11 HFC annual reports were submitted to the portal (see figure 3). As a result of the small number of reports collected, OMB collected much less data than it had initially expected to receive to test the capabilities of the Central Reporting Portal. If the procurement portion of the pilot had been executed as planned, it could have theoretically resulted in 9,360 Davis-Bacon submissions for analysis. A larger data set of contractors’ experiences using the Central Reporting Portal could have informed OMB’s decision-making process through analysis of more, and potentially more varied data.

In addition to the small number of submitted HFC annual reports, the decision to switch to using HFC data had another limitation. These data could not be used to examine changes in reporting burden as a result of using the Central Reporting Portal. This is because HFC reporting was a new reporting requirement, and as such, it did not have an established reporting process to use as a point of comparison to assess changes in reporting burden. The objective of the procurement pilot was to assess how centralized reporting can reduce reporting burden. This objective could not be achieved without data on the existing reporting burden.

OMB’s Recommendations for Streamlining Reporting Were Not Supported by Findings from the Procurement Portion of the Pilot

Evidence from the procurement portion of the pilot did not support OMB’s government-wide recommendations for reducing reporting burden in its August 2017 report to Congress. As previously stated, OMB’s report to Congress included three recommendations that focused on (1) standardizing core data elements, (2) eliminating duplication by using data auto-population, and (3) leveraging information technology open standards to develop new tools. As support for the first recommendation, the report stated that results from the procurement pilot test models demonstrated that standard data elements—coupled with uniform data adoption—and the ability to centrally collect and share information reduces administrative burden. Since the procurement portion of the pilot did not gather or analyze any pilot data from the Davis-Bacon participants, OMB did not assess the extent to which the ability to centrally collect data actually reduces burden.

Recommendation two stated that support from the procurement test model demonstrated that recipient burden is reduced when identical data can be entered once in one place and reused. However, the HFC data collection process did not reuse data when capturing information and did not have the ability to auto-populate data. HFC data collection was the only part of the procurement portion of the pilot that collected information that could have been used to inform this recommendation. According to OFPP staff, the Davis-Bacon portion of the portal had the capability to auto-populate data. However, no Davis-Bacon data were collected that would have allowed quantification of the effects of reusing data on reporting burden. OMB stated that support for the third recommendation included data and information collected from the pilot. Although there was some consultation with stakeholders during initial planning and design of the procurement portion of the pilot and the early development of the portal, the pilot did not actually collect any data from either Davis-Bacon contractors or through the HFC portion of the pilot in the data gathering and analysis portion of the pilot related to this recommendation.

OMB Plans to Expand Use of the Central Reporting Portal to Streamline Reporting of FAR Requirements

In August 2018, OMB announced plans to expand the use of the Central Reporting Portal for FAR reporting, stating that the portal allows contractors to report data to one central location. OFPP staff told us that they are considering centralizing a third FAR requirement using the portal in the future but have not yet determined what that will be. As discussed above, the procurement portion of the pilot did not collect sufficient data to test the effect of the portal on reporting burden. In addition, the plan for the procurement portion states that OFPP intended to analyze feedback on pilot data collection and, depending on that feedback, decide whether to expand the pilot to other FAR reporting requirements. However, the pilot did not collect any such feedback to inform its determination to expand the Central Reporting Portal in the future. As a result, OFPP has limited information regarding issues that could affect expanded use of the Centralized Reporting Portal. In the absence of such information, it is difficult for OFPP to determine whether continued or expanded use of the Central Reporting Portal will reduce reporting burden, and which additional FAR requirements, if any, to include.

Conclusions

To reduce the burden and cost of reporting for recipients of federal funds, Congress included specific provisions in the DATA Act to encourage OMB to take a deliberate and evidence-based approach toward developing guidance for federal agencies in this area. The Section 5 Pilot offered OMB a valuable opportunity—namely, to test a variety of methods and techniques at a small scale before applying them more widely. Such a process may enhance the quality, credibility, and usefulness of evaluations in addition to helping to ensure that time and resources are used more effectively. Similar to what we found when we analyzed the design of the Section 5 Pilot in 2016, our review of its implementation and the results it produced found differences between the grant and procurement portions.

OMB and HHS designed and executed a robust grants portion of the pilot that tested several different approaches for reducing the reporting burden experienced by federal grant recipients. The resulting findings were used to develop OMB’s government-wide recommendations, and to inform two subsequent goals in the 2018 President’s Management Agenda related to reducing recipient reporting burden.

In contrast, OMB did not fully implement the procurement portion of the pilot consistent with its plans. The procurement portion did not collect data to test the hypotheses associated with any of its five test models, and therefore could not provide empirical support for either OMB’s government-wide recommendations or guidance related to reducing reporting burden. Among the factors responsible for this were the lack of Davis-Bacon contractor participation and OMB’s inability to find a suitable alternative. OMB has announced its intention to expand centralized reporting for FAR requirements across government. In the absence of timely information regarding the needs and concerns of stakeholders, OMB faces the risk of experiencing implementation challenges similar to those it experienced during the pilot. Although the use of a centralized reporting portal could ultimately prove useful for reducing burden, the lack of information from stakeholders—including the contractors who would use it—raises concerns about the future success of plans for expanding the Central Reporting Portal.

Recommendation for Executive Action

The Director of OMB should ensure that information is collected regarding how centralized reporting of procurement requirements might reduce recipient reporting burden—including input from stakeholders such as contractors through an iterative and ongoing process—to inform OMB’s planned expansion of the Central Reporting Portal.

Agency Comments and Our Evaluation

We provided a draft of this report to OMB, HHS, and GSA for review and comment. HHS and GSA informed us that they had no comments. OMB provided technical comments, which we incorporated as appropriate. OMB neither agreed nor disagreed with our recommendation.

We are sending copies of this report to the appropriate congressional committees, The Secretary of Health and Human Services, The Acting Director of OMB, the Administrator of GSA, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at 202-512-6806 or sagerm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

This report assesses the extent to which (1) the Section 5 Pilot met the statutory requirements of the act, (2) the grants portion of the Section 5 Pilot demonstrated changes in federal award recipients’ reporting burden, and (3) the procurement portion of the Section 5 Pilot demonstrated changes in federal award recipients’ reporting burden.

To assess the extent to which the pilot met statutory requirements we reviewed section 5 of the Federal Funding Accountability and Transparency Act of 2006, as amended by the Digital Accountability and Transparency Act of 2014, to determine the legal requirements set forth in the act pertaining to establishing, designing, and executing the Section 5 Pilot. We compared these requirements to documents from the Office of Management and Budget (OMB) and designated agencies. These documents included pilot plans for the grants and procurement portions of the pilot, OMB’s August 2017 report to Congress, M-18-23: Shifting from Low-Value to High-Value Work and M-18-24: Strategies to Reduce Grant Recipient Reporting Burden. We also interviewed staff from agencies involved in administering and executing the pilot on how they carried out their responsibilities. These agencies included the Department of Health and Human Services (HHS), OMB’s Offices of Federal Financial Management (OFFM) and Federal Procurement Policy (OFPP), and the General Services Administration (GSA).

To assess the extent to which the grants portion of the Section 5 Pilot demonstrated changes in federal award recipients’ reporting burden, we reviewed HHS’ plans. We analyzed the plans compared to information collected from the various test models throughout the pilot. The data we assessed included survey data and analyses. We also assessed whether statements on changes in grantees’ reporting burden made in OMB’s August 2017 report to Congress were supported by documentation. We did this by verifying the statements against supporting information. We determined that the pilot data we reviewed were reliable for the purposes of our work by reviewing the data, tracing them back to underlying agency source documents, and interviewing relevant agency staff. We also interviewed OFFM staff and HHS officials on how the grants portion of the pilot was executed.

To assess the extent to which the procurement portion of the pilot demonstrated changes in reporting burden, we reviewed OMB’s plans and compared them to actions OMB took to execute the pilot. We compared OMB’s actions to execute the procurement portion of the pilot against criteria identified in Standards for Internal Control in the Federal Government. We viewed a demonstration of the Central Reporting Portal tool for reporting Davis-Bacon and hydrofluorocarbon (HFC) submissions. GSA developed the portal and OFPP provided oversight for the portal’s development. We also reviewed documentation including HFC reporting submissions made through the portal. In addition, we interviewed OFPP staff, GSA officials responsible for administering the portal, and three contracting officials from GSA who were assigned to participate in the Davis-Bacon component of the procurement portion of the pilot regarding their actions related to implementing the procurement portion of the pilot.

We conducted this performance audit from November 2017 to April 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Description of Test Models from Grants Portion of the Section 5 Pilot

This appendix provides detailed information regarding the test models from the grants portion of the Section 5 Pilot.

The Common Data Element Repository Library 1 Test Model

The Common Data Element Repository (CDER) Library is an online repository for federal grants-related data standards, definitions, and context. The library is intended to be an authorized source for data elements and definitions for use by the federal government and for recipients reporting grant information.

Hypothesis: If grant recipients are provided with definitions of data elements through the CDER Library, then they will be able to accurately complete forms in a timely manner.

Methodology: The Department of Health and Human Services (HHS) divided test model participants into two groups to read a scenario based on the grants lifecycle and complete a data collection tool. The first group used the CDER Library to complete the data collection tool while the second group used all other available sources to complete the data collection tool. After completion of the data collection tool, test model participants filled out a survey about their experiences using the CDER Library.

Test Model Metrics: Accuracy and completeness of captured data within a period of time and survey results.

Example of Test Model Results: On average, test model participants that completed a data collection tool using the CDER Library scored 11 percent higher in the accuracy of information requested and, on average, spent 6 fewer minutes when completing the tool.

Number of Test Model Participants: Fifty-nine.

The Common Data Element Repository Library 2 Test Model

The CDER Library 2 Test Model focused on identifying duplication in grant forms and data elements across the federal government based on the data standards, definitions, and context within the CDER Library 1.

Hypothesis: If duplication across forms can be identified using the CDER Library, then agencies can update or reduce forms to reduce grant recipient burden.

Methodology: HHS conducted an internal analysis of SF-424 form families, using the CDER Library, to identify duplication in data elements to determine which forms could be consolidated.

Test Model Metrics: Number of duplicative fields within form families and across forms for selected federal entities

Example of Test Model Results: The internal analysis conducted by HHS identified 371 instances of data element duplication across 10 agency grant funding applications when using standardized data elements from the CDER Library 1.

Number of Test Model Participants: Not Applicable; the CDER 2 Library Test model did not collect information from test model participants because the test model was an internal document review. The CDER Library 2 test model tested the utility of the data element definitions within the CDER Library 1.

The Consolidated Federal Financial Report Test Model

The Consolidated Federal Financial Report Test Model focused on examining the potential early validation of consolidated CFFR data and potential future streamlining of the close-out process by allowing the submission of Federal Financial Report (FFR) data in one system, rather than in multiple entry systems.

Hypothesis: If grant recipients can enter complete FFR information systematically through one entry point instead of multiple different avenues and that information could be shared electronically from that point forward, then grant recipient burden will be reduced and data accuracy will be improved.

Methodology: HHS surveyed Administration for Children and Families grant recipients on their experience submitting a consolidated FFR via HHS’s Payment Management System, and grantees on their perceptions of the process for using a consolidated FFR through facilitated discussions.

Test Model Metrics: Survey results.

Example of Test Model Results: Sixty-four percent of the CFFR test model participants reported that submitting their FFR through a single system would result in reduced reporting time. In addition, 65 percent of the CFFR test model participants believed using the payment management system for submitting FFR data would improve the accuracy of the information they submitted.

Number of Test Model Participants: One-hundred fifteen tested the pilot environment and 30 participated in the facilitated discussions.

The Single Audit Test Model

The Single Audit Test Model consisted of (1) an audit and opinions on the fair presentation of the financial statements and the Schedule of Expenditures of Federal Awards; (2) gaining an understanding of and testing internal control over financial reporting and the entity’s compliance with laws, regulations, and contract or grant provisions that have a direct and material effect on certain federal programs (i.e., the program requirements); and (3) an audit and an opinion on compliance with applicable program requirements for certain federal programs. The Single Audit Test Model focused on reducing reporting of data on duplicative forms.

Hypothesis: If grant recipients do not have to report the same information on duplicative forms—for example, the SEFA compared to the Single Audit Report Package and Data Collection Form—then grant recipients’ burden will be reduced.

Methodology: HHS collaborated with the Office of Management and Budget’s Office of Federal Financial Management and the Department of Commerce Federal Audit Clearinghouse (FAC) to create a pilot environment for test model participants to submit key portions of a modified Standard Form—Single Audit Collection. HHS conducted two focus groups with test model participants subject to the Single Audit. The first focus group discussed and completed a survey on the new form. The second group, a sample of test model participants who are subject to perform a Single Audit submitted the existing form in the FAC pilot environment, completed a separate data collection form similar to the new form, and completed a survey on the effectiveness and burden of the new form.

Test Model Metrics: Focus group feedback and survey results.

Example of Test Model Results: All test model participants with access to the Single Audit’s pilot environment believed the upload feature for reporting requirements could decrease duplication in required grant reporting.

Number of Test Model Participants: Thirteen tested the pilot environment and 123 participated in facilitated discussions.

The Notice of Award Test Model

This model focused on the feasibility of developing a standardized Notice of Award (NOA) to reduce reporting burden and facilitate access to standardized data needed to populate Single Audit information collection.

Hypothesis: If grant recipients have a standardized NOA for federal awards, then grant-reporting burden may be reduced for recipients by standardizing access to data needed to populate information collections.

Methodology: HHS divided test model participants into two groups and completed a data collection tool. The first group completed the data collection tool using three standardized NOAs, while the second group completed the data collection tool using three non-standardized NOAs. After completion of the data collection tool, test model participants self-reported their respective times to complete the data collection tool. They also filled out a survey about the standardized NOA’s impact on reporting burden and provided input on elements to include in a standardized NOA.

Test Model Metrics: Self-reported form completion time, accuracy, and survey results.

Example of Test Model Results: Test model participants with access to the standardized NOA coversheets spent an average of 3 minutes less when completing the test model’s data collection tool.

Number of Test Model Participants: One-hundred four.

The Learn Grants Test Model

The Learn Grants Test Model is a website on Grants.gov that summarizes and provides links to new and important grants information such as policies, processes, funding, and other information needed throughout the grants life cycle. The website intended to make it easier for stakeholders to find, learn about, and apply for federal grants and promote the standardization of grants terminology and data.

Hypothesis: If grant recipients are supplied with grants life cycle information in one website, then they will have increased access to grants resources and knowledge of the grants life cycle process.

Methodology: HHS developed a grants knowledge quiz from information on the Learn Grants website. HHS administered the knowledge quiz to test model participants in two phases. First, test model participants completed the knowledge quiz using existing knowledge and without the Learn Grants website. Next, test model participants completed the knowledge quiz with access to the Learn Grants website. HHS compared the results from both knowledge quizzes. After completion of the knowledge quiz, test model participants completed a survey on the usefulness of the Learn Grants website and its impact on increasing knowledge quiz scores.

Test Model Metrics: Knowledge quiz accuracy and survey results on the usefulness of Learn Grants website.

Example of Test Model Results: Test model participants experienced an average 10 percent (one quiz point) increase in their grant knowledge quiz scores when using the Learn Grants website. New grantees who participated in the test model also reported that the Learn Grants website provided useful grants information.

Number of Test Model Participants: Fifty-seven.

Appendix III: Assessment of Test Models in the Procurement Portion of the Section 5 Pilot

Appendix III: Assessment of Test Models in the Procurement Portion of the Section 5 Pilot Hypothesis not tested.

Hypothesis: Verification of FAR standards for post award reporting will confirm the value of existing data standards and reduce variations that will, in turn, reduce contractor burden and cost.

Original plan (Davis-Bacon): OFPP planned to execute this test model through focus groups. According to OFPP, no focus groups were conducted. Revised Strategy (HFC): This hypothesis could not be tested through HFC reporting because it was a reporting requirement without an existing reporting method through which to compare reporting burden.

3. Prepopulate data into the Central

Hypothesis not tested.

Reporting Data

Original Strategy (Davis-Bacon): OFPP planned to test this hypothesis by gathering data on the time it takes to submit reporting data through the Central Reporting Portal and outside of the portal, with self-reported data from contractors. According to OFPP, data were not collected due to a lack of participation in the Davis-Bacon portion of pilot. Revised Strategy (HFC): This hypothesis could not be tested through HFC reporting because it was a reporting requirement without an existing reporting method through which to compare reporting burden.

Procurement Test Model and Hypothesis GAO’s Assessment 4. Consolidate data collection and Hypothesis not tested.

access (proof of concept)

Hypothesis: If contractors can enter FAR-required reporting data systematically through one entry point instead of multiple different avenues, and that information can be shared electronically with appropriate individuals, then contractor burden will be reduced and data access improved.

Assessment Rationale Original plan (Davis-Bacon): OFPP planned to test this hypothesis by gathering data on the time it takes to submit reporting data through the Central Reporting Portal and outside of the portal, with self-reported data from contractors. OMB also planned to conduct guided discussions. According to OFPP, data were not collected due to a lack of participation in the Davis-Bacon portion of pilot. Revised Strategy (HFC): This hypothesis could not be tested through HFC reporting because it was a reporting requirement without an existing reporting method with which to compare reporting burden.

5. Central Reporting Portal can

Hypothesis not tested, but metric associated with test model was met.

Hypothesis: If interfaces can be built to support access to other reporting systems, contractor burden will be reduced.

Original plan (Davis-Bacon): According to OFPP staff, the Davis-Bacon part of the Central Reporting Portal was able to provide prepopulating of data by interfacing with other reporting systems or drop down menus for all reporting fields. However, it could not demonstrate that such prepopulation resulted in a reduction of contractor burden. Revised Strategy (HFC): This is not applicable for HFC reporting which is reported through open fields.

Although OFPP did not actually test the hypothesis associated with this test model, it did meet the metric that it had associated with the test model in its pilot plan. That metric is to develop prepopulating capabilities in the Central Reporting Portal by interfacing with other reporting systems.

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Peter Del Toro, Assistant Director; Silvia Porres-Hernandez, Analyst-in-Charge; Jazzmin Cooper; and Jimmy Nunnally made major contributions to this report. Also contributing to this report in their areas of expertise were Michael Bechetti, Jenny Chanley, Mike LaForge, Carl Ramirez, Stewart Small, Andrew J. Stephens, James Sweetman Jr., and Tatiana Winger.