The DATA Act became law in May 2014 and holds considerable promise for shedding more light on how federal funds are spent. To improve the transparency and quality of the federal spending data made available to the public, the DATA Act directed OMB and Treasury to establish government-wide data standards that include common data elements for reporting financial and payment information by May 2015. Under the act, federal agencies must begin reporting financial spending data using these standards by May 2017 and publicly post spending data in a machine- readable format by May 2018. The DATA Act also requires that OMB, or an agency it designates, establish a pilot program to facilitate the development of recommendations to (1) standardize reporting elements across the federal government, (2) eliminate unnecessary duplication in financial reporting, and (3) reduce compliance costs for recipients of federal awards. The act established reporting requirements and timeframes for implementation of the pilot. See figure 1 for a timeline of these deadlines. The DATA Act also sets specific requirements related to the pilot’s design. First, the pilot must collect data during a 12-month reporting cycle. The pilot must also include a diverse group of recipients such as awardees receiving a range of awards as long as the total value of the awards falls within the statutory range. To the extent practicable, the pilot is to include recipients who receive federal awards from multiple programs across multiple agencies. Finally, the pilot must include a combination of federal contracts, grants, and subawards with an aggregate value between $1 billion and $2 billion. In addition, OMB must review the information recipients are required to report to identify common reporting elements across the federal government, unnecessary duplication in financial reporting, and unnecessarily burdensome reporting requirements for recipients of federal awards. This review is to be done in consultation with relevant federal agencies and recipients of federal awards, including state and local governments and institutions of higher education. A well-developed and documented pilot program can help ensure that agency assessments produce information needed to make effective program and policy decisions. Such a process enhances the quality, credibility, and usefulness of evaluations in addition to helping to ensure that time and resources are used effectively. We have identified five leading practices that, taken together, form a framework for effective pilot design. To identify these practices, we reviewed our prior work as well as academic literature related to the design of pilot and evaluation programs. By following these leading practices, agencies can promote a consistent and effective pilot design process. We shared these practices with OMB, HHS, and GSA staff, who found them to be reasonable and appropriate, and applicable to the Section 5 Pilot. 1. Establish well-defined, appropriate, clear, and measurable objectives. Such objectives should have specific statements of the accomplishments necessary to meet the objectives. Clear and measurable objectives can help ensure that appropriate evaluation data are collected from the outset of pilot implementation so that data will subsequently be available to measure performance against the objectives. Broad study objectives should be translated into specific, researchable questions that articulate what will be assessed. 2. Clearly articulate assessment methodology and data gathering strategy that addresses all components of the pilot program and includes key features of a sound plan. Key features of a clearly articulated methodology include a strategy for comparing the pilot implementation and results with other efforts, a clear plan that details the type and source of the data necessary to evaluate the pilot, and methods for data collection including the timing and frequency. 3. Identify criteria or standards for identifying lessons about the pilot to inform decisions about scalability and whether, how, and when to integrate pilot activities into overall efforts. The purpose of a pilot is generally to inform a decision on whether and how to implement a new approach in a broader context. Therefore, it is critically important to consider how well the lessons learned from the pilot can be applied in other, broader settings. To assess scalability, criteria should relate to the similarity or comparability of the pilot to the range of circumstances and population expected in full implementation. The criteria or standards can be based on lessons from past experiences or other related efforts known to influence implementation and performance as well as on literature reviews and stakeholder input, among other sources. The criteria and standards should be observable and measureable events, actions, or characteristics that provide evidence that the pilot objectives have been met. Choosing well-regarded criteria against which to make comparisons can lead to strong, defensible conclusions. 4. Develop a detailed data-analysis plan to track the pilot program’s implementation and performance and evaluate the final results of the project and draw conclusions on whether, how, and when to integrate pilot activities into overall efforts. A detailed data-analysis plan identifies who will do the analysis as well as when and how data will be analyzed to measure the pilot program’s implementation and performance. The results will show the successes and challenges of the pilot, and in turn, how the pilot can be incorporated into broader efforts. Some elements of a detailed data-analysis plan include talking to users, managers, and developers; evaluating the lessons learned to improve procedures moving forward; and other appropriate measures. 5. Ensure appropriate two-way stakeholder communication and input at all stages of the pilot project, including design, implementation, data gathering, and assessment Appropriate two-way stakeholder communication and input should occur at all stages of the pilot, including design, implementation, data gathering, and assessment. Failure to effectively engage with stakeholders, and understand and address their views can undermine or derail an initiative. To that end, it is critical that agencies identify who the relevant stakeholders are, and communicate early and often to address their concerns and convey the initiative’s overarching benefits. OMB has established a Section 5 Pilot with two primary focus areas—one on federal grants and another on federal contracts (procurement). OMB’s Office of Federal Financial Management is responsible for the grants portion of the pilot and has designated the Department of Health and Human Services (HHS) to serve as its executing agent. On the contracting side, OMB’s OFPP is responsible for leading the procurement portion and is working with various entities including 18F and the Chief Acquisitions Officers’ Council (CAOC). Specifically, 18F is designing the system to be tested as part of the pilot. GSA’s Office of Government-wide Policy is responsible for providing federal register notices; and its Integrated Award Environment provides guidance and technical considerations. OMB launched a number of pilot-related initiatives in May 2015 and expects to continue activities until at least May 2017. As the executing agent for the grants portion of the pilot, HHS has developed six “test models” that will evaluate different approaches to potentially reducing grantee reporting burden. These six models are the specific grants tools, forms, or processes that will be tested and analyzed under the pilot to determine if adopting these changes will actually contribute to the program’s objectives of reducing reporting burden, duplication, and compliance costs. Taken as a whole, the six test models examine a variety of grant reporting issues that HHS has identified as presenting challenges. HHS officials told us that they have received comments through the National Dialogue, a website for grant recipients and contractors to discuss issues including compliance costs, reporting burden, eliminating duplication, and standardizing processes. In addition, the officials obtained feedback on areas of concern from grantees involved in earlier HHS efforts to streamline grants reporting. They used that information to inform the development of the six test models. Officials from advocacy groups representing grant recipients and federal contractors told us that they initially expected the grants portion of the pilot to be an extension of the Grants Reporting Information Project (GRIP) proof of concept that was launched following the enactment of the American Recovery and Reinvestment Act of 2009 rather than the six test models. HHS officials told us they would have liked to more fully replicate the GRIP, however, that would have required broader participation from agencies than was available for the Section 5 Pilot. The following provides high-level summaries of each of the six test models. For additional details, see appendix II. HHS intends to assess whether an online and searchable repository for data standards will facilitate grant reporting. To do this, HHS developed the Common Data Element Repository (CDER) Library, which is intended to be an authorized source for data elements and definitions for use by the federal government and recipients reporting grant information. The CDER Library is also intended to encourage the use of common definitions for grants-related terms by nonfederal stakeholders and federal agencies. As of March 2016, the publicly-available version of the CDER Library contained 112 data elements from a variety of sources, including the Federal Acquisition Regulation (FAR), OMB Circular A-11, and the Uniform Grant Guidance. It also included several data elements standardized in accordance with DATA Act requirements. HHS has developed a version of the CDER Library, accessible only to federal agencies, that contains a much more detailed database of more than 9,000 elements. This federal-agency-only version of the CDER Library also identifies which grant reporting forms these data elements come from so that users can see how many forms require the same data element and which agencies request that information from grantees. HHS officials told us that they believe the CDER Library has the potential to be a powerful tool for streamlining definitions and forms. HHS intends to test whether it will be possible to use a consolidated Federal Financial Report (FFR) to allow grantees to submit multiple reporting forms into one system. The FFR, reported on the Standard Form 425, is used for reporting grants expenditures for the recipients of federal assistance. HHS believes that a consolidated FFR will allow participants to submit complete information once instead of through multiple entry points. A consolidated FFR could provide a single point of data entry, earlier validation of FFR data, and potential future streamlining of the grants close-out process. According to HHS officials, this test model is intended to be a continuation of the GRIP launched during the American Recovery and Reinvestment Act of 2009.