HUD implemented the MTW demonstration program in 1999. As of June 2013, 35 public housing agencies (PHA) were participating through the end of their fiscal year 2018. To put in place the innovations intended under the program’s authorizing legislation, agencies may request waivers of certain provisions in the United States Housing Act of 1937, as amended. For example, housing agencies may combine the funding they are awarded annually from different programs—such as public housing capital funds, public housing operating funds, and voucher funds—into a single, authoritywide funding source. In addition to addressing the program’s three statutory purposes—reduce costs and achieve greater cost-effectiveness in federal housing expenditures, give families with children incentives to obtain employment and become self-sufficient, and increase housing choices for low-income families—MTW agencies must meet five requirements. The agencies must (1) serve substantially the same total number of eligible low-income families that they would have served had funding amounts not been combined; (2) maintain a mix of families (by family size) comparable to those they would have served without the demonstration; (3) ensure that at least 75 percent of households served are very low-income;establish a reasonable rent policy to encourage employment and self- sufficiency; and (5) assure that the housing provided meets HUD’s housing quality standards. (4) A standard agreement (between HUD and each MTW agency) governs the conditions of participation in the program. The agreement includes an attachment that sets out reporting requirements, as well as the information that MTW agencies must include in annual reports. For example, these reports must include detailed information on the impact of each activity. MTW agencies also must self-certify that they are in compliance with three of the five statutory requirements: assisting substantially the same total number of eligible low-income families that they would have served had funding amounts not been combined; maintaining a mix (by family size) comparable to those they would have served had funding amounts not been combined under the demonstration; and ensuring that at least 75 percent of households served are very low-income. In our 2012 report, we identified a number of weaknesses related to MTW data, performance indicators, and identification of lessons learned—all of which resulted in a limited ability to determine program outcomes as they related to statutory purposes. Although MTW agencies reported annually on their activities, which included efforts to reduce administrative costs and encourage residents to work, the usefulness of this information was limited because it was not consistently outcome-oriented. For example, for similar activities designed to promote family self-sufficiency, one MTW agency reported only the number of participants, which is generally considered an output, and another did not provide any performance information. In contrast, a third agency reported on the average income of program graduates, which we consider an outcome. To be consistent with GPRAMA, HUD’s guidance on reporting performance information should indicate the importance of outcome-oriented information.specific guidance on the reporting of performance information—for Without more example, to report quantifiable and outcome-oriented information—HUD could not be assured of collecting information that reflected the outcomes of individual activities. As we reported in 2012, HUD had not identified the performance data needed to assess the results of similar MTW activities or of the program as a whole. Obtaining performance information from demonstration programs is critical—because the purpose of a demonstration is to test which approach obtains positive results. Although HUD started collecting additional data from MTW agencies (including household size, income, and educational attainment) in its MTW database, it had not analyzed the data. And since 2009, HUD had required agencies to provide information on the impact of activities, including benchmarks and metrics, in their annual MTW reports. While these reports were informative, they did not lend themselves to quantitative analysis because the reporting requirements did not call for standardized data, such as the number of residents who found employment. Whether these data would be sufficient to assess similar activities and the program as a whole was not clear, and as of April 2012 HUD had not identified the data it would need for such an assessment. HUD also had not established performance indicators for MTW. According to GPRAMA, federal agencies should establish efficiency, output, and outcome indicators for each program activity as appropriate. Federal internal control standards also require the establishment of performance indicators. As we noted in 2012, specific performance indicators for the MTW program could be based on the three statutory purposes of the program. For example, agencies could report on the savings achieved (reducing costs). However, without performance indicators HUD could not demonstrate the results of the program. The shortage of standard performance data and performance indicators had hindered comprehensive evaluation efforts, which are key to determining the success of any demonstration program. We recommended in 2012 that HUD (1) improve its guidance to MTW agencies on providing performance information in their annual reports by requiring that such information be quantifiable and outcome-oriented, (2) develop and implement a plan for quantitatively assessing the effectiveness of similar activities and for the program, and (3) establish performance indicators for the program. HUD partially agreed with these recommendations. Since our report, HUD has revised the performance reporting requirements for MTW agencies. The Office of Management and Budget (OMB) approved these revisions on May 31, 2013. The new requirements state that MTW agencies are to report standard metrics and report outcome information on the effects of MTW policy changes on residents. HUD also provided a standard format to allow analysis and aggregation across agencies for similar activities. We are currently assessing the extent to which these new requirements address our recommendations. Furthermore, as we indicated in our 2012 report, while HUD had identified some lessons learned on an ad hoc basis, it did not have a systematic process in place for identifying such lessons. As previously noted, obtaining impact information from demonstration programs is critical. Since 2000, HUD had identified some activities that could be replicated by other housing agencies. For example, a HUD-sponsored contractor developed five case studies to describe issues and challenges involved in implementing MTW. However, these and subsequent efforts had shortcomings. In most cases, the choice of lessons learned was based on the opinions of HUD or contracted staff and largely involved anecdotal (or qualitative) data rather than quantitative data. Because HUD had not developed criteria and a systematic process for identifying lessons learned, we reported in 2012 that it was limited in its ability to promote useful practices for broader implementation. Thus, we recommended that HUD create a process to systematically identify lessons learned. HUD agreed and in response, stated that once its revised reporting requirements were implemented, the resulting data would inform an effort to establish lessons learned. Consistent with this, HUD noted that one purpose of the revised reporting requirements that OMB approved in May 2013 was to identify promising practices learned through the MTW demonstration. HUD had policies and procedures in place to monitor MTW agencies but could have done more to ensure that MTW agencies demonstrated compliance with statutory requirements and to identify possible risks relating to each agency’s activities. For example, as noted in our 2012 report, HUD had not issued guidance to MTW agencies clarifying key program terms, including definitions of the purposes and statutory requirements of the MTW program. Federal internal control standards require the establishment of clear, consistent goals and objectives. Agencies also must link each of their activities to one of the three program purposes cited in the MTW authorizing legislation. However, at that time HUD had not clearly defined what some of the statutory language meant, such as “increasing housing choices for low-income families.” HUD officials acknowledged that the guidance could be strengthened. At the time, they told us that they planned to update the guidance to more completely collect information related to the program’s statutory purposes and requirements. As discussed later, HUD has since updated its guidance. Additionally, we reported in 2012 that HUD had only recently assessed agencies’ compliance with two (self-certified) requirements—to serve substantially the same total number of eligible low-income families that they would have served had funding amounts not been combined and ensure that at least 75 percent of households served were very low- income. Also, HUD had not assessed compliance with the third (also self- certified) requirement—to maintain a comparable mix of families. Federal internal control standards require control activities to be in place to address program risks. formulate an approach for assessing compliance with program requirements. Without a process for systematically assessing compliance with statutory requirements, HUD lacked assurance that agencies were complying with them. GAO/AIMD-00-21.3.1. program offices to perform an annual risk assessment of their programs or administrative functions using a HUD risk-assessment worksheet. By not performing annual risk assessments or tailoring its monitoring efforts to reflect the perceived risk of each MTW agency, HUD lacked assurance that it had properly identified and addressed risks that may prevent agencies from addressing program purposes and meeting statutory requirements. HUD also lacked assurance that it had been using its limited monitoring resources efficiently. Finally, we reported that HUD did not have policies or procedures in place to verify the accuracy of key information that agencies self-report, such as the number of program participants and the average income of residents “graduating” from MTW programs. Internal control standards and guidance emphasize the need for federal agencies to have control activities in place to help ensure that program participants report information accurately. reported performance information during their reviews of annual reports or annual site visits. GAO guidance on data reliability recommends tracing a sample of data records to source documents to determine whether the data accurately and completely reflect the source documents.performance information, it lacked assurance that this information was accurate. To the extent that HUD relied on this information to assess program compliance with statutory purposes and requirements, its analyses were limited. GAO/AIMD-00-21.3.1 and GAO-01-1008G.