Rental Housing Assistance: HUD Data on Self-Sufficiency Programs Should Be Improved
Released: 2013-08-08T00:00:00

Letter

Background

HUD housing serves populations that include persons in a position to increase their self-sufficiency and those who need long-term support (for example, the elderly and persons with severe disabilities). To assist those who can improve their self-sufficiency, HUD permits PHAs to allocate space in its public housing to offer training programs and information sessions. It also awards grants to PHAs to encourage them to work with local social service providers in offering education, job training, and other supportive services to residents of public housing and recipients of vouchers. The purpose of these programs is to increase the earned incomes of residents and reduce their reliance on cash assistance and housing assistance.

HUD-Funded Self- Sufficiency Programs

For this report, we examined five HUD programs that fund self-sufficiency efforts in whole or in part. Three grant programs only award grants to support self-sufficiency activities: HCV FSS, PH FSS, and ROSS SC. FSS was authorized in 1990 to help families receiving vouchers or in public housing become self-sufficient through referrals to education, training, and other supportive services and case management. PHAs use grant funds to pay program coordinators who link residents to the supportive services they need to achieve self-sufficiency. Families in either FSS program sign a contract with their PHA, requiring that all family members become independent of cash welfare assistance and the head of the family seek and maintain employment. Both programs feature case management, referrals to supportive services, and an escrow account that accumulates balances, or savings, for the tenant based on increases in tenant contributions toward rent. The ROSS SC program provides funding to hire service coordinators to assess the needs of public housing residents and coordinate available resources in the community to meet those needs.

For the FSS programs, the escrow accounts are incentives to increase work effort and earnings. Specifically, when participants have to pay a higher rent after their earned income increases, the PHA calculates an escrow credit that is deposited each month into an interest-bearing account (see fig. 1). Families that successfully complete their contract for either FSS program receive their accrued escrow funds. In general, in order to complete an FSS contract, the family head must certify that no member of the family has received welfare for at least one year, and the family head must be employed.

HUD has two other programs—MTW and HOPE VI Revitalization Grants (HOPE VI)—that allow participating PHAs to determine how they will encourage self-sufficiency. The purposes of HUD’s MTW demonstration program include providing PHAs the flexibility to design and test policies that give incentives to families with children to become economically self- sufficient. For example, PHAs in MTW can alter eligibility and rent policies. Through the HOPE VI program, participating PHAs (HOPE VI agencies) can use HOPE VI grants to demolish, rehabilitate, or replace severely distressed public housing and also provide community and supportive services to help residents achieve self-sufficiency. HOPE VI agencies can spend up to 15 percent of their revitalization grant funds for community and supportive services activities, which HUD defines as any activity designed to promote upward mobility, self-sufficiency, and improved quality of life for residents of the public housing project involved.

HUD Indicators of Self-Sufficiency

For the two FSS programs, HUD regulations state that self-sufficiency means that a family is no longer receiving housing assistance or welfare assistance. However, the regulations also indicate that while achieving self-sufficiency is an FSS objective, no longer receiving housing assistance is not a requirement for completing the program and receiving escrow funds. HUD does not define self-sufficiency for the other three programs we reviewed. Thus, HUD does not have a uniform measure of self-sufficiency across the programs. HUD officials with responsibility for administering all five programs noted that a single definition of self- sufficiency would not be useful, particularly because levels of self- sufficiency could differ for different groups. However, officials noted that the concept is commonly understood to indicate that a family does not rely on government programs that are intended to address poverty.

HUD, HUD Strategic Plan Fiscal Year 2010-2015 (Washington, D.C.: May 2010). disabilities, services should help improve living conditions and enable residents to age in-place. For the HOPE VI program, indicators of progress towards self-sufficiency include the number of residents who obtained a high school or equivalent education, obtained new employment, and completed a job training program.

In contrast, MTW agencies determine their own measures of residents’ progress towards self-sufficiency. In a 2012 report on the MTW program, we reported that HUD had not defined key program terms, including the statutory purpose of encouraging employment and self-sufficiency, and We recommended that HUD issue guidance that clarifies such terms.also noted the limited usefulness of having MTW agencies devise their own metrics, particularly when they are not outcome-oriented. We recommended that HUD improve its guidance to MTW agencies by requiring that they provide performance information that is quantifiable and outcome-oriented. HUD has taken steps to revise its reporting guidance, and the Office of Management and Budget (OMB) approved revised guidance on May 31, 2013.

Data Collection

HUD requires all PHAs to collect detailed data from their residents. PHAs must submit these data electronically through the PIC at least annually for each household that participates in assisted housing programs. According to HUD guidance, PIC can be used to create reports, analyze programs, monitor PHAs, and provide information about those that live in HUD- subsidized housing. The data collected include amounts and sources of income; the amount of rent paid; and the presence of household heads and members who are elderly or have disabilities. MTW agencies can submit some tenant-related data into a separate module in PIC called MTW-PIC, which was created in 2007 to better accommodate some of the activities allowed under MTW. Most MTW agencies had transitioned to it by 2008.

PHAs that receive either of the FSS grants, including MTW agencies, must report additional data about participating households into a section of the PIC and MTW-PIC systems called the FSS Addendum. The addendum captures data on each family when it enters either of the two FSS programs and should be updated annually. Specifically, the addendum includes fields for PHAs to report whether the head of the household works full-time, part-time, or is unemployed; the highest grade of education for the head of the household; and the sources of assistance received by the family, such as cash or food assistance.

Participation Information for Self- Sufficiency Programs Was Limited

Available Data Indicate That Annual Amounts of Grant Awards in Three Self-Sufficiency Programs Increased

The annual totals for grant awards in the two FSS and ROSS SC programs have increased in recent years. As shown in table 2, the amount HUD awarded to PHAs in constant 2013 dollars through the two FSS programs increased from about $64 million in fiscal year 2006 to about $78 million in fiscal year 2011, with the majority of funds being awarded through the HCV FSS program. For the ROSS SC program, the total amount awarded, in constant 2013 dollars, increased from $30 million in fiscal year 2008 to $35 million in fiscal year 2011.

Escrow payments to households represent additional program expenditures associated with the two FSS programs. As previously described, PHAs establish escrow accounts for households that participate in FSS. PHAs generally disburse the amount in the escrow account (the excess of any amount owed to the PHA by the family) to the head of the family when the contract of participation has been completed. As previously noted, in order to successfully complete an FSS contract, the head of the family must be employed and no members may be receiving welfare. According to HUD data adjusted to 2013 dollars, the agency disbursed nearly $82 million to FSS participants between fiscal years 2006 and 2011 (see table 3).

Expenditures for community and supportive services, adjusted to 2013 dollars, represented 7 percent or less of HOPE VI grant awards during fiscal years 2006-2010. HUD data indicate that the 29 PHAs that were awarded HOPE VI revitalization grants between fiscal years 2006 and 2010 had spent about 5 percent of these funds on community and supportive services as of December 2012 (see table 4). Of the 29 grants, 4 were closed and 25 were open. HOPE VI agencies with open grants could spend more of their revitalization grant on supportive services, up to 15 percent. HOPE VI agencies also can leverage other sources of funding for community and supportive services.

The amounts that MTW agencies spend on activities intended to increase resident self-sufficiency are not known for the program as a whole. MTW agencies are not required to expend a specific proportion of their HUD funds on activities that encourage work and self-sufficiency. However, they must implement activities that address the program’s statutory purposes (which include encouraging work and self-sufficiency), and annually submit written reports to HUD with information on each activity they undertake and its linkage to the program’s statutory purposes. MTW agencies had been required to annually report to HUD financial data on the sources and uses of their funds. A HUD official with responsibility for administering the MTW program stated that the agency had not analyzed this information and that MTW agencies had reported it using varying formats. According to HUD, with the implementation of revised MTW reporting requirements, MTW agencies will be required to report data on the costs of self-sufficiency activities using standardized metrics.

Program Participation Not Comprehensively Known Because of Data Limitations

Residents’ participation in the five self-sufficiency programs was not comprehensively known because the data were not reliable, aggregated programwide, or collected for all participants. Internal control standards state that transactions should be promptly recorded to maintain their relevance and value to management in controlling operations and making decisions. This applies to the entire life cycle of a transaction or event. They also state that program managers need operational data to determine whether they are meeting their agencies’ strategic plans and meeting their goals for the effective and efficient use of resources.

The total number of families in either of the two FSS programs cannot be reliably assessed based on available PIC data for fiscal years 2006-2011 because of missing program start dates, exit dates, and annual updates. According to HUD guidance, PHAs that receive FSS grants must update information about each participating family at least annually. As a part of these updates, PHAs are supposed to indicate whether the family is active or has exited the program. For those who exited the program, PHAs are supposed to indicate whether they completed the program or stopped participating for other reasons. These data must be reported in the FSS addendum of PIC or MTW-PIC. NOFAs consistently have cited PIC as a data source for FSS program participation counts, which HUD uses in part to determine eligibility for funding. For some years, the notices also stated that applicants for grant funding could use other documentation for participation counts, such as a separate HUD form or MTW documents.

We excluded MTW households from our analysis of FSS participation because available data were not reliable prior to 2011. these families. For 11 percent of the families that began the FSS programs in 2006, HUD data do not indicate the families had exited the programs, although other HUD data indicated that the families no longer received housing assistance. Similarly, HUD data also indicate that 11,596 families exited the two programs in fiscal year 2011 (some completing the program and others leaving before completion), but did not indicate a start date for about one quarter of these families. Because both FSS programs are intended to be 5-year programs, it is reasonable to expect that most of the families that began participating in fiscal year 2006 would have exited or completed the program by fiscal year 2011.

HUD began posting FSS participation data from the PIC system online beginning with the fiscal year 2009 funding announcement for HCV FSS and the fiscal year 2012 funding announcement for PH FSS. According to these data, the number of families participating in HCV FSS has increased in recent years (see table 5).

HUD officials told us that posting participation data online has emphasized to PHAs the importance of ensuring the data they submit into PIC are accurate. If a PHA believed the participation number HUD posted was incorrect and that it would be underfunded or ineligible, that PHA could submit documentation to confirm a higher number. However, as a part of the NOFA process, neither HUD nor the PHA were required to make corrections to the PIC system. Officials from one of the PHAs with whom we met stated that the enrollment data in PIC generally are inconsistent with their internal records. The staff stated that they have worked with HUD to correct PIC, but noted that once some errors were fixed, new errors emerged. Staff from another PHA, which is also a MTW agency, stated that their FSS enrollment data do not appear in PIC; thus, they must create ad hoc reports using their own internal systems.

According to HUD officials with responsibility for implementing the FSS programs, the FSS records could be incomplete or incorrect for several reasons. For example, if a participating family left public housing or the voucher program, the responsible PHA might not update the FSS portion of PIC to reflect the departure. PHAs are supposed to enter the exit date, whether the family completed their contract of participation, or one of five primary reasons for leaving the program without completing it. According to these officials, no HUD staff have specific responsibility for monitoring the completeness of FSS participants’ records in PIC. HUD officials also stated that there are challenges with the PIC system. They were aware of cases in which PHAs have entered data into PIC, but either the entries were not saved or they overwrote previously entered data. PHA officials with whom we met, as well as HUD staff with responsibility for the PIC system, stated that PHAs cannot readily run reports that show FSS participation data, a feature which would enable them to identify and correct missing or incorrect data. While PHAs can run ad hoc reports from PIC, the functionality of the system is limited, and records older than 18 months cannot be accessed. We previously reported on the weaknesses associated with HUD’s antiquated technologies. Without complete information on families’ participation in either FSS program (including a reliable count of program participants, participants’ length of time in the program, and reasons participants do not complete the programs), HUD lacks accurate information to make grant funding decisions. Moreover, by not analyzing the extent to which PHAs have reported required data into PIC (and MTW-PIC), HUD’s ability to effectively oversee the program is limited.

Participation data for the ROSS SC program from fiscal years 2008 (the year the current version of this program started) through 2011 also were limited, primarily because of the lack of reporting guidance for the program and the difficulty of aggregating the data. HUD does not require PHAs to report whether a resident received services through this program in its information systems; rather, HUD collects ROSS SC participation data from PHAs through individual Excel-based reporting tools. According to HUD officials with responsibility for the PIC and MTW-PIC systems, these systems were intended to collect resident characteristics and not to enable PHAs to report data on residents’ participation in specific programs, such as ROSS SC. PHAs must use the tool to apply for grant funding, inserting projections for the number of residents or households they intend to serve. Grant recipients then must submit annual updates, reporting on the number of residents or households actually assisted. Internal control standards state that transactions should be promptly recorded to maintain their relevance and value to management in controlling operations and making decisions. They also state that program managers need operational data to determine whether they have been meeting their agencies’ strategic plans and meeting their goals for the effective and efficient use of resources. As we previously reported, according to OMB, being able to track and measure specific program data can help agencies diagnose problems, identify drivers of future performance, evaluate risk, support collaboration, and inform follow-up actions. Analyses of patterns and anomalies in program information also can help agencies discover ways to achieve more value for the taxpayers’ money.

According to HUD officials, the data that individual PHAs report using the Excel-based reporting tool cannot be easily or reliably aggregated to provide a count of resident participation across all PHAs. HUD officials who manage the ROSS SC program did not attempt to aggregate data reported through individual reporting tools until 2013 in response to our review. While HUD provided us with annual program participation data based on the tools, we determined that the data were not sufficiently reliable for these purposes. Specifically, we found duplicate records; differences in the time periods for which PHAs were reporting data; and different results for the “number of persons receiving services,” the “number of persons served,” and the sum of counts of persons served by age category. HUD officials told us that there are no line-level instructions for reporting these data. Thus, recipients of ROSS SC grants have not been given any formal guidance on what should be reported into these fields. Because of these limitations, HUD does not use the reporting tools to prepare official counts of residents assisted through the ROSS SC program. Rather, HUD multiplies the number of coordinators funded each year by 50 (the minimum number of residents who must be assisted under the terms of the grant). Based on this formula, they can report on the minimum number of residents who likely were assisted. Using this approach, HUD estimated that from 6,450 to 7,800 residents received services through the ROSS SC program annually from fiscal years 2008 through 2011. While this estimate provides information on the minimum number of residents who likely received assistance, it is not based on a count of the number of residents who actually received assistance.

HUD officials who manage the ROSS SC program stated that the current process for collecting data from the reporting tools was supposed to be temporary, and a planned web-based replacement was never developed. Additionally, the officials stated that the data PHAs report into the tool were not always comparable because PHAs have varying interpretations of what they should report, and HUD never developed program-specific reporting guidance. Without developing a reliable process for collecting and analyzing data on the number of residents assisted through the ROSS SC program, HUD lacks basic information needed to manage the program.

According to HUD participation data for HOPE VI (available on a cumulative basis), at least 73,000 working-age, nondisabled residents have participated in the community and supportive services component of the HOPE VI program since the program began in 1993. For the 280 revitalization grants awarded through the program since its inception, HOPE VI agencies self-reported that about 55,000 original residents (those who lived at the site before revitalization) participated in a program or service designed to help them progress towards self-sufficiency. In addition, HOPE VI agencies self-reported that nearly 18,000 additional individuals who became residents at the revitalized HOPE VI sites also participated in community and supportive services.

Programwide data on residents’ participation in MTW activities related to increasing self-sufficiency from fiscal years 2006 through 2011 generally were unavailable. While MTW-PIC was created in 2007 to better fit the needs of MTW agencies, this system was not designed to collect activity- level data (unless a household was participating in one of the two FSS programs). Moreover, HUD officials do not consider the data in MTW-PIC to be reliable prior to 2011 because some MTW agencies were still transitioning to it through the end of 2010. As a result, officials with responsibility for administering the MTW program have not used MTW- PIC as a tool for analyzing residents’ participation in activities related to increasing self-sufficiency. In addition, HUD does not analyze the data that MTW agencies provide in their annual MTW reports, including data on residents’ participation in activities related to self-sufficiency, because reporting requirements do not call for the reporting of standardized data, such as the number of residents who found employment.

We previously recommended that HUD develop and implement a plan for quantitatively assessing the effectiveness of similar activities, which would include activities related to encouraging self-sufficiency. HUD agreed that quantitatively assessing the effectiveness of similar activities was an important step. HUD has made revisions to its reporting requirements, which were approved by OMB in May 2013.

According to our analysis of available HUD data, 338,900 households received rental housing assistance from an MTW agency in fiscal year 2011. Because HUD officials do not believe their PIC or MTW-PIC systems contain reliable data from MTW agencies prior to 2011, we did not attempt to analyze these data. Of the MTW households, 14,314 were reported as participating in the two FSS programs in fiscal year 2011. According to HUD officials, most MTW agencies participate in FSS, but the quality of their FSS data reporting is unknown.

HUD Has Not Fully Utilized the Data It Collects on Self- Sufficiency Programs

HUD Collects Data on Indicators of Self-Sufficiency

HUD requirements for collecting data on indicators of self-sufficiency vary by program. HUD requires PHAs to collect and report into PIC or MTW- PIC certain types of detailed information on every resident of HUD- assisted rental housing. For instance, PHAs collect data on the amount and sources of residents’ income, including whether it is earned or provided through disability payments, Temporary Assistance for Needy Families, or other sources. PHAs are supposed to collect and report this type of information into HUD’s PIC system at least annually. For residents affiliated with the MTW program, this information is reported into MTW- PIC, a separate module in the PIC system.

In addition to reporting basic demographic and income data on each resident, HUD requires PHAs that implement the five programs to report additional indicators of residents’ progress towards self-sufficiency using information systems, an Excel-based reporting tool, and narrative reports.

FSS programs and ROSS SC. As discussed earlier in this report, HUD requires PHAs that receive FSS grants to enter additional data in the FSS Addendum section of PIC on each family upon entry into an FSS program and update this information annually. For example, PHAs enter whether the head of the household works full-time, part- time, or is unemployed; and the highest grade of education for the head of the household. For the two FSS programs and the ROSS SC program, HUD also requires PHAs to annually enter summary output and outcome data related to their residents’ progress towards self- sufficiency into the previously described Excel-based reporting tool. Data fields in the tool include the number of households that increased their income, moved to nonsubsidized housing, and the number of residents that obtained a high school diploma.

HOPE VI. HUD requires participating agencies to submit quarterly reports of summary data on residents’ progress towards self- sufficiency into the HOPE VI Quarterly Reporting System. For example, HOPE VI agencies must report the total number of residents who participated in activities that facilitate self-sufficiency, including the numbers enrolled in counseling programs, job training, and General Education Development classes.

MTW. HUD requires MTW agencies to submit annual reports containing summary information about the impact of the activities intended to encourage resident self-sufficiency they have been implementing. The agencies were able to use metrics of their choosing at the time of our review.

HUD Has Lacked a Strategy for Using Collected Data to Evaluate the Impact of Four Programs

HUD has performed limited analyses—to assess outcomes for the programs as a whole—of data related to self-sufficiency outcomes that FSS grant recipients must report into its information systems. In addition, HUD has not analyzed similar data that FSS, ROSS SC, and MTW agencies must report through other mechanisms to assess each program as a whole. Standards for internal control emphasize the need for federal agencies both to collect reliable information with which to manage their programs and review the integrity of performance measures. Moreover, these standards emphasize the need for program managers to collect operational data to determine whether they have been meeting their goals for the efficient and effective use of resources. Additionally, the GPRA Modernization Act of 2010 (GPRAMA) emphasizes the need for information on the effectiveness of federal programs to improve congressional decision making.

Based on the information that PHAs must report in PIC and the FSS Addendum in PIC, HUD has performed limited analyses of the data related to self-sufficiency outcomes for the two FSS programs. As previously discussed, FSS participation data were sometimes incomplete and therefore of questionable reliability. Specifically, HUD’s data lacked start dates and annual updates for some FSS participants. Despite these limitations, HUD has used these data in its Congressional Budget Justifications. For example, in the 2014 Congressional Budget Justification HUD reported that as of March 30, 2012, a total of 57,087 families were enrolled in the two FSS programs. Without annually updated records for each family, HUD cannot reliably determine whether families were still active program participants.

Additionally, HUD has not analyzed and reported on the experiences of all families that start the FSS programs, including the extent to which they completed the program, the primary reasons they exited the program without completing it, and the extent to which required data are missing. According to HUD, they have not conducted such analysis due their own concerns about the usefulness of available data. As described previously, internal control standards state that such information can help agencies determine whether they have been meeting operational goals and using resources effectively. According to HUD officials with responsibility for administering the two FSS programs, they were aware that some PHAs did a better job of reporting into PIC than others, which affected the completeness and reliability of the data. We acknowledge that analyzing this information and summarizing overall changes in indicators of self- sufficiency among participants would not yield definitive results on the impact of FSS on resident self-sufficiency. But by not analyzing the data that FSS grant recipients must report in HUD’s information systems more thoroughly, HUD has been missing an opportunity to gain valuable information about the results of FSS programs for certain agencies. That is, for those agencies that have submitted complete data (all annual updates as well as information on whether and how the participant exited the program), HUD could review available information, such as changes in income and employment. Also, HUD has been missing an opportunity to identify PHAs with notably effective or ineffective FSS programs (or data reporting) and learn from their experiences. Ultimately, HUD’s lack of complete data on FSS participants limits the usefulness of analysis reported to Congress.

In part due to the data’s weaknesses, HUD also has not assessed the data related to self-sufficiency outcomes that it requires FSS and ROSS- SC grant recipients to report through the Excel-based reporting tool for each program as a whole. According to HUD officials, HUD field office staff review the outcomes data that individual PHAs report, but HUD had no process in place for assessing the outcomes reported through this tool programwide. Standards for internal control emphasize the need for federal agencies to collect reliable information with which to manage their programs and to review the integrity of performance measures. HUD officials with responsibility for these programs told us that headquarters staff did not use the reporting tools to assess the effectiveness of the FSS or ROSS SC programs because a system had not been developed to do such an assessment and because the data submitted were sometimes incomplete, not comparable, and unreliable. HUD officials also stated that PHAs likely vary in their interpretations of what to report because HUD never developed program-specific guidance. In 2013, in response to our review, HUD had its contractor (which collects data PHAs report using the tool) aggregate the outcomes data reported for the PH FSS and ROSS SC programs for fiscal years 2008 through 2011. However, HUD staff found that the results did not appear to be accurate or reliable.

While HUD’s effort to aggregate the performance data that they require grant recipients to report was a step in the right direction, the agency lacks a strategy for better ensuring that the outcomes data it collects are reliable and permit comparison across PHAs. Without a plan for helping to ensure that the outcome data FSS and ROSS SC grant recipients report are comparable and reliable, HUD will be unable to fully use the data it requires PHAs to report.

HUD has not assessed the effectiveness of the MTW program using the information that it requires MTW agencies to submit in their annual performance reports on the impact of their MTW activities, including activities related to increasing resident self-sufficiency. MTW agencies generally have devised their own metrics for activities and reporting performance information, so the usefulness of this information for assessing programwide results is limited. That is, because the data are not consistent across agencies, they cannot be used to assess the performance of similar activities across MTW agencies. Additionally, in some cases the information is not outcome-oriented and thus cannot be effectively used to assess performance.

We previously recommended that HUD (1) improve its guidance to MTW agencies on providing information in their performance reports by requiring that such information be quantifiable and outcome-oriented to the extent possible; and (2) develop and implement a plan for quantitatively assessing the program as a whole, including the identification of standard performance data needed. HUD generally agreed with our recommendations, and in May 2013 OMB approved the revised guidance. Additionally, according to HUD, the agency continues to seek funding for a full evaluation that will better analyze the MTW information that is already collected and assess the effectiveness of the program, including self-sufficiency activities.

We recommended in July 1998 that HUD develop consistent national, outcome-based measures for community and supportive services at HOPE VI sites. HUD has used its HOPE VI reporting system to collect data from grantees on the major types of community and supportive services they provide and outcomes achieved by some of these services. See GAO, HOPE VI: Progress and Problems in Revitalizing Distressed Public Housing, GAO/RCED-98-187 (Washington, D.C.: July 20, 1998); and Public Housing: HOPE VI Resident Issues and Changes in Neighborhoods Surrounding Grant Sites, GAO-04-109 (Washington, D.C.: Nov. 21, 2003). many residents enrolled in services related to education and job training, and from one-third to half of those who enrolled completed the activity (see table 6). However, these outputs may not be directly attributable to the community and supportive services program and are self-reported by HOPE VI agencies. HUD officials with responsibility for collecting these data noted that it would be excessively time and resource-intensive to verify the accuracy of these data. HUD staff also noted that if a HOPE VI agency’s community and supportive services data appeared to be inconsistent with past trends, HUD staff would follow up with the agency.

Where HUD had data, the data suggest positive changes in income and employment for families that participated in the two FSS programs, but these results are not conclusive. Data on program completion were missing for nearly half of the records we evaluated. More specifically, the FSS Addendum data in PIC lacked exit, completion, or extension data on 6,819 (46 percent) of 14,690 families that started either of the programs in 2006. Of these families, the “family report” section of PIC indicated that 1,671 (25 percent) had left subsidized housing. Thus, after comparing both the FSS Addendum and the main “family report” data, we found that HUD’s systems lacked information on whether 5,148—35 percent—of the families that started either of the FSS programs in 2006 exited the FSS programs or subsidized housing.

Of the subset of families for which exit, completion, or extension data were available in the FSS Addendum of PIC (that is, 54 percent of program participants), about 60 percent exited the program without completing it. In contrast, 25 percent of participants completed the program in 5 years or less (see table 7).

Considering only those families for which HUD had complete data, we observed positive changes in income and employment. We observed these changes for the families that started an FSS program in 2006 and completed it in 5 years or less.based on the 25 percent of families for which HUD data indicated completed the programs in 5 years or less. But, these findings do not take into account other factors that may have affected the families’ progress towards self-sufficiency. Specifically, of the 1,937 families that started either of the two FSS programs in fiscal year 2006 and completed them within 5 years, HUD’s data suggest positive changes in income and employment (see table 8). For example, median income increased from about $17,000 per year to about $25,000 per year. Families with total incomes of $35,000 or more when they started the program (ninetieth percentile) experienced income gains of 34 percent, compared with income gains of 106 percent for families that had total incomes of about $3,000 or less (tenth percentile). These families also experienced positive changes in employment. For example, full-time employment among these program graduates increased 76 percent. The proportion of these graduates who were working part-time decreased 30 percent and the proportion not employed decreased 61 percent.

HUD Encourages PHAs to Coordinate with Local Service Providers

HUD’s Strategic Plan and Guidance Encourage Coordination, and Some Programs Require It

HUD aims to improve self-sufficiency among residents of HUD-assisted rental housing by encouraging coordination between PHAs and agencies offering services that promote work and self-sufficiency. HUD’s 2010- 2015 strategic plan presents strategies for increasing resident self- sufficiency, which include coordination with federal, state, and local programs to increase access to job training, career services, and work support.

Consistent with this strategy, HUD formed a partnership with the Department of Labor (Labor) to improve residents’ access to Labor programs and services. Residents of HUD-assisted rental housing may be eligible for a variety of Labor programs and services, including those funded through the Workforce Investment Act (WIA). These services are delivered locally through American Job Centers, also known as one-stop centers, which provide education and career training, job search tools, and assistance with developing resumes and interview skills. Since 2009, HUD officials have held meetings with Labor’s Employment and Training Administration to share information about their respective programs and seek opportunities to collaborate. One product of these meetings has been a joint toolkit for PHAs and local workforce agencies intended to improve HUD-assisted residents’ access to employment services offered through the workforce agencies’ one-stop centers, intended for release in 2013. The draft toolkit offers PHAs and workforce agencies a baseline understanding of each other’s functions, examples and lessons learned from successful partnerships currently in place, information on available online resources, and sample Memorandums of Understanding (MOU), or partnership agreements.

HUD guidance also encourages PHAs to coordinate with local entities such as social service agencies and job training providers to increase residents’ self-sufficiency. For example, HUD issued a notice to all PHAs in 2011 promoting partnerships with such agencies. The notice describes the benefits of collaboration, provides examples of possible partnerships and strategies for partnership development, and includes model MOUs. The notice states that HUD encourages such partnerships and recognizes that they benefit both PHAs and the households they serve. In addition, HUD’s guidance to PHAs implementing the self- sufficiency programs we reviewed emphasizes the value of coordinating with local agencies. For example, HUD developed a resource website for HOPE VI grantees that provides information on interagency coordination and examples of PHA practices. Also, HUD’s training materials for ROSS SC grantees include resources that PHAs can use to form partnerships, such as a sample partner outreach letter. While this guidance is directed to HOPE VI and ROSS SC grantees, it is publicly available on HUD’s website and can be accessed by all PHAs.

HUD’s program requirements for four of the self-sufficiency programs we reviewed call for PHAs to coordinate with local service providers (see table 9). For example, grant funds for the two FSS programs and the ROSS SC program cannot be used to fund direct services. Instead, they must be used to hire coordinators who refer residents to local service providers, making coordination a key component of these programs’ designs. Furthermore, applicants for ROSS SC and HOPE VI grants must demonstrate financial or in-kind support from partner organizations or agencies. Additionally, supportive services funded through HOPE VI grants must be coordinated with other service providers, including state and local programs. Finally, all four programs require participating PHAs to form coordinating committees to help ensure that residents are linked to the services that they need. For example, PHAs that receive either of the two FSS grants must establish a Program Coordinating Committee, which is charged with securing commitments of public or private resources for the operation of self-sufficiency programs. PHAs are encouraged to include local service providers, including welfare and workforce agencies, on the committees.

Selected PHAs worked with local service providers to implement HUD’s self-sufficiency programs. We interviewed officials from a sample of five PHAs that had implemented one or more of HUD’s self-sufficiency programs, as well as officials from workforce and welfare agencies in those locations.local agencies that provided a broad range of services, including job training, mental health services, child care, transportation, food assistance, and homeownership counseling. Some of these local agencies received federal funding—for example, workforce agencies that administered WIA programs. PHA and local agency officials with whom we met stated that coordination efforts sometimes were formally established through MOUs, contracts, or regular meetings but added that most coordination efforts were informal. This informal coordination included referrals to each other’s services or the sharing of information or updates on new programs and services.

All of the PHAs we interviewed connected residents to Officials from the five PHAs with whom we met identified few barriers to coordinating with local service providers and found current HUD guidance related to such coordination sufficient. Officials said that, despite resource constraints, they generally were able to obtain the services their residents needed from local agencies. PHA officials also stated that local agencies, including federally funded agencies, were receptive to their coordination efforts. However, while officials from two PHAs stated that they had strong relationships with their local workforce agencies, officials from the other three PHAs noted that these agencies’ one-stop centers could be intimidating for residents and might not always be able to provide residents with appropriate services. Officials from workforce agencies we interviewed generally agreed, noting that residents might not have the education or work experience needed for some of the training opportunities that the centers offered. For example, a workforce agency official told us that individuals interested in job training in certain fields, such as trucking or nursing, must take tests to demonstrate required levels of academic readiness to participate. He said that many assisted housing residents were not prepared to pass these tests and might first need remedial adult education. HUD officials with responsibility for administering the self-sufficiency programs we reviewed and Labor officials with responsibilities related to workforce agencies told us that the joint HUD-Labor toolkit described above was developed in part to improve resident access to services offered through the one-stop centers.

In general, awareness of federal efforts to improve coordination was mixed among the PHAs, workforce, and welfare agencies with which we met. For example, officials from two of the PHAs were familiar with the HUD notice on promoting partnerships, but officials from the other three PHAs were not. Additionally, officials from two of the PHAs and all five workforce agencies were not aware of a 2009 joint HUD-Labor letter on collaboration. Still, officials from these agencies generally did not express a need for further federal guidance or assistance in facilitating relationships between PHAs and local agencies.

Conclusions

HUD’s housing assistance programs serve millions of low-income residents. Statutory and programmatic requirements for HUD and PHAs also direct the organizations to undertake activities that would help these households increase their economic self-sufficiency. In particular, HUD has five programs that, in whole or in part, fund activities intended to help families become self-sufficient. However, HUD faces two major impediments to effectively operating these programs and achieving their goals. First, it does not have reliable data on participation in self- sufficiency activities across PHAs. Second, even though it has complete data for some PHAs, it generally does not use these data to review the progress participants may have made in, for example, finding a job or completing more education.

In relation to participation data for the programs we reviewed, HUD has missed opportunities to help ensure that FSS participation data are complete. For example, in the two FSS programs significant gaps exist in participant entry and completion dates or reasons for leaving. These gaps are detrimental for several reasons, including their effects on grant funding and congressional reporting. Specifically, HUD uses resident participation data as a factor in making grant funding decisions and has reported this information in Congressional Budget Justifications. Moreover, without such data HUD cannot identify PHAs that have low or high completion rates. Federal internal control standards state that transactions should be promptly recorded to maintain their relevance and value to management and HUD’s own reporting guidance also directs grant recipients to record program start dates, exit dates, reasons for exiting the program prior to completion, and completion dates. HUD has recognized the importance of having reliable participation data in recent years, emphasizing its importance to grantees in funding notices. But HUD could further work with agencies to correct participation data during the grant award process, or analyze the data outside of this process to help ensure that all required data are complete. By doing so, HUD will improve the accuracy of these data, improve its ability to assess FSS grantee activities and thus make better-informed decisions about funding them, and provide Congress a more complete view of program performance.

Similarly, ROSS SC is designed to provide grant recipients with funds to hire staff to help residents progress towards economic independence and self-sufficiency. Since 2008, HUD has required grant recipients to report participation data. However, HUD has not provided grant recipients with program-specific reporting guidance; thus, the reported data vary and cannot be easily or reliably used for assessing programwide participation. Federal internal control standards state that program managers need operational data to determine whether they are meeting their agencies’ strategic plans and meeting their goals for the effective and efficient use of resources. By developing program-specific reporting guidance, HUD could help ensure the collection of accurate participation data, and establish data sets that can be used to assess participation for the ROSS SC program as a whole.

In relation to outcome data for the programs we reviewed, HUD has not optimized its use of the information it requires PHAs to collect. We acknowledge that determining the outcomes of self-sufficiency activities is difficult and requires rigorous analyses. It is also difficult to isolate the impact of such activities from other factors that may influence participant outcomes. But HUD could do more to put itself in a position to look across a program to review participant accomplishments. While HUD has collected some indicators for the FSS and ROSS SC programs (such as information on hours worked and receipt of welfare), it lacks a strategy for using the data it collects, whether through PIC or its Excel-based reporting tool. And, as with participation data, PHAs have not consistently reported such information, a condition exacerbated by the lack of program-specific reporting guidance. As stated above, internal control standards underline the importance not only of collecting but also using information to achieve programmatic goals—helping families increase self-sufficiency. Additionally, GPRAMA emphasizes the need for information on the effectiveness of federal programs to help improve congressional decision making. A strategy for using these data could inform overall management review, congressional oversight, and planning for these programs. For instance, using such data could help HUD identify from which PHAs to draw lessons to help improve HUD management of the grant programs as well as PHA management of self- sufficiency-related activities.

Recommendations for Executive Action

To better inform Congress and improve what is known about residents’ participation in key grant programs designed to facilitate resident self- sufficiency, and their progress towards self-sufficiency, the Secretary of the Department of Housing and Urban Development should develop and implement a process to better ensure that data on FSS participant grants are complete; such a process should include steps for identifying missing data, identifying the reasons for missing data, and taking steps to help ensure data are complete; a process to better ensure that PHAs awarded ROSS SC grants annually report required participation and outcome data that are comparable among grant recipients; this process should include the issuance of program-specific reporting guidance; a strategy for regularly analyzing FSS participation and outcome data; such a strategy could include identification of PHAs from which lessons could be learned and PHAs that may need assistance improving completion rates or outcomes; and a strategy for regularly analyzing ROSS SC participation and outcome data; such a strategy could include identification of PHAs from which lessons could be learned and PHAs that may need assistance improving participation rates or outcomes.

Agency Comments and Our Evaluation

We provided a draft of this report to HUD, Labor, Treasury, and HHS. HUD provided written comments, which are reprinted in appendix II. Labor and HHS provided technical comments, which we incorporated as appropriate, and Treasury did not provide comments. HUD agreed with three of our recommendations, and pointed to actions it intends to take to implement them. While agreeing with these recommendations, HUD noted some concerns about our recommendation that it develop and implement a strategy for regularly analyzing FSS participation and outcome data, and disagreed with our similar recommendation for the ROSS SC program.

While agreeing with our recommendation that it develop and implement a strategy for regularly analyzing FSS participation and outcome data, HUD stated that the data captured in PIC for this program are not designed for rigorous statistical analysis. However, according to HUD guidance, PIC can be used to create reports, analyze programs, monitor PHAs, and provide information about residents in HUD-subsidized housing. In addition, if participating PHAs annually updated data, as required, in the FSS Addendum, PIC would include data on sources of assistance received by the family; whether the head of the household worked full- time, part-time, or was unemployed; and the highest grade of education for the head of the household. By implementing our first recommendation—that HUD develop and implement a process to better ensure that data on FSS participants are complete—the completeness and therefore reliability of outcome data should improve. Consequently, usefulness of analysis of such data would also be improved.

HUD disagreed that it should develop a strategy for regularly analyzing ROSS SC participation and outcome data. In doing so, the agency noted the small size of the program, the difficulty of analyzing outcomes, and that the data it collects are administrative in nature and not intended to serve as the basis for analysis. However, HUD’s training materials on the Excel-based tool that it uses to collect participation and outcomes data for the ROSS-SC program state that the tool is intended to be used to manage, monitor, and evaluate program services. These training materials and HUD staff with whom we met also indicated that the tool, when populated by participating PHAs, contains data on outputs and outcomes. And, as noted above, HUD agreed that its reporting guidance to ROSS SC grantees should be improved to help make data more meaningful. HUD’s disagreement with our recommendation to analyze the information collected does not accord with its requirement for PHAs to submit operational data or with its assessment that data quality ought to be improved. Consistent with internal controls for the federal government, which apply to programs of all sizes, regular analysis of ROSS SC operational data would help HUD determine whether it was meeting goals for the effective and efficient use of resources. It can be difficult to isolate and definitively assess program outcomes. But program data can help identify patterns and anomalies in program operations, which can help agencies discover ways to achieve more value for the taxpayer’s money. Consequently, we continue to recommend that HUD develop and implement a strategy for regularly analyzing ROSS SC participation and program data.

In its technical comments, HUD also raised concerns about the accuracy of dollar amounts reported and the characterization of outputs related to the community and supportive services component of HOPE VI. First, HUD questioned the accuracy of the amounts awarded to PHAs through the FSS and ROSS SC programs and the amounts HOPE VI agencies spent on community and supportive services. Because trends in nominal spending may reflect changes in both price and quantity, we chose to present inflation-adjusted values by removing the general effects of inflation using a price index. Specifically, we adjusted these figures to fiscal year 2013 dollars using the fiscal year chain-weighted Gross Domestic Product price index. Second, for the HOPE VI program, HUD stated that without a costly and time-intensive experimental research design, it would not be possible to know whether the outputs residents’ experienced were directly attributable to the program. We acknowledged the difficulties of isolating program outcomes in this report, and have not made any recommendations related to HOPE VI. Finally, during the course of our review, OMB approved revised reporting requirements for the MTW program, which are intended to establish standard metrics for activities related to self-sufficiency, among other things. We revised the report to recognize these changes. HUD provided additional technical comments, which we incorporated as appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of Housing and Urban Development and other interested committees. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-8678 or sciremj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Appendix I: Scope and Methodology

Our objectives were to (1) discuss what is known about the costs of and residents’ participation in Department of Housing and Urban Development (HUD) grant programs that encourage work and self-sufficiency, (2) determine what is known about the effect on residents of HUD grant programs to promote self-sufficiency, and (3) describe steps HUD has taken to coordinate with other federal agencies and increase residents’ access to non-HUD programs that encourage work and self-sufficiency.

To discuss what is known about the costs of and residents’ participation in HUD programs that encourage self-sufficiency we reviewed documentation of HUD’s programs and determined whether and how they defined and addressed resident self-sufficiency. Because HUD did not have an agencywide definition of self-sufficiency, and included 20 programs in its list of programs that contribute to its subgoal of increasing economic security and self-sufficiency, we established criteria to narrow the scope of programs to include in our review. First, we limited our review to grant programs that were intended to encourage work and self- sufficiency among residents of HUD-assisted rental housing. Second, we included grant programs that did not have self-sufficiency as their primary focus but rather as a secondary program goal. We sought to include programs that were actively awarding grants, or for which grants were still open. Based on these criteria, we identified the following programs:

Public Housing Family Self-Sufficiency (PH FSS)

Housing Choice Voucher Family Self-Sufficiency (HCV FSS)

Resident Opportunity and Self-Sufficiency Service Coordinators (ROSS SC)

Moving to Work (MTW)

HOPE VI HUD officials agreed that we had identified the key grant programs that encourage work and self-sufficiency. To describe what is known about the programs’ costs, we met with program staff and obtained documentation of the total grant amounts awarded for specific periods, as follows:

For the Family Self-Sufficiency (FSS) programs, we obtained grant award amounts for fiscal years 2006 through 2011 and analyzed data from HUD’s Public and Indian Housing Information Center (PIC) system on escrow account disbursements from fiscal years 2006 through 2011. We adjusted all dollar amounts to fiscal year 2013 dollars using the chain-weighted Gross Domestic Product price index.

For ROSS SC, we obtained grant award amounts for fiscal years 2008 through 2011, because 2008 was the first year of the current version of the program.

For the MTW program, we met with program staff and staff from HUD’s Grants Management Office to determine whether HUD collected information on MTW agencies’ expenditures for activities related to self-sufficiency.

For the HOPE VI program, we reviewed and summarized the fiscal year 2006-2010 Notice of Funding Availability (NOFA) requirements related to community and supportive services expenditures and the amount of HOPE VI revitalization grant funds that HOPE VI agencies expended on community and supportive services in that period.

To analyze residents’ participation in each of the five programs, we met with HUD staff to determine how participation data were collected and summarized any readily available information.

For the two FSS programs, we analyzed participation data from PIC for fiscal years 2006 through 2011. Specifically, we analyzed data that public housing agencies (PHA) entered in the FSS Addendum, a PIC subcomponent. As a part of this analysis, we identified the extent to which program start dates, end dates, and annual updates for participating households were entered. We interviewed HUD officials with responsibility for administering both FSS programs and the PIC system about the completeness of the data and HUD’s use of the information. We evaluated the information available on families’ participation in the two FSS programs in relation to internal control standards for the federal government as well as HUD’s own guidance. Based on interviews with HUD officials and our review of the completeness of data fields in the FSS Addendum component of PIC, we determined that the available data were not sufficiently reliable to provide an accurate count of residents’ participation in FSS for fiscal years 2006 through 2011. We summarized data that HUD posted on its website as a part of the NOFA process for fiscal years 2009 through 2012.

For the ROSS SC program, we obtained and reviewed aggregated data on program participation that a HUD contractor created in 2013. We evaluated the information available on residents’ participation in ROSS SC in relation to internal control standards for the federal government. We determined that the aggregated participation data were not sufficiently reliable to provide an accurate count of residents’ participation in ROSS SC for fiscal years 2008 through 2011.

For the HOPE VI program, we summarized data from the HOPE VI Quarterly Reporting System on the total number of nondisabled residents between the ages of 18 and 64 who had received services through the community and supportive services component of HOPE VI from the program’s inception through the end of 2012.

For the MTW program, we interviewed program administrators to determine whether and how they used information from participating agencies’ Annual MTW Reports or MTW-PIC to determine the number of MTW households participating in MTW and activities related to self- sufficiency. We analyzed the FSS Addendums associated with MTW- PIC in an effort to determine the number of MTW families that had participated in either of the two FSS programs. We determined, based on interviews with HUD officials and our analysis of FSS Addendum data for MTW agencies, that these data were not sufficiently reliable prior to 2011. We summarized available data on MTW families’ participation in the two FSS programs in 2011.

To determine what is known about the effect on residents of HUD’s grant programs that encourage work and self-sufficiency, we examined whether and how HUD collected information that could indicate progress toward self-sufficiency, such as information on income and employment, for each program. We reviewed available data dictionaries and guidance HUD provided to PHAs required to report this information. We determined that for the five programs we reviewed, HUD used PIC, MTW-PIC, Excel- based reporting tools, written reports, and other reporting systems to collect this information. We interviewed HUD staff about their use of the self-sufficiency indicators they require PHAs to report, the reliability of the data collected, and program-specific guidance. We evaluated HUD’s processes for aggregating self-sufficiency-related outcomes data in relation to our standards for internal control. For the two FSS programs, we analyzed PIC data on the 14,690 families that HUD’s data indicated started the programs in 2006. Based on PIC data for fiscal years 2006 through 2012, we identified the number of families that completed the two FSS programs, exited the programs without completing them, and received an extension to continue the program past 2011, and for which HUD’s system lacked either program exit, completion, or extension information. For the subset of families that HUD’s data indicated had completed the two FSS programs in 5 years or less (1,937 households), we analyzed changes in median and mean income, income for those at the tenth and ninetieth percentiles, and employment experiences. We adjusted all dollar amounts to fiscal year 2013 dollars using the chain- weighted Gross Domestic Product price index. This sample represented about 25 percent of the families that started in 2006 and for which exit, completion, or extension data were available. In addition, this analysis does not control for other factors that may have affected participants’ progress towards self-sufficiency. We excluded any households that were receiving rental assistance from an MTW agency because HUD officials indicated that their FSS participation data were not as reliable, particularly before 2011.

Through interviews and a literature search, we identified several studies of the two FSS programs, HOPE VI, and MTW. We reviewed these studies to identify information on the programs’ impact on resident self- sufficiency. We determined that these reports were methodologically sound and reliable for our purposes. We did not identify any studies of the ROSS SC program.

To describe the steps HUD has taken to coordinate with other federal agencies and increase residents’ access to non-HUD programs, we reviewed HUD regulations, policies, and guidance related to coordination. We also interviewed officials from HUD, the Department of Health and Human Services, and the Department of Labor, and reviewed materials related to interagency coordination provided by these officials. To better understand how PHAs connect residents with non-HUD programs, we interviewed staff from PHAs administering each of the self-sufficiency programs included in our review. To select PHAs, we compiled lists of PHAs that received grant funding (or permission to participate in MTW) between 2004 and 2011. We then randomly selected a PHA to interview for each program and adjusted our initial sample to ensure that the five selected PHAs varied in terms of size and region. Four of the selected PHAs implement more than one of the programs in our review. The selected PHAs were:

Boulder County Housing Authority (Boulder, Colorado)

Jersey City Housing Authority (Jersey City, New Jersey)

Kingsport Housing and Redevelopment Authority (Kingsport, Tennessee)

Louisville Metro Housing Authority (Louisville, Kentucky)

Washington County Housing and Redevelopment Authority (Woodbury, Minnesota)

For each of these locations, we also interviewed officials from the local agencies that administered Temporary Assistance for Needy Families and the Workforce Investment Act programs.

We conducted this performance audit from August 2012 through July 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on the audit objectives.

Appendix II: Comments from the Department of Housing and Urban Development

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Paul Schmidt (Assistant Director), Stephen Brown, Emily Chalmers, Geoff King, John McGrail, Marc Molino, Lisa Moore, Michael Pahr, Barbara Roesmann, Beverly Ross, and Andrew Stavisky made key contributions to this report.