Rental Housing Assistance: HUD Data on Self-Sufficiency Programs Should Be Improved
Published: 2013-07-09T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: HUD reported in 2011 that nearly 8.5 million lower-income families paid more than half their monthly income for rent, lived in substandard housing, or both. As the number of those needing assistance is greater than existing federal programs can serve, if families were able to increase their income and no longer require housing assistance, spaces could become available for other needy families. HUD offers several competitive grants that PHAs can use to hire staff who link residents to services or implement programs that encourage self-sufficiency. GAO was asked to examine the effectiveness of HUD's efforts to promote self-sufficiency among residents. Among its objectives, this report describes (1) costs and resident participation in HUD grant programs for PHAs that encourage work and self-sufficiency and (2) available information on the programs' effects on residents.
GAO reviewed HUD's goals for encouraging self-sufficiency, program descriptions, and regulations; analyzed grant award data for fiscal years 2006-2011 and available outcome information; and interviewed HUD and PHA officials.
section_title: What GAO Found
paragraphs: The Department of Housing and Urban Development (HUD) funds five key grant programs that encourage resident self-sufficiency. In fiscal year 2011, HUD awarded $113 million to the Housing Choice Voucher Family Self-Sufficiency (FSS), Public Housing FSS, and Resident Opportunity and Self-Sufficiency Service Coordinators (ROSS SC) programs. Public housing agencies (PHA) with HOPE VI grants or designated as Moving to Work (MTW) agencies spent a portion of their funds on activities that encourage self-sufficiency, but the amounts MTW agencies spent are not known for the program as a whole. Additionally, data on resident participation in the five programs were limited. The number of families that participated in the FSS programs and ROSS SC cannot be reliably assessed due to missing start dates, end dates, and annual updates, and a lack of reporting guidance. HOPE VI data on residents' participation does not include information on the elderly or persons with disabilities. Programwide MTW data on participation generally were unavailable. Internal control standards for the federal government state that program managers need operational data to determine whether they are meeting goals for accountability (effective and efficient use of resources). Without complete participation data, HUD lacks key information to effectively manage and evaluate its programs and Congress lacks data needed to oversee the programs.
HUD lacks a strategy for using data it requires of PHAs to expand what is known about outcomes in four of the programs. HUD has performed limited analysis of the data related to self-sufficiency outcomes for both types of FSS grants reported into its information systems. HUD has not analyzed similar data reported for ROSS SC and MTW activities. However, for HOPE VI HUD collects consistent, outcome-based measures for participation in self-sufficiency activities and uses the data to track residents' progress towards self-sufficiency. Internal control standards underline the importance not only of collecting but also using information to achieve programmatic goals. Also, the GPRA Modernization Act of 2010 (GPRAMA) emphasizes the need for information on the effectiveness of federal programs to improve congressional decision making. A strategy for using these data could inform overall management review, congressional oversight, and planning for these programs. Using such data could help HUD identify from which PHAs to draw lessons to help improve HUD management of the programs as well as PHA management of self-sufficiency-related activities. GAO's analysis of available data on residents who participated in the FSS programs suggests positive changes for those who completed the programs, but the results are not conclusive because data indicating whether a family exited FSS or subsidized housing were missing for 35 percent of families that started an FSS program in 2006.
section_title: What GAO Recommends
paragraphs: For three of its self-sufficiency programs, HUD should develop processes and program-specific reporting guidance to better ensure required data on participation and outcomes are complete. HUD agreed with three recommendations but disagreed that it should analyze data for the ROSS SC program. GAO believes that analysis of program data is critical for assessing outcomes.

section_title: Letter
paragraphs: The Department of Housing and Urban Development (HUD) provided  more than $33 billion in rental assistance in fiscal year 2012 to millions of  low-income households through a variety of programs. These programs— primarily housing choice vouchers, project-based rental assistance, and  public housing—serve an estimated 5.4 million low-income households.  However, HUD reported in 2011 that nearly 8.5 million lower-income  families paid more than half their monthly income for rent, lived in  severely substandard housing, or both. As the number of those needing  assistance is greater than can be served by existing federal programs, if  families were able to increase their income and move out of assisted  housing, spaces could become available to assist other needy families.
The Quality Housing and Work Responsibility Act of 1998 (QHWRA)  requires public housing agencies (PHA), which administer public housing  and voucher programs, to establish policies and practices that encourage  all but elderly residents and residents with disabilities to work and  become more economically self-sufficient.requires PHAs to establish minimum rent policies (with exceptions for  hardship circumstances).  For the public housing program, it also requires  nonelderly, nondisabled residents to work or participate in community  service or a self-sufficiency program for 8 hours a month; requires PHAs  to establish rent policies that encourage and reward employment and  economic self-sufficiency; and allows PHAs to phase-in rent increases 1  year after public housing residents find employment. To encourage  resident self-sufficiency, PHAs can apply for several HUD grants. Some  of these grants provide funds for PHA staff to link residents to the  services that they need to become increasingly self-sufficient. Other grant  programs enable selected PHAs to establish their own policies—such as  rent policies—that encourage work and self-sufficiency.
You asked us to review the effectiveness of HUD programs that  encourage resident self-sufficiency. This report (1) discusses what is  known about the costs of and residents’ participation in HUD grant  programs that encourage work and self-sufficiency, (2) determines what  is known about the effect on residents of HUD’s grant programs, and (3)  describes steps HUD has taken to coordinate with other federal agencies  and increase residents’ access to non-HUD programs that encourage  work and self-sufficiency.
To describe what is known about the costs of HUD grant programs that  encourage self-sufficiency, we reviewed documentation of HUD’s  programs and determined whether and how they defined and addressed  resident self-sufficiency. Because HUD did not have an agencywide  definition of self-sufficiency, we established criteria for what constitutes a  self-sufficiency program, and identified the Housing Choice Voucher  Family Self-Sufficiency (FSS), Public Housing Family Self-Sufficiency,  Resident Opportunity and Self-Sufficiency Service Coordinators (ROSS  SC), Moving to Work (MTW), and HOPE VI programs. HUD officials  agreed that we had identified the key grant programs that encourage  work and self-sufficiency. We then interviewed HUD officials about costs  associated with the five programs, and reviewed documentation of grant  awards for the programs. To assess resident participation, we determined  how HUD collects program participation data and analyzed data from  HUD’s Public and Indian Housing Information Center (PIC) system and  the HOPE VI Quarterly Reporting System. To determine what is known  about the effects of the five programs on residents’ self-sufficiency, we  determined how HUD collects data on indicators of self-sufficiency for  each of the five programs and interviewed HUD officials about how they  use the data. For the Family Self-Sufficiency (FSS) programs, we  analyzed changes in indicators of self-sufficiency for a cohort of  participants who completed the programs. We also summarized available  research on self-sufficiency-related outcomes for four programs. To  describe how HUD has coordinated with other federal agencies to  increase residents’ access to non-HUD programs, we reviewed HUD  regulations and policies related to such coordination and interviewed  officials from PHAs administering each HUD program in our review. For  all three objectives, we interviewed HUD officials and officials from a  sample of five PHAs and research organizations. We determined, based  on interviews with HUD officials and our own analyses, that the fiscal  years 2006 through 2011 data were not sufficiently complete to enable us  to reliably analyze the total number of families participating over time and  their self-sufficiency-related outcomes. Further, for those families that  began either of the FSS programs in 2006, we described outcomes for  those families for which HUD’s data indicated that they successfully  completed the program. HUD’s data for approximately one-third of these  families did not indicate whether the family successfully completed the  program or otherwise exited the public housing or voucher programs.  Additionally, we determined based on interviews with HUD officials and  our own analysis that HUD data on the extent to which MTW families  participated in FSS was not reliable prior to 2011. Appendix I provides  further details on our scope and methodology.
We conducted this performance audit from August 2012 through July  2013 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on the audit objectives.
subsections: section_title: Background
paragraphs: HUD housing serves populations that include persons in a position to  increase their self-sufficiency and those who need long-term support (for  example, the elderly and persons with severe disabilities). To assist those  who can improve their self-sufficiency, HUD permits PHAs to allocate  space in its public housing to offer training programs and information  sessions. It also awards grants to PHAs to encourage them to work with  local social service providers in offering education, job training, and other  supportive services to residents of public housing and recipients of  vouchers. The purpose of these programs is to increase the earned  incomes of residents and reduce their reliance on cash assistance and  housing assistance.
subsections: section_title: HUD-Funded Self- Sufficiency Programs
paragraphs: For this report, we examined five HUD programs that fund self-sufficiency  efforts in whole or in part. Three grant programs only award grants to  support self-sufficiency activities: HCV FSS, PH FSS, and ROSS SC.  FSS was authorized in 1990 to help families receiving vouchers or in  public housing become self-sufficient through referrals to education,  training, and other supportive services and case management. PHAs  use grant funds to pay program coordinators who link residents to the  supportive services they need to achieve self-sufficiency. Families in  either FSS program sign a contract with their PHA, requiring that all family  members become independent of cash welfare assistance and the head  of the family seek and maintain employment. Both programs feature  case management, referrals to supportive services, and an escrow  account that accumulates balances, or savings, for the tenant based on  increases in tenant contributions toward rent. The ROSS SC program  provides funding to hire service coordinators to assess the needs of  public housing residents and coordinate available resources in the  community to meet those needs.
For the FSS programs, the escrow accounts are incentives to increase  work effort and earnings. Specifically, when participants have to pay a  higher rent after their earned income increases, the PHA calculates an  escrow credit that is deposited each month into an interest-bearing  account (see fig. 1). Families that successfully complete their contract for  either FSS program receive their accrued escrow funds. In general, in  order to complete an FSS contract, the family head must certify that no  member of the family has received welfare for at least one year, and the  family head must be employed.
HUD has two other programs—MTW and HOPE VI Revitalization Grants  (HOPE VI)—that allow participating PHAs to determine how they will  encourage self-sufficiency. The purposes of HUD’s MTW demonstration  program include providing PHAs the flexibility to design and test policies  that give incentives to families with children to become economically self- sufficient. For example, PHAs in MTW can alter eligibility and rent  policies. Through the HOPE VI program, participating PHAs (HOPE VI  agencies) can use HOPE VI grants to demolish, rehabilitate, or replace  severely distressed public housing and also provide community and  supportive services to help residents achieve self-sufficiency. HOPE VI  agencies can spend up to 15 percent of their revitalization grant funds for  community and supportive services activities, which HUD defines as any  activity designed to promote upward mobility, self-sufficiency, and  improved quality of life for residents of the public housing project  involved.
section_title: HUD Indicators of Self-Sufficiency
paragraphs: For the two FSS programs, HUD regulations state that self-sufficiency  means that a family is no longer receiving housing assistance or welfare  assistance. However, the regulations also indicate that while achieving  self-sufficiency is an FSS objective, no longer receiving housing  assistance is not a requirement for completing the program and receiving  escrow funds. HUD does not define self-sufficiency for the other three  programs we reviewed. Thus, HUD does not have a uniform measure of  self-sufficiency across the programs. HUD officials with responsibility for  administering all five programs noted that a single definition of self- sufficiency would not be useful, particularly because levels of self- sufficiency could differ for different groups. However, officials noted that  the concept is commonly understood to indicate that a family does not  rely on government programs that are intended to address poverty.
HUD, HUD Strategic Plan Fiscal Year 2010-2015 (Washington, D.C.: May 2010). disabilities, services should help improve living conditions and enable  residents to age in-place. For the HOPE VI program, indicators of  progress towards self-sufficiency include the number of residents who  obtained a high school or equivalent education, obtained new  employment, and completed a job training program.
In contrast, MTW agencies determine their own measures of residents’  progress towards self-sufficiency. In a 2012 report on the MTW program,  we reported that HUD had not defined key program terms, including the  statutory purpose of encouraging employment and self-sufficiency, and   We  recommended that HUD issue guidance that clarifies such terms.also noted the limited usefulness of having MTW agencies devise their  own metrics, particularly when they are not outcome-oriented. We  recommended that HUD improve its guidance to MTW agencies by  requiring that they provide performance information that is quantifiable  and outcome-oriented. HUD has taken steps to revise its reporting  guidance, and the Office of Management and Budget (OMB) approved  revised guidance on May 31, 2013.
section_title: Data Collection
paragraphs: HUD requires all PHAs to collect detailed data from their residents. PHAs  must submit these data electronically through the PIC at least annually for  each household that participates in assisted housing programs. According  to HUD guidance, PIC can be used to create reports, analyze programs,  monitor PHAs, and provide information about those that live in HUD- subsidized housing. The data collected include amounts and sources of  income; the amount of rent paid; and the presence of household heads  and members who are elderly or have disabilities. MTW agencies can  submit some tenant-related data into a separate module in PIC called  MTW-PIC, which was created in 2007 to better accommodate some of  the activities allowed under MTW. Most MTW agencies had transitioned  to it by 2008.
PHAs that receive either of the FSS grants, including MTW agencies,  must report additional data about participating households into a section  of the PIC and MTW-PIC systems called the FSS Addendum. The  addendum captures data on each family when it enters either of the two  FSS programs and should be updated annually. Specifically, the  addendum includes fields for PHAs to report whether the head of the  household works full-time, part-time, or is unemployed; the highest grade  of education for the head of the household; and the sources of assistance  received by the family, such as cash or food assistance.
section_title: Participation Information for Self- Sufficiency Programs Was Limited
subsections: section_title: Available Data Indicate That Annual Amounts of Grant Awards in Three Self-Sufficiency Programs Increased
paragraphs: The annual totals for grant awards in the two FSS and ROSS SC  programs have increased in recent years. As shown in table 2, the  amount HUD awarded to PHAs in constant 2013 dollars through the two  FSS programs increased from about $64 million in fiscal year 2006 to  about $78 million in fiscal year 2011, with the majority of funds being  awarded through the HCV FSS program. For the ROSS SC program, the  total amount awarded, in constant 2013 dollars, increased from $30  million in fiscal year 2008 to $35 million in fiscal year 2011.
Escrow payments to households represent additional program  expenditures associated with the two FSS programs. As previously  described, PHAs establish escrow accounts for households that  participate in FSS. PHAs generally disburse the amount in the escrow  account (the excess of any amount owed to the PHA by the family) to the  head of the family when the contract of participation has been  completed. As previously noted, in order to successfully complete an  FSS contract, the head of the family must be employed and no members  may be receiving welfare. According to HUD data adjusted to 2013  dollars, the agency disbursed nearly $82 million to FSS participants  between fiscal years 2006 and 2011 (see table 3).
Expenditures for community and supportive services, adjusted to 2013  dollars, represented 7 percent or less of HOPE VI grant awards during  fiscal years 2006-2010. HUD data indicate that the 29 PHAs that were  awarded HOPE VI revitalization grants between fiscal years 2006 and  2010 had spent about 5 percent of these funds on community and  supportive services as of December 2012 (see table 4). Of the 29 grants,  4 were closed and 25 were open. HOPE VI agencies with open grants  could spend more of their revitalization grant on supportive services, up to  15 percent. HOPE VI agencies also can leverage other sources of  funding for community and supportive services.
The amounts that MTW agencies spend on activities intended to increase  resident self-sufficiency are not known for the program as a whole. MTW  agencies are not required to expend a specific proportion of their HUD  funds on activities that encourage work and self-sufficiency. However,  they must implement activities that address the program’s statutory  purposes (which include encouraging work and self-sufficiency), and  annually submit written reports to HUD with information on each activity  they undertake and its linkage to the program’s statutory purposes. MTW  agencies had been required to annually report to HUD financial data on  the sources and uses of their funds. A HUD official with responsibility for  administering the MTW program stated that the agency had not analyzed  this information and that MTW agencies had reported it using varying  formats. According to HUD, with the implementation of revised MTW  reporting requirements, MTW agencies will be required to report data on  the costs of self-sufficiency activities using standardized metrics.
section_title: Program Participation Not Comprehensively Known Because of Data Limitations
paragraphs: Residents’ participation in the five self-sufficiency programs was not  comprehensively known because the data were not reliable, aggregated  programwide, or collected for all participants. Internal control standards  state that transactions should be promptly recorded to maintain their  relevance and value to management in controlling operations and making  decisions. This applies to the entire life cycle of a transaction or event.  They also state that program managers need operational data to  determine whether they are meeting their agencies’ strategic plans and  meeting their goals for the effective and efficient use of resources.
The total number of families in either of the two FSS programs cannot be  reliably assessed based on available PIC data for fiscal years 2006-2011  because of missing program start dates, exit dates, and annual updates. According to HUD guidance, PHAs that receive FSS grants must update  information about each participating family at least annually. As a part of  these updates, PHAs are supposed to indicate whether the family is  active or has exited the program. For those who exited the program,  PHAs are supposed to indicate whether they completed the program or  stopped participating for other reasons. These data must be reported in  the FSS addendum of PIC or MTW-PIC. NOFAs consistently have cited  PIC as a data source for FSS program participation counts, which HUD  uses in part to determine eligibility for funding. For some years, the  notices also stated that applicants for grant funding could use other  documentation for participation counts, such as a separate HUD form or  MTW documents.
We excluded MTW households from our analysis of FSS participation because available  data were not reliable prior to 2011. these families. For 11 percent of the families that began the FSS  programs in 2006, HUD data do not indicate the families had exited the  programs, although other HUD data indicated that the families no longer  received housing assistance. Similarly, HUD data also indicate that  11,596 families exited the two programs in fiscal year 2011 (some  completing the program and others leaving before completion), but did  not indicate a start date for about one quarter of these families. Because  both FSS programs are intended to be 5-year programs, it is reasonable  to expect that most of the families that began participating in fiscal year  2006 would have exited or completed the program by fiscal year 2011.
HUD began posting FSS participation data from the PIC system online  beginning with the fiscal year 2009 funding announcement for HCV FSS  and the fiscal year 2012 funding announcement for PH FSS. According to  these data, the number of families participating in HCV FSS has  increased in recent years (see table 5).
HUD officials told us that posting participation data online has  emphasized to PHAs the importance of ensuring the data they submit into  PIC are accurate. If a PHA believed the participation number HUD posted  was incorrect and that it would be underfunded or ineligible, that PHA  could submit documentation to confirm a higher number. However, as a  part of the NOFA process, neither HUD nor the PHA were required to  make corrections to the PIC system. Officials from one of the PHAs with  whom we met stated that the enrollment data in PIC generally are  inconsistent with their internal records. The staff stated that they have  worked with HUD to correct PIC, but noted that once some errors were  fixed, new errors emerged. Staff from another PHA, which is also a MTW  agency, stated that their FSS enrollment data do not appear in PIC; thus,  they must create ad hoc reports using their own internal systems.
According to HUD officials with responsibility for implementing the FSS  programs, the FSS records could be incomplete or incorrect for several  reasons. For example, if a participating family left public housing or the  voucher program, the responsible PHA might not update the FSS portion  of PIC to reflect the departure. PHAs are supposed to enter the exit date,  whether the family completed their contract of participation, or one of five  primary reasons for leaving the program without completing it. According  to these officials, no HUD staff have specific responsibility for monitoring  the completeness of FSS participants’ records in PIC. HUD officials also  stated that there are challenges with the PIC system. They were aware of  cases in which PHAs have entered data into PIC, but either the entries  were not saved or they overwrote previously entered data. PHA officials  with whom we met, as well as HUD staff with responsibility for the PIC  system, stated that PHAs cannot readily run reports that show FSS  participation data, a feature which would enable them to identify and  correct missing or incorrect data. While PHAs can run ad hoc reports from  PIC, the functionality of the system is limited, and records older than 18  months cannot be accessed. We previously reported on the weaknesses  associated with HUD’s antiquated technologies. Without complete information on families’ participation in either FSS program (including a  reliable count of program participants, participants’ length of time in the  program, and reasons participants do not complete the programs), HUD  lacks accurate information to make grant funding decisions. Moreover, by  not analyzing the extent to which PHAs have reported required data into  PIC (and MTW-PIC), HUD’s ability to effectively oversee the program is  limited.
Participation data for the ROSS SC program from fiscal years 2008 (the  year the current version of this program started) through 2011 also were  limited, primarily because of the lack of reporting guidance for the  program and the difficulty of aggregating the data. HUD does not require  PHAs to report whether a resident received services through this program  in its information systems; rather, HUD collects ROSS SC participation  data from PHAs through individual Excel-based reporting tools.  According to HUD officials with responsibility for the PIC and MTW-PIC  systems, these systems were intended to collect resident characteristics  and not to enable PHAs to report data on residents’ participation in  specific programs, such as ROSS SC. PHAs must use the tool to apply  for grant funding, inserting projections for the number of residents or  households they intend to serve. Grant recipients then must submit  annual updates, reporting on the number of residents or households  actually assisted. Internal control standards state that transactions should  be promptly recorded to maintain their relevance and value to  management in controlling operations and making decisions. They also  state that program managers need operational data to determine whether  they have been meeting their agencies’ strategic plans and meeting their  goals for the effective and efficient use of resources. As we previously  reported, according to OMB, being able to track and measure specific  program data can help agencies diagnose problems, identify drivers of  future performance, evaluate risk, support collaboration, and inform  follow-up actions. Analyses of patterns and anomalies in program  information also can help agencies discover ways to achieve more value  for the taxpayers’ money.
According to HUD officials, the data that individual PHAs report using the  Excel-based reporting tool cannot be easily or reliably aggregated to  provide a count of resident participation across all PHAs. HUD officials  who manage the ROSS SC program did not attempt to aggregate data  reported through individual reporting tools until 2013 in response to our  review. While HUD provided us with annual program participation data  based on the tools, we determined that the data were not sufficiently  reliable for these purposes. Specifically, we found duplicate records;  differences in the time periods for which PHAs were reporting data; and  different results for the “number of persons receiving services,” the  “number of persons served,” and the sum of counts of persons served by  age category. HUD officials told us that there are no line-level instructions  for reporting these data. Thus, recipients of ROSS SC grants have not  been given any formal guidance on what should be reported into these  fields. Because of these limitations, HUD does not use the reporting tools  to prepare official counts of residents assisted through the ROSS SC  program. Rather, HUD multiplies the number of coordinators funded each  year by 50 (the minimum number of residents who must be assisted  under the terms of the grant). Based on this formula, they can report on  the minimum number of residents who likely were assisted. Using this  approach, HUD estimated that from 6,450 to 7,800 residents received  services through the ROSS SC program annually from fiscal years 2008  through 2011. While this estimate provides information on the minimum  number of residents who likely received assistance, it is not based on a  count of the number of residents who actually received assistance.
HUD officials who manage the ROSS SC program stated that the current  process for collecting data from the reporting tools was supposed to be  temporary, and a planned web-based replacement was never developed.  Additionally, the officials stated that the data PHAs report into the tool  were not always comparable because PHAs have varying interpretations  of what they should report, and HUD never developed program-specific  reporting guidance. Without developing a reliable process for collecting  and analyzing data on the number of residents assisted through the  ROSS SC program, HUD lacks basic information needed to manage the  program.
According to HUD participation data for HOPE VI (available on a  cumulative basis), at least 73,000 working-age, nondisabled residents  have participated in the community and supportive services component of  the HOPE VI program since the program began in 1993. For the 280  revitalization grants awarded through the program since its inception,  HOPE VI agencies self-reported that about 55,000 original residents  (those who lived at the site before revitalization) participated in a program  or service designed to help them progress towards self-sufficiency. In  addition, HOPE VI agencies self-reported that nearly 18,000 additional  individuals who became residents at the revitalized HOPE VI sites also  participated in community and supportive services.
Programwide data on residents’ participation in MTW activities related to  increasing self-sufficiency from fiscal years 2006 through 2011 generally  were unavailable. While MTW-PIC was created in 2007 to better fit the  needs of MTW agencies, this system was not designed to collect activity- level data (unless a household was participating in one of the two FSS  programs). Moreover, HUD officials do not consider the data in MTW-PIC  to be reliable prior to 2011 because some MTW agencies were still  transitioning to it through the end of 2010. As a result, officials with  responsibility for administering the MTW program have not used MTW- PIC as a tool for analyzing residents’ participation in activities related to  increasing self-sufficiency. In addition, HUD does not analyze the data  that MTW agencies provide in their annual MTW reports, including data  on residents’ participation in activities related to self-sufficiency, because  reporting requirements do not call for the reporting of standardized data,  such as the number of residents who found employment.
We previously recommended that HUD develop and implement a plan for  quantitatively assessing the effectiveness of similar activities, which  would include activities related to encouraging self-sufficiency. HUD  agreed that quantitatively assessing the effectiveness of similar activities  was an important step. HUD has made revisions to its reporting  requirements, which were approved by OMB in May 2013.
According to our analysis of available HUD data, 338,900 households  received rental housing assistance from an MTW agency in fiscal year  2011. Because HUD officials do not believe their PIC or MTW-PIC  systems contain reliable data from MTW agencies prior to 2011, we did  not attempt to analyze these data. Of the MTW households, 14,314 were  reported as participating in the two FSS programs in fiscal year 2011.  According to HUD officials, most MTW agencies participate in FSS, but  the quality of their FSS data reporting is unknown.
section_title: HUD Has Not Fully Utilized the Data It Collects on Self- Sufficiency Programs
subsections: section_title: HUD Collects Data on Indicators of Self-Sufficiency
paragraphs: HUD requirements for collecting data on indicators of self-sufficiency vary  by program. HUD requires PHAs to collect and report into PIC or MTW- PIC certain types of detailed information on every resident of HUD- assisted rental housing. For instance, PHAs collect data on the amount  and sources of residents’ income, including whether it is earned or  provided through disability payments, Temporary Assistance for Needy  Families, or other sources. PHAs are supposed to collect and report this  type of information into HUD’s PIC system at least annually. For residents  affiliated with the MTW program, this information is reported into MTW- PIC, a separate module in the PIC system.
In addition to reporting basic demographic and income data on each  resident, HUD requires PHAs that implement the five programs to report  additional indicators of residents’ progress towards self-sufficiency using  information systems, an Excel-based reporting tool, and narrative reports.
FSS programs and ROSS SC. As discussed earlier in this report,  HUD requires PHAs that receive FSS grants to enter additional data  in the FSS Addendum section of PIC on each family upon entry into  an FSS program and update this information annually. For example,  PHAs enter whether the head of the household works full-time, part- time, or is unemployed; and the highest grade of education for the  head of the household. For the two FSS programs and the ROSS SC  program, HUD also requires PHAs to annually enter summary output  and outcome data related to their residents’ progress towards self- sufficiency into the previously described Excel-based reporting tool.  Data fields in the tool include the number of households that  increased their income, moved to nonsubsidized housing, and the  number of residents that obtained a high school diploma.
HOPE VI. HUD requires participating agencies to submit quarterly  reports of summary data on residents’ progress towards self- sufficiency into the HOPE VI Quarterly Reporting System. For  example, HOPE VI agencies must report the total number of residents  who participated in activities that facilitate self-sufficiency, including  the numbers enrolled in counseling programs, job training, and  General Education Development classes.
MTW. HUD requires MTW agencies to submit annual reports  containing summary information about the impact of the activities  intended to encourage resident self-sufficiency they have been  implementing. The agencies were able to use metrics of their  choosing at the time of our review.
section_title: HUD Has Lacked a Strategy for Using Collected Data to Evaluate the Impact of Four Programs
paragraphs: HUD has performed limited analyses—to assess outcomes for the  programs as a whole—of data related to self-sufficiency outcomes that  FSS grant recipients must report into its information systems. In addition,  HUD has not analyzed similar data that FSS, ROSS SC, and MTW  agencies must report through other mechanisms to assess each program  as a whole. Standards for internal control emphasize the need for federal  agencies both to collect reliable information with which to manage their  programs and review the integrity of performance measures. Moreover,  these standards emphasize the need for program managers to collect  operational data to determine whether they have been meeting their goals  for the efficient and effective use of resources. Additionally, the GPRA   Modernization Act of 2010 (GPRAMA) emphasizes the need for  information on the effectiveness of federal programs to improve  congressional decision making.
Based on the information that PHAs must report in PIC and the FSS  Addendum in PIC, HUD has performed limited analyses of the data  related to self-sufficiency outcomes for the two FSS programs. As  previously discussed, FSS participation data were sometimes incomplete  and therefore of questionable reliability. Specifically, HUD’s data lacked  start dates and annual updates for some FSS participants. Despite these  limitations, HUD has used these data in its Congressional Budget  Justifications. For example, in the 2014 Congressional Budget  Justification HUD reported that as of March 30, 2012, a total of 57,087  families were enrolled in the two FSS programs. Without annually  updated records for each family, HUD cannot reliably determine whether  families were still active program participants.
Additionally, HUD has not analyzed and reported on the experiences of  all families that start the FSS programs, including the extent to which they  completed the program, the primary reasons they exited the program  without completing it, and the extent to which required data are missing.  According to HUD, they have not conducted such analysis due their own  concerns about the usefulness of available data. As described previously,  internal control standards state that such information can help agencies  determine whether they have been meeting operational goals and using  resources effectively. According to HUD officials with responsibility for  administering the two FSS programs, they were aware that some PHAs  did a better job of reporting into PIC than others, which affected the  completeness and reliability of the data. We acknowledge that analyzing  this information and summarizing overall changes in indicators of self- sufficiency among participants would not yield definitive results on the  impact of FSS on resident self-sufficiency. But by not analyzing the data  that FSS grant recipients must report in HUD’s information systems more  thoroughly, HUD has been missing an opportunity to gain valuable  information about the results of FSS programs for certain agencies. That  is, for those agencies that have submitted complete data (all annual  updates as well as information on whether and how the participant exited  the program), HUD could review available information, such as changes  in income and employment. Also, HUD has been missing an opportunity  to identify PHAs with notably effective or ineffective FSS programs (or  data reporting) and learn from their experiences. Ultimately, HUD’s lack of  complete data on FSS participants limits the usefulness of analysis  reported to Congress.
In part due to the data’s weaknesses, HUD also has not assessed the  data related to self-sufficiency outcomes that it requires FSS and ROSS- SC grant recipients to report through the Excel-based reporting tool for  each program as a whole. According to HUD officials, HUD field office  staff review the outcomes data that individual PHAs report, but HUD had  no process in place for assessing the outcomes reported through this tool  programwide. Standards for internal control emphasize the need for  federal agencies to collect reliable information with which to manage their  programs and to review the integrity of performance measures. HUD  officials with responsibility for these programs told us that headquarters  staff did not use the reporting tools to assess the effectiveness of the FSS  or ROSS SC programs because a system had not been developed to do  such an assessment and because the data submitted were sometimes  incomplete, not comparable, and unreliable. HUD officials also stated that  PHAs likely vary in their interpretations of what to report because HUD  never developed program-specific guidance. In 2013, in response to our  review, HUD had its contractor (which collects data PHAs report using the  tool) aggregate the outcomes data reported for the PH FSS and ROSS  SC programs for fiscal years 2008 through 2011. However, HUD staff  found that the results did not appear to be accurate or reliable.
While HUD’s effort to aggregate the performance data that they require  grant recipients to report was a step in the right direction, the agency  lacks a strategy for better ensuring that the outcomes data it collects are  reliable and permit comparison across PHAs. Without a plan for helping  to ensure that the outcome data FSS and ROSS SC grant recipients  report are comparable and reliable, HUD will be unable to fully use the  data it requires PHAs to report.
HUD has not assessed the effectiveness of the MTW program using the  information that it requires MTW agencies to submit in their annual  performance reports on the impact of their MTW activities, including  activities related to increasing resident self-sufficiency. MTW agencies  generally have devised their own metrics for activities and reporting  performance information, so the usefulness of this information for  assessing programwide results is limited. That is, because the data are  not consistent across agencies, they cannot be used to assess the  performance of similar activities across MTW agencies. Additionally, in  some cases the information is not outcome-oriented and thus cannot be  effectively used to assess performance.
We previously recommended that HUD (1) improve its guidance to MTW  agencies on providing information in their performance reports by  requiring that such information be quantifiable and outcome-oriented to  the extent possible; and (2) develop and implement a plan for  quantitatively assessing the program as a whole, including the  identification of standard performance data needed. HUD generally  agreed with our recommendations, and in May 2013 OMB approved the  revised guidance. Additionally, according to HUD, the agency continues  to seek funding for a full evaluation that will better analyze the MTW  information that is already collected and assess the effectiveness of the  program, including self-sufficiency activities.
We recommended in July 1998 that HUD develop consistent national, outcome-based  measures for community and supportive services at HOPE VI sites. HUD has used its  HOPE VI reporting system to collect data from grantees on the major types of community  and supportive services they provide and outcomes achieved by some of these services.  See GAO, HOPE VI: Progress and Problems in Revitalizing Distressed Public Housing,  GAO/RCED-98-187 (Washington, D.C.: July 20, 1998); and Public Housing: HOPE VI  Resident Issues and Changes in Neighborhoods Surrounding Grant Sites, GAO-04-109  (Washington, D.C.: Nov. 21, 2003). many residents enrolled in services related to education and job training,  and from one-third to half of those who enrolled completed the activity  (see table 6). However, these outputs may not be directly attributable to  the community and supportive services program and are self-reported by  HOPE VI agencies. HUD officials with responsibility for collecting these  data noted that it would be excessively time and resource-intensive to  verify the accuracy of these data. HUD staff also noted that if a HOPE VI  agency’s community and supportive services data appeared to be  inconsistent with past trends, HUD staff would follow up with the agency.
Where HUD had data, the data suggest positive changes in income and  employment for families that participated in the two FSS programs, but  these results are not conclusive. Data on program completion were  missing for nearly half of the records we evaluated. More specifically, the  FSS Addendum data in PIC lacked exit, completion, or extension data on  6,819 (46 percent) of 14,690 families that started either of the programs in  2006. Of these families, the “family report” section of PIC indicated that  1,671 (25 percent) had left subsidized housing. Thus, after comparing  both the FSS Addendum and the main “family report” data, we found that  HUD’s systems lacked information on whether 5,148—35 percent—of the  families that started either of the FSS programs in 2006 exited the FSS  programs or subsidized housing.
Of the subset of families for which exit, completion, or extension data  were available in the FSS Addendum of PIC (that is, 54 percent of  program participants), about  60 percent exited the program without  completing it.  In contrast, 25 percent of participants completed the  program in 5 years or less (see table 7).
Considering only those families for which HUD had complete data, we  observed positive changes in income and employment.  We observed  these changes for the families that started an FSS program in 2006 and  completed it in 5 years or less.based on the 25 percent of families for which HUD data indicated  completed the programs in 5 years or less. But, these findings do not take  into account other factors that may have affected the families’ progress  towards self-sufficiency. Specifically, of the 1,937 families that started  either of the two FSS programs in fiscal year 2006 and completed them  within 5 years, HUD’s data suggest positive changes in income and  employment (see table 8). For example, median income increased from  about $17,000 per year to about $25,000 per year. Families with total  incomes of $35,000 or more when they started the program (ninetieth  percentile) experienced income gains of 34 percent, compared with  income gains of 106 percent for families that had total incomes of about  $3,000 or less (tenth percentile). These families also experienced positive  changes in employment. For example, full-time employment among these  program graduates increased 76 percent. The proportion of these  graduates who were working part-time decreased 30 percent and the  proportion not employed decreased 61 percent.
section_title: HUD Encourages PHAs to Coordinate with Local Service Providers
subsections: section_title: HUD’s Strategic Plan and Guidance Encourage Coordination, and Some Programs Require It
paragraphs: HUD aims to improve self-sufficiency among residents of HUD-assisted  rental housing by encouraging coordination between PHAs and agencies  offering services that promote work and self-sufficiency. HUD’s 2010- 2015 strategic plan presents strategies for increasing resident self- sufficiency, which include coordination with federal, state, and local  programs to increase access to job training, career services, and work  support.
Consistent with this strategy, HUD formed a partnership with the  Department of Labor (Labor) to improve residents’ access to Labor  programs and services. Residents of HUD-assisted rental housing may  be eligible for a variety of Labor programs and services, including those  funded through the Workforce Investment Act (WIA). These services are  delivered locally through American Job Centers, also known as one-stop  centers, which provide education and career training, job search tools,  and assistance with developing resumes and interview skills. Since 2009,  HUD officials have held meetings with Labor’s Employment and Training  Administration to share information about their respective programs and  seek opportunities to collaborate. One product of these meetings has  been a joint toolkit for PHAs and local workforce agencies intended to  improve HUD-assisted residents’ access to employment services offered  through the workforce agencies’ one-stop centers, intended for release in  2013. The draft toolkit offers PHAs and workforce agencies a baseline  understanding of each other’s functions, examples and lessons learned  from successful partnerships currently in place, information on available  online resources, and sample Memorandums of Understanding (MOU), or  partnership agreements.
HUD guidance also encourages PHAs to coordinate with local entities  such as social service agencies and job training providers to increase  residents’ self-sufficiency. For example, HUD issued a notice to all PHAs  in 2011 promoting partnerships with such agencies. The notice  describes the benefits of collaboration, provides examples of possible  partnerships and strategies for partnership development, and includes  model MOUs. The notice states that HUD encourages such partnerships  and recognizes that they benefit both PHAs and the households they  serve. In addition, HUD’s guidance to PHAs implementing the self- sufficiency programs we reviewed emphasizes the value of coordinating  with local agencies. For example, HUD developed a resource website for  HOPE VI grantees that provides information on interagency coordination  and examples of PHA practices. Also, HUD’s training materials for  ROSS SC grantees include resources that PHAs can use to form  partnerships, such as a sample partner outreach letter. While this  guidance is directed to HOPE VI and ROSS SC grantees, it is publicly  available on HUD’s website and can be accessed by all PHAs.
HUD’s program requirements for four of the self-sufficiency programs we  reviewed call for PHAs to coordinate with local service providers (see  table 9). For example, grant funds for the two FSS programs and the  ROSS SC program cannot be used to fund direct services. Instead, they  must be used to hire coordinators who refer residents to local service  providers, making coordination a key component of these programs’  designs. Furthermore, applicants for ROSS SC and HOPE VI grants must  demonstrate financial or in-kind support from partner organizations or  agencies. Additionally, supportive services funded through HOPE VI  grants must be coordinated with other service providers, including state  and local programs. Finally, all four programs require participating PHAs  to form coordinating committees to help ensure that residents are linked  to the services that they need. For example, PHAs that receive either of  the two FSS grants must establish a Program Coordinating Committee,  which is charged with securing commitments of public or private  resources for the operation of self-sufficiency programs. PHAs are  encouraged to include local service providers, including welfare and  workforce agencies, on the committees.
Selected PHAs worked with local service providers to implement HUD’s  self-sufficiency programs. We interviewed officials from a sample of five  PHAs that had implemented one or more of HUD’s self-sufficiency  programs, as well as officials from workforce and welfare agencies in   those locations.local agencies that provided a broad range of services, including job  training, mental health services, child care, transportation, food  assistance, and homeownership counseling. Some of these local  agencies received federal funding—for example, workforce agencies that  administered WIA programs. PHA and local agency officials with whom  we met stated that coordination efforts sometimes were formally  established through MOUs, contracts, or regular meetings but added that  most coordination efforts were informal. This informal coordination  included referrals to each other’s services or the sharing of information or  updates on new programs and services.
All of the PHAs we interviewed connected residents to  Officials from the five PHAs with whom we met identified few barriers to  coordinating with local service providers and found current HUD guidance  related to such coordination sufficient. Officials said that, despite resource  constraints, they generally were able to obtain the services their residents  needed from local agencies. PHA officials also stated that local agencies,  including federally funded agencies, were receptive to their coordination  efforts. However, while officials from two PHAs stated that they had  strong relationships with their local workforce agencies, officials from the  other three PHAs noted that these agencies’ one-stop centers could be  intimidating for residents and might not always be able to provide  residents with appropriate services. Officials from workforce agencies we  interviewed generally agreed, noting that residents might not have the  education or work experience needed for some of the training  opportunities that the centers offered. For example, a workforce agency  official told us that individuals interested in job training in certain fields,  such as trucking or nursing, must take tests to demonstrate required  levels of academic readiness to participate. He said that many assisted   housing residents were not prepared to pass these tests and might first  need remedial adult education. HUD officials with responsibility for  administering the self-sufficiency programs we reviewed and Labor  officials with responsibilities related to workforce agencies told us that the  joint HUD-Labor toolkit described above was developed in part to improve  resident access to services offered through the one-stop centers.
In general, awareness of federal efforts to improve coordination was  mixed among the PHAs, workforce, and welfare agencies with which we  met. For example, officials from two of the PHAs were familiar with the  HUD notice on promoting partnerships, but officials from the other three  PHAs were not. Additionally, officials from two of the PHAs and all five  workforce agencies were not aware of a 2009 joint HUD-Labor letter on  collaboration. Still, officials from these agencies generally did not  express a need for further federal guidance or assistance in facilitating  relationships between PHAs and local agencies.
section_title: Conclusions
paragraphs: HUD’s housing assistance programs serve millions of low-income  residents. Statutory and programmatic requirements for HUD and PHAs  also direct the organizations to undertake activities that would help these  households increase their economic self-sufficiency. In particular, HUD  has five programs that, in whole or in part, fund activities intended to help  families become self-sufficient. However, HUD faces two major  impediments to effectively operating these programs and achieving their  goals. First, it does not have reliable data on participation in self- sufficiency activities across PHAs.  Second, even though it has complete  data for some PHAs, it generally does not use these data to review the  progress participants may have made in, for example, finding a job or  completing more education.
In relation to participation data for the programs we reviewed, HUD has  missed opportunities to help ensure that FSS participation data are  complete. For example, in the two FSS programs significant gaps exist in  participant entry and completion dates or reasons for leaving. These gaps  are detrimental for several reasons, including their effects on grant  funding and congressional reporting. Specifically, HUD uses resident  participation data as a factor in making grant funding decisions and has  reported this information in Congressional Budget Justifications.  Moreover, without such data HUD cannot identify PHAs that have low or  high completion rates. Federal internal control standards state that  transactions should be promptly recorded to maintain their relevance and  value to management and HUD’s own reporting guidance also directs  grant recipients to record program start dates, exit dates, reasons for  exiting the program prior to completion, and completion dates. HUD has  recognized the importance of having reliable participation data in recent  years, emphasizing its importance to grantees in funding notices. But  HUD could further work with agencies to correct participation data during  the grant award process, or analyze the data outside of this process to  help ensure that all required data are complete. By doing so, HUD will  improve the accuracy of these data, improve its ability to assess FSS  grantee activities and thus make better-informed decisions about funding  them, and provide Congress a more complete view of program  performance.
Similarly, ROSS SC is designed to provide grant recipients with funds to  hire staff to help residents progress towards economic independence and  self-sufficiency. Since 2008, HUD has required grant recipients to report  participation data. However, HUD has not provided grant recipients with  program-specific reporting guidance; thus, the reported data vary and  cannot be easily or reliably used for assessing programwide participation.  Federal internal control standards state that program managers need  operational data to determine whether they are meeting their agencies’  strategic plans and meeting their goals for the effective and efficient use  of resources. By developing program-specific reporting guidance, HUD  could help ensure the collection of accurate participation data, and  establish data sets that can be used to assess participation for the ROSS  SC program as a whole.
In relation to outcome data for the programs we reviewed, HUD has not  optimized its use of the information it requires PHAs to collect. We  acknowledge that determining the outcomes of self-sufficiency activities is  difficult and requires rigorous analyses. It is also difficult to isolate the  impact of such activities from other factors that may influence participant  outcomes. But HUD could do more to put itself in a position to look across  a program to review participant accomplishments. While HUD has  collected some indicators for the FSS and ROSS SC programs (such as  information on hours worked and receipt of welfare), it lacks a strategy for  using the data it collects, whether through PIC or its Excel-based  reporting tool. And, as with participation data, PHAs have not consistently  reported such information, a condition exacerbated by the lack of  program-specific reporting guidance. As stated above, internal control  standards underline the importance not only of collecting but also using  information to achieve programmatic goals—helping families increase  self-sufficiency. Additionally, GPRAMA emphasizes the need for  information on the effectiveness of federal programs to help improve  congressional decision making. A strategy for using these data could  inform overall management review, congressional oversight, and planning  for these programs. For instance, using such data could help HUD  identify from which PHAs to draw lessons to help improve HUD  management of the grant programs as well as PHA management of self- sufficiency-related activities.
section_title: Recommendations for Executive Action
paragraphs: To better inform Congress and improve what is known about residents’  participation in key grant programs designed to facilitate resident self- sufficiency, and their progress towards self-sufficiency, the Secretary of  the Department of Housing and Urban Development should develop and  implement a process to better ensure that data on FSS participant grants are  complete; such a process should include steps for identifying missing  data, identifying the reasons for missing data, and taking steps to help  ensure data are complete; a process to better ensure that PHAs awarded ROSS SC grants  annually report required participation and outcome data that are  comparable among grant recipients; this process should include the  issuance of program-specific reporting guidance; a strategy for regularly analyzing FSS participation and outcome data;  such a strategy could include identification of PHAs from which  lessons could be learned and PHAs that may need assistance  improving completion rates or outcomes; and a strategy for regularly analyzing ROSS SC participation and outcome  data; such a strategy could include identification of PHAs from which  lessons could be learned and PHAs that may need assistance  improving participation rates or outcomes.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to HUD, Labor, Treasury, and HHS.  HUD provided written comments, which are reprinted in appendix II.  Labor and HHS provided technical comments, which we incorporated as  appropriate, and Treasury did not provide comments.  HUD agreed with  three of our recommendations, and pointed to actions it intends to take to  implement them.  While agreeing with these recommendations, HUD  noted some concerns about our recommendation that it develop and  implement a strategy for regularly analyzing FSS participation and  outcome data, and disagreed with our similar recommendation for the  ROSS SC program.
While agreeing with our recommendation that it develop and implement a  strategy for regularly analyzing FSS participation and outcome data, HUD  stated that the data captured in PIC for this program are not designed for  rigorous statistical analysis. However, according to HUD guidance, PIC  can be used to create reports, analyze programs, monitor PHAs, and  provide information about residents in HUD-subsidized housing. In  addition, if participating PHAs annually updated data, as required, in the  FSS Addendum, PIC would include data on sources of assistance  received by the family; whether the head of the household worked full- time, part-time, or was unemployed; and the highest grade of education  for the head of the household. By implementing our first  recommendation—that HUD develop and implement a process to better  ensure that data on FSS participants are complete—the completeness  and therefore reliability of outcome data should improve. Consequently,  usefulness of analysis of such data would also be improved.
HUD disagreed that it should develop a strategy for regularly analyzing  ROSS SC participation and outcome data. In doing so, the agency noted  the small size of the program, the difficulty of analyzing outcomes, and  that the data it collects are administrative in nature and not intended to  serve as the basis for analysis. However, HUD’s training materials on the  Excel-based tool that it uses to collect participation and outcomes data for  the ROSS-SC program state that the tool is intended to be used to  manage, monitor, and evaluate program services. These training  materials and HUD staff with whom we met also indicated that the tool,  when populated by participating PHAs, contains data on outputs and  outcomes.  And, as noted above, HUD agreed that its reporting guidance  to ROSS SC grantees should be improved to help make data more  meaningful. HUD’s disagreement with our recommendation to analyze the  information collected does not accord with its requirement for PHAs to  submit operational data or with its assessment that data quality ought to  be improved. Consistent with internal controls for the federal government,  which apply to programs of all sizes, regular analysis of ROSS SC  operational data would help HUD determine whether it was meeting goals  for the effective and efficient use of resources. It can be difficult to isolate  and definitively assess program outcomes. But program data can help  identify patterns and anomalies in program operations, which can help  agencies discover ways to achieve more value for the taxpayer’s money.  Consequently, we continue to recommend that HUD develop and  implement a strategy for regularly analyzing ROSS SC participation and  program data.
In its technical comments, HUD also raised concerns about the accuracy  of dollar amounts reported and the characterization of outputs related to  the community and supportive services component of HOPE VI.  First,  HUD questioned the accuracy of the amounts awarded to PHAs through  the FSS and ROSS SC programs and the amounts HOPE VI agencies  spent on community and supportive services. Because trends in nominal  spending may reflect changes in both price and quantity, we chose to  present inflation-adjusted values by removing the general effects of  inflation using a price index. Specifically, we adjusted these figures to  fiscal year 2013 dollars using the fiscal year chain-weighted Gross  Domestic Product price index. Second, for the HOPE VI program, HUD  stated that without a costly and time-intensive experimental research  design, it would not be possible to know whether the outputs residents’  experienced were directly attributable to the program. We acknowledged  the difficulties of isolating program outcomes in this report, and have not   made any recommendations related to HOPE VI. Finally, during the  course of our review, OMB approved revised reporting requirements for  the MTW program, which are intended to establish standard metrics for  activities related to self-sufficiency, among other things. We revised the  report to recognize these changes. HUD provided additional technical  comments, which we incorporated as appropriate.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretary of Housing  and Urban Development and other interested committees. In addition, the  report will be available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-8678 or sciremj@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. Key contributors to this report are listed in  appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: Our objectives were to (1) discuss what is known about the costs of and  residents’ participation in Department of Housing and Urban Development  (HUD) grant programs that encourage work and self-sufficiency, (2)  determine what is known about the effect on residents of HUD grant  programs to promote self-sufficiency, and (3) describe steps HUD has  taken to coordinate with other federal agencies and increase residents’  access to non-HUD programs that encourage work and self-sufficiency.
To discuss what is known about the costs of and residents’ participation  in HUD programs that encourage self-sufficiency we reviewed  documentation of HUD’s programs and determined whether and how they  defined and addressed resident self-sufficiency. Because HUD did not  have an agencywide definition of self-sufficiency, and included 20  programs in its list of programs that contribute to its subgoal of increasing  economic security and self-sufficiency, we established criteria to narrow  the scope of programs to include in our review. First, we limited our  review to grant programs that were intended to encourage work and self- sufficiency among residents of HUD-assisted rental housing. Second, we  included grant programs that did not have self-sufficiency as their primary  focus but rather as a secondary program goal. We sought to include  programs that were actively awarding grants, or for which grants were still  open. Based on these criteria, we identified the following programs:
Public Housing Family Self-Sufficiency (PH FSS)
Housing Choice Voucher Family Self-Sufficiency (HCV FSS)
Resident Opportunity and Self-Sufficiency Service Coordinators   (ROSS SC)
Moving to Work (MTW)
HOPE VI  HUD officials agreed that we had identified the key grant programs that  encourage work and self-sufficiency. To describe what is known about the  programs’ costs, we met with program staff and obtained documentation  of the total grant amounts awarded for specific periods, as follows:
For the Family Self-Sufficiency (FSS) programs, we obtained grant  award amounts for fiscal years 2006 through 2011 and analyzed data  from HUD’s Public and Indian Housing Information Center (PIC)  system on escrow account disbursements from fiscal years 2006  through 2011. We adjusted all dollar amounts to fiscal year 2013  dollars using the chain-weighted Gross Domestic Product price index.
For ROSS SC, we obtained grant award amounts for fiscal years  2008 through 2011, because 2008 was the first year of the current  version of the program.
For the MTW program, we met with program staff and staff from  HUD’s Grants Management Office to determine whether HUD  collected information on MTW agencies’ expenditures for activities  related to self-sufficiency.
For the HOPE VI program, we reviewed and summarized the fiscal  year 2006-2010 Notice of Funding Availability (NOFA) requirements  related to community and supportive services expenditures and the  amount of HOPE VI revitalization grant funds that HOPE VI agencies  expended on community and supportive services in that period.
To analyze residents’ participation in each of the five programs, we met  with HUD staff to determine how participation data were collected and  summarized any readily available information.
For the two FSS programs, we analyzed participation data from PIC  for fiscal years 2006 through 2011. Specifically, we analyzed data that  public housing agencies (PHA) entered in the FSS Addendum, a PIC  subcomponent. As a part of this analysis, we identified the extent to  which program start dates, end dates, and annual updates for  participating households were entered. We interviewed HUD officials  with responsibility for administering both FSS programs and the PIC  system about the completeness of the data and HUD’s use of the  information. We evaluated the information available on families’  participation in the two FSS programs in relation to internal control  standards for the federal government as well as HUD’s own  guidance. Based on interviews with HUD officials and our review of  the completeness of data fields in the FSS Addendum component of  PIC, we determined that the available data were not sufficiently  reliable to provide an accurate count of residents’ participation in FSS  for fiscal years 2006 through 2011. We summarized data that HUD  posted on its website as a part of the NOFA process for fiscal years  2009 through 2012.
For the ROSS SC program, we obtained and reviewed aggregated  data on program participation that a HUD contractor created in 2013.  We evaluated the information available on residents’ participation in  ROSS SC in relation to internal control standards for the federal  government. We determined that the aggregated participation data were not sufficiently reliable to provide an accurate count of residents’  participation in ROSS SC for fiscal years 2008 through 2011.
For the HOPE VI program, we summarized data from the HOPE VI  Quarterly Reporting System on the total number of nondisabled  residents between the ages of 18 and 64 who had received services  through the community and supportive services component of HOPE  VI from the program’s inception through the end of 2012.
For the MTW program, we interviewed program administrators to  determine whether and how they used information from participating  agencies’ Annual MTW Reports or MTW-PIC to determine the number  of MTW households participating in MTW and activities related to self- sufficiency. We analyzed the FSS Addendums associated with MTW- PIC in an effort to determine the number of MTW families that had  participated in either of the two FSS programs. We determined, based  on interviews with HUD officials and our analysis of FSS Addendum  data for MTW agencies, that these data were not sufficiently reliable  prior to 2011. We summarized available data on MTW families’  participation in the two FSS programs in 2011.
To determine what is known about the effect on residents of HUD’s grant  programs that encourage work and self-sufficiency, we examined whether  and how HUD collected information that could indicate progress toward  self-sufficiency, such as information on income and employment, for each  program. We reviewed available data dictionaries and guidance HUD  provided to PHAs required to report this information. We determined that  for the five programs we reviewed, HUD used PIC, MTW-PIC, Excel- based reporting tools, written reports, and other reporting systems to  collect this information. We interviewed HUD staff about their use of the  self-sufficiency indicators they require PHAs to report, the reliability of the  data collected, and program-specific guidance. We evaluated HUD’s  processes for aggregating self-sufficiency-related outcomes data in  relation to our standards for internal control. For the two FSS programs,  we analyzed PIC data on the 14,690 families that HUD’s data indicated  started the programs in 2006. Based on PIC data for fiscal years 2006  through 2012, we identified the number of families that completed the two  FSS programs, exited the programs without completing them, and  received an extension to continue the program past 2011, and for which  HUD’s system lacked either program exit, completion, or extension  information. For the subset of families that HUD’s data indicated had  completed the two FSS programs in 5 years or less (1,937 households),  we analyzed changes in median and mean income, income for those at  the tenth and ninetieth percentiles, and employment experiences. We  adjusted all dollar amounts to fiscal year 2013 dollars using the chain- weighted Gross Domestic Product price index. This sample represented  about 25 percent of the families that started in 2006 and for which exit,  completion, or extension data were available. In addition, this analysis  does not control for other factors that may have affected participants’  progress towards self-sufficiency. We excluded any households that were  receiving rental assistance from an MTW agency because HUD officials  indicated that their FSS participation data were not as reliable, particularly  before 2011.
Through interviews and a literature search, we identified several studies  of the two FSS programs, HOPE VI, and MTW. We reviewed these  studies to identify information on the programs’ impact on resident self- sufficiency. We determined that these reports were methodologically  sound and reliable for our purposes. We did not identify any studies of the  ROSS SC program.
To describe the steps HUD has taken to coordinate with other federal  agencies and increase residents’ access to non-HUD programs, we  reviewed HUD regulations, policies, and guidance related to coordination.  We also interviewed officials from HUD, the Department of Health and  Human Services, and the Department of Labor, and reviewed materials  related to interagency coordination provided by these officials. To better  understand how PHAs connect residents with non-HUD programs, we  interviewed staff from PHAs administering each of the self-sufficiency  programs included in our review. To select PHAs, we compiled lists of  PHAs that received grant funding (or permission to participate in MTW)  between 2004 and 2011. We then randomly selected a PHA to interview  for each program and adjusted our initial sample to ensure that the five  selected PHAs varied in terms of size and region. Four of the selected  PHAs implement more than one of the programs in our review. The  selected PHAs were:
Boulder County Housing Authority (Boulder, Colorado)
Jersey City Housing Authority (Jersey City, New Jersey)
Kingsport Housing and Redevelopment Authority (Kingsport,  Tennessee)
Louisville Metro Housing Authority (Louisville, Kentucky)
Washington County Housing and Redevelopment Authority  (Woodbury, Minnesota)
For each of these locations, we also interviewed officials from the local  agencies that administered Temporary Assistance for Needy Families  and the Workforce Investment Act programs.
We conducted this performance audit from August 2012 through July  2013 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on the audit objectives.
section_title: Appendix II: Comments from the Department of Housing and Urban Development
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Paul Schmidt (Assistant  Director), Stephen Brown, Emily Chalmers, Geoff King, John McGrail,  Marc Molino, Lisa Moore, Michael Pahr, Barbara Roesmann, Beverly  Ross, and Andrew Stavisky made key contributions to this report.