Troubled Asset Relief Program: Monitoring of the Hardest Hit Fund Program Could Be Strengthened
Released: 2018-12-21T00:00:00

Letter

Background

Treasury established HHF in February 2010 to help stabilize the housing market and assist homeowners facing foreclosure in the states hardest hit by the housing crisis. The HHF program is implemented by Treasury’s Office of Financial Stability. Treasury obligated funds to 18 states and the District of Columbia. Treasury allocated funds to each state’s HFA to help unemployed homeowners and others affected by house price declines. HFAs, in turn, design their own programs under HHF specific to local economic needs and circumstances pursuant to their contracts with Treasury.

Treasury allocated $9.6 billion in HHF funding to 19 HFAs in five rounds. As described below, Treasury allocated $7.6 billion to participating HFAs during the first four rounds of funding, all of which occurred in 2010. HFAs were required to disburse these funds by December 2017.

Round one: In February 2010, Treasury allocated $1.5 billion to the HFAs in the five states that had experienced the greatest housing price declines—Arizona, California, Florida, Michigan, and Nevada.

Round two: In March 2010, Treasury allocated $600 million to the HFAs in five states with a large proportion of their populations living in counties with unemployment rates above 12 percent in 2009—North Carolina, Ohio, Oregon, Rhode Island, and South Carolina.

Round three: In August 2010, Treasury allocated $2 billion to the HFAs in nine of the states funded in the previous rounds, along with the HFAs for eight additional states and the District of Columbia, all of which had unemployment rates higher than the national average in 2009. The additional HFAs that received funding were Alabama, the District of Columbia, Georgia, Illinois, Indiana, Kentucky, Mississippi, New Jersey, and Tennessee.

Round four: In September 2010, Treasury allocated an additional $3.5 billion to the same 19 HFAs that received HHF funding through the previous rounds.

In December 2015, the Consolidated Appropriations Act, 2016 authorized Treasury to make an additional $2 billion in unused TARP funds available to existing HHF participants. In early 2016, Treasury announced a fifth round of HHF funding. According to Treasury and HFA officials and other stakeholders, by that time some of the participating HFAs had begun to wind down their programs by letting go of program staff or making other changes after they had disbursed most of their funding from the first four rounds. Treasury allocated this additional $2 billion in two phases.

Round five, phase one: In February 2016, Treasury allocated $1 billion to 18 of the HFAs that had previously been awarded HHF funds based on each state’s population and utilization of previous HHF funds. In order to qualify for phase one funding, states had to have drawn at least 50 percent of their previously received funding.

Round five, phase two: In April 2016, Treasury allocated an additional $1 billion to 13 HFAs that applied and sufficiently demonstrated to Treasury their states’ ongoing housing market needs and the ability to effectively utilize additional funds. The HFAs that received funding were California, District of Columbia, Illinois, Indiana, Kentucky, Michigan, Mississippi, New Jersey, North Carolina, Ohio, Oregon, Rhode Island, and Tennessee.

In conjunction with the fifth round of funding, Treasury extended the deadline for disbursement to December 31, 2021. Treasury also determined that HFAs must finish reviewing and underwriting all applications for final approval to participate in the program no later than December 31, 2020. HFAs that do not disburse HHF funds by the December 31, 2021, deadline will have to return the remainder of the funds to Treasury. See figure 1 for an overview of the allocation amounts and disbursement deadlines.

HHF Programs

Under HHF, HFAs designed locally tailored programs that address HHF’s goals of preventing foreclosures and stabilizing housing markets. These programs had to meet the requirements of the Emergency Economic Stabilization Act of 2008 and be approved by Treasury. Treasury categorizes programs into six types, which are discussed in detail later in this report, including programs that provide monthly mortgage payment assistance and programs that reduce the principal of a mortgage. Programs vary by state in terms of eligibility criteria and other details.

HFAs contract with various stakeholders to implement HHF programs, including mortgage servicers and, in some cases, housing counseling agencies and land banks. The types of stakeholders involved vary depending on program design. For example, HFAs with blight elimination programs may choose to provide HHF funding to a local land bank to demolish and green blighted properties in distressed housing markets.

Also, HFAs may contract with housing counseling agencies approved by the Department of Housing and Urban Development (HUD) to identify eligible applicants at risk of foreclosure.

HFAs are required to report performance information on each of their HHF programs to Treasury on a quarterly basis. This information includes outputs, such as the number of homeowners assisted or properties demolished, as well as outcomes, such as the number of homeowners who are no longer participating in HHF programs. The specific types of performance information that Treasury requires HFAs to report vary depending on the program type and include both intended and unintended consequences of the program. For example, HFAs with mortgage payment assistance programs must report on the number of homeowners who have transitioned out of the program due to specific changes in their circumstances, such as regaining employment. HFAs do not have to report on the number of borrowers who transitioned out of the program into foreclosure sales, short sales, or deeds-in-lieu of foreclosure for their down payment assistance programs because the assistance is provided on behalf of a buyer who is purchasing, not selling or otherwise exiting, the home. Treasury provides HFAs with spreadsheet templates, which HFAs are to fill out and submit back to Treasury. The templates include data-reporting guidance in the form of a data dictionary, which describes the data elements HFAs are to report.

Participation Agreements

Participating HFAs’ HHF programs are governed by a participation agreement, or contract, with Treasury that outlines the terms and conditions in providing services that the HFA must meet as a recipient of HHF funds. Each agreement includes reporting requirements, program deadlines, and descriptions of permitted administrative expenses. Additionally, agreements include detailed descriptions of the HHF programs that Treasury has approved. Program descriptions include details such as eligibility criteria, structure of assistance, and the estimated number of participating homeowners.

Participation agreements may be amended with Treasury approval to reflect changes to HHF programs, such as new requirements from Treasury or changes in the amounts HFAs allocate to each program. As an example, in 2015 Treasury added new conditions, called utilization thresholds, to each HFA’s participation agreement. The thresholds establish the percentage of allocated funds each HFA was required to draw from its Treasury account by the end of each year from 2016 through 2018. If an HFA did not meet a threshold, Treasury reallocated a portion of the additional funds received during the fifth round to HFAs that did meet the threshold. If an HFA would like to make a change to an HHF program, the HFA must submit a request to Treasury that outlines the proposed change. Treasury reviews the proposal through an interdisciplinary committee and, if the proposal is approved, amends the participation agreement. As of December 2017, the 19 participating HFAs had each received approval from Treasury and executed between 9 and 21 amendments to their individual participation agreements.

Treasury’s Monitoring of HHF Addresses or Partially Addresses Leading Practices for Program Oversight

Treasury’s policies and procedures to monitor HFAs’ implementation of the HHF program address 10 leading monitoring practices, including practices related to the collection of periodic performance reports and validation of performance through site visits. However, Treasury’s assessment of HFAs’ internal control programs, development of performance indicators, documentation of goals and measures, and documentation of HFAs’ monitoring could better address leading practices (see fig. 2).

Treasury Addressed 10 Leading Practices for Monitoring

Regular Monitoring of Policies and Procedures

Treasury created policies and procedures to guide regular oversight of HFAs’ implementation of HHF. According to internal control standards for the federal government, management should design control activities to achieve objectives and implement control activities through policies— such as by periodically reviewing policies, procedures, and related control activities. In addition, management should establish and operate activities to monitor the internal control system and evaluate the results— for example, through ongoing monitoring procedures and separate evaluations. Treasury documented procedures for key areas of its monitoring framework, including providing funds to HFAs, evaluating HFAs’ requests to change their programs, collecting financial and performance information from HFAs, conducting site visits, and addressing fraud detection and mitigation for Treasury’s staff. Treasury regularly updates the policies and procedures it created and reviews its compliance oversight procedures annually. In addition, Treasury regularly conducts site visits to HFAs, as discussed below.

Risk-Based Monitoring Approach

Treasury uses a risk-based approach to selecting HFAs for its regular site visits. This approach is consistent with leading practices we have developed for managing fraud risk, which state that agencies should employ a risk-based approach to fraud monitoring by taking into account internal and external factors that can influence the control environment. In 2018, Treasury began using a point-based, 29-factor approach to selecting HFAs for site visits for compliance reviews, taking into account factors such as whether prior fraud was detected or reported, observations from HFAs’ compliance reviews, administrative dollars spent compared to program assistance provided, and whether HFAs have documented blight-specific policies and procedures. According to Treasury staff, during site visits Treasury determines its test and sample sizes for a risk-based review of an HFA’s programs.

Treasury also uses a risk-based approach to responding to potentially impermissible payments, and according to Treasury staff, its responses depend on the circumstances. If an HFA notifies Treasury of issues related to inappropriate payments involving fraud, waste, or abuse, Treasury staff notify and work with the Office of the Special Inspector General for the Troubled Asset Relief Program (SIGTARP) to provide technical assistance as needed. In 2017, Treasury implemented additional procedures with regard to HFAs’ administrative expenses. If Treasury identifies an administrative expense issue during a site visit, Treasury requires the visited HFA to undertake a multistep review of its administrative expenses, including reviewing additional administrative expenses if similar problems are identified during the initial review. The HFA is required to reimburse HHF for any administrative expenses that were not made in accordance with federal cost principles. Additionally, Treasury may require the HFA to create a plan for corrective action.

Periodic Collection of Performance Reports and Data from Implementing Partners

Treasury collects performance information from participating HFAs on a regular basis, which a compliance team receives and reviews. These efforts are consistent with internal control standards, which state that management should use quality information to achieve the entity’s objectives, such as by obtaining relevant data from reliable sources. Treasury tracks its receipt of agencies’ quarterly performance reports and financial statements, as well as HFAs’ annual internal control certifications. Quarterly performance reports include information about homeowners, such as the number of homeowners who receive or are denied assistance. These reports also include program-specific performance data, such as the median assistance amount, and outcomes, such as the number of program participants who still own their home. According to HFAs’ participation agreements, HFAs are required to report performance information through the end of their programs. In addition, Treasury collects informal monthly updates from HFAs on their program performance and is in frequent contact with HFAs by phone to obtain information on HFAs’ performance, including any challenges states are facing, according to Treasury staff and HFAs with whom we met. Treasury also collects reports on the impact of blight elimination programs, which HFAs with these programs are required to submit to Treasury.

Periodic Analysis of Performance Data

Treasury regularly analyzes the performance and financial data that it collects through quarterly performance reports, quarterly unaudited financial statements, and annual audited financial statements that HFAs are required to submit. Periodic analysis of these materials is consistent with standards for internal control, which state that management should design control activities to achieve objectives and respond to risks—for example, by establishing activities to monitor performance measures and indicators. Treasury uses information from quarterly performance reports to produce quarterly reports for the public on the number of homeowners who received or were denied assistance, among other things. Treasury also includes data on the extent to which states have spent their HHF funding in monthly reports to Congress. Additionally, Treasury analyzes quarterly unaudited and annual audited financial statements to monitor HFAs’ spending of program funds and identify any areas of concern. According to Treasury staff, the agency also uses performance information HFAs report quarterly, such as the number of homeowners who receive or are denied assistance, to assess whether HFAs are making sufficient progress in effectively utilizing program funds to reach the targets for assisting homeowners.

Procedures for Ensuring Quality of Performance Data

Treasury has procedures to assess the quality of HFAs’ performance data when reviewing quarterly performance reports and conducting site visits. These procedures are consistent with internal control standards, which state that management should use quality information to achieve the entity’s objectives, such as by evaluating data sources for reliability. According to Treasury staff, beginning in the first quarter of 2018, Treasury required all participating HFAs to upload their performance data into a system that does basic data reliability testing, such as ensuring the numbers submitted by HFAs are consistent with data submitted for previous quarters. This system flags outliers or large changes for further review. Prior to this requirement, HFAs could use the system optionally. HFAs are able to upload their data as frequently as they want to check for errors or inconsistencies.

After performance information is uploaded into the system, two Treasury staff review any issues flagged by the system and follow up with HFAs to resolve them. According to Treasury staff, as an additional validation step, Treasury staff conducts a reconciliation by checking whether the funds reported in HFAs’ performance reports match the data in the HFAs’ quarterly financial reports. After Treasury reviews each HFA’s performance data, it combines that information to create quarterly reports. In addition, Treasury staff told us that they do a detailed review of HFAs’ financial statements during site visits, including but not limited to the timeliness of financial reporting, corrections to reports after the reporting cycle, and supporting documentation for all categories of expenditures sampled during the review.

Roles and Responsibilities of Personnel Responsible for Monitoring

Treasury documents the offices that are responsible for receiving and reviewing monitoring materials, the deadlines for receiving this information, and the responsibilities of staff who execute internal control. This documentation is consistent with internal control standards, which state that management should implement control activities through policies, such as by documenting each unit’s internal control responsibilities. The standards also state that management should remediate identified internal control deficiencies on a timely basis, such as by having personnel report internal control issues through established reporting lines. Treasury’s policies and procedures document which offices are in charge of executing its monitoring procedures, such as collecting required documentation, conducting site visits, and evaluating HHF performance. Treasury informs HFAs of reporting lines to Treasury through phone calls and emails. Treasury and HFA staff also noted that they are in frequent contact with each other regarding administration of the program.

Validation of Implementing Partners’ Performance through Site Visits or Other Means of Verification

Treasury uses regular (at least biennial) site visits, biweekly calls with HFAs, and monthly informal performance updates as means of validating HFAs’ performance. These practices are consistent with OMB guidance, which states that a federal awarding agency may make site visits as warranted by program needs. Treasury uses its site visits to assess HFAs’ program implementation, conduct its own analyses of program results, review HFAs’ use of program funds, and review HFAs’ implementation of internal controls. According to Treasury staff, Treasury also uses site visits to corroborate the information HFAs report on their program performance and use of HHF funds.

According to HFAs with whom we met, site visits typically last multiple days and include entrance and exit conferences between Treasury and HFA staff. During site visits, Treasury staff review documentation related to homeowners and properties associated with the programs, quality assurance processes, antifraud procedures, information technology and data security, finances, and legal matters. After the site visit, Treasury issues a report documenting its observations. Within 30 days of receiving Treasury’s written report, HFAs are required to provide Treasury with a written response describing how they will address any issues of concern.

Procedures for Project Closeout

Treasury included some procedures for project closeout in HFAs’ participation agreements. Creating procedures for project closeout is consistent with OMB guidance, which states that agencies should close out federal awards when they determine that applicable administrative actions and all required work have been completed by the nonfederal entity. Participation agreements describe various procedures for closing out HHF programs, including requirements for the return of unexpended funds to Treasury and final reporting and provisions for reimbursement of expenses.

In addition, according to Treasury staff, Treasury is in the process of developing and issuing wind-down guidance for HFAs in stages to address specific areas of program activity. Agency officials also discussed winding down the HHF program during Treasury’s 2018 Annual Hardest Hit Fund Summit. The annual summit is a meeting that HFAs, servicers, and other stakeholders are invited to attend to facilitate information sharing among stakeholders involved in HHF. At the 2018 summit, the agency discussed topics that included final compliance and financial reviews, program change requests, operational timelines, and budgeting and staffing as they relate to the wind-down of HHF programs and operations. In addition, as states have begun to close some of their programs, Treasury has issued clarifying guidance to HFAs in order to effectively wind down the HHF program—including on streamlining the process for requesting changes to programs. Treasury staff also performed outreach to each HFA in April 2018 about their wind-down plans and, according to Treasury staff, the agency expects to prepare written guidelines for HFAs on certain other topics related to winding down the program, including reporting requirements, as appropriate.

Consideration of Performance Information in Making Management Decisions

Treasury uses performance information to assess whether HFAs are performing at a satisfactory level. This practice is consistent with internal control standards, which state that management should establish and operate monitoring activities to monitor the internal control system and evaluate results, which can include evaluating and documenting the results of ongoing monitoring and separate evaluations to identify internal control issues. In addition, management should remediate identified internal control deficiencies on a timely basis. This can entail management completing and documenting corrective actions to remediate internal control deficiencies on a timely basis. Treasury staff described the agency’s process of assessing HFAs’ performance as “holistic.” As a part of this process, Treasury staff review the targets HFAs set for assisting households or demolishing blighted properties and monitor HFAs’ utilization rates. According to Treasury staff, if performance and financial data suggest that an HFA is not making sufficient progress toward its performance targets or is drawing funds too slowly, Treasury collaborates with the HFA and the HFA must create a plan to improve its performance. If an HFA is not responsive to Treasury’s efforts, Treasury issues a performance memorandum requiring the HFA to create a plan to address its deficiencies. As of October 2018, Treasury had issued performance memorandums to seven HFAs—five in 2012 and two in 2015.

Additionally, as mentioned previously, Treasury issues a report to each HFA following each site visit describing any issues of concern Treasury identified. Treasury requires HFAs to provide the agency with a written response to the report within 30 days of the report date describing the HFA’s plan for addressing any deficiencies.

Communication with External Parties to Address Risks and Achieve Objectives

Treasury regularly communicates with HFAs, servicers, and other stakeholders interested in HHF, which is consistent with internal control standards that state management should externally communicate the necessary quality information to achieve the entity’s objectives. This can include communicating with, and obtaining quality information from, external parties using established reporting lines. According to Treasury staff, Treasury holds biweekly calls with HFAs and servicers, facilitates issue-specific working groups between HFAs and stakeholders, and holds an annual summit related to HHF. HFA staff said Treasury staff are very responsive to program-related questions. Treasury’s annual summit allows interested parties, such as HFAs, servicers, and other stakeholders, to discuss important issues related to HHF.

Treasury Partially Addressed Four Leading Practices

Identification, Evaluation, and Monitoring of Risks

To assist HFAs in designing their internal control activities, including defining program objectives, Treasury created an optional risk assessment matrix to help HFAs and their auditors identify and assess HFAs’ risks. The matrix includes control objectives and example control activities, and it allows HFAs to determine their risk tolerances for each control objective. For example, for the risk of improper use of administrative funds, the matrix includes “ensuring that appropriate documentation exists to support HHF administrative expenses” as a control objective, and it lists routine review of administrative payments by internal auditors as an example control activity. HFAs can identify their risk tolerances as low, medium, or high in the matrix. This matrix is consistent with federal internal control standards, which state that management should define objectives clearly to enable the identification of risks and define risk tolerances.

However, Treasury does not systematically collect or evaluate HFAs’ risk assessments. HFAs’ participation agreements require them to submit an annual certification of their internal control programs by an independent auditor to Treasury. According to Treasury staff, independent auditors sometimes choose to include HFAs’ risk assessments with the annual certification, and during site visits Treasury obtains documentation of HFAs’ internal control programs, which sometimes includes their risk assessments. Outside of these instances, Treasury does not routinely collect HFAs’ risk assessments. Further, in those instances when Treasury does collect them, it does not analyze the assessments to evaluate whether the risk levels are appropriate. While Treasury does a more in-depth evaluation of HFAs’ internal controls during site visits, this review does not include evaluating the appropriateness of the risk levels HFAs identified. For example, one of the risk assessment matrixes we reviewed listed the HFAs’ administrative expenses as low-risk despite this HFA having a history of alleged improper-payment related issues with its HHF program, which Treasury’s review would not have evaluated. Treasury officials told us that during site visits they may discuss the risk levels that HFAs determine, but Treasury has not asked or required any HFAs to change a risk level.

Failure to collect and evaluate HFAs’ risk assessments is inconsistent with an important practice for preventing fraud we have previously identified—monitoring and evaluating the effectiveness of preventive activities, including fraud risk assessments and the antifraud strategy, as well as controls to detect fraud and response efforts. Further, according to internal control standards, management should identify, analyze, and respond to risks related to achieving the defined objectives, and an oversight body may oversee management’s estimates of significance so that risk tolerances have been properly defined.

According to Treasury staff, the risk assessment matrixes are intended for use by HFAs and their independent auditors in preparing for the annual certification. They said that risk tolerances, or levels, are to be assigned by HFAs and their independent auditors, not by Treasury, and that it would be inappropriate for Treasury to interfere with their determination. However, agreed-upon procedures performed by HFAs’ independent auditors do not provide assurance or conclusion as to whether HFAs’ risk levels are appropriate. For example, in two agreed-upon procedures reports we reviewed, the auditors stated that the procedures performed were based on the HFAs’ risk matrixes, but they did not mention assessing whether the risk levels assigned to different controls were appropriate. Treasury staff also said that Treasury expands its sample size and criteria for specific programs or categories of expenses during a compliance review where repeated or significant observations have been previously found. However, by not collecting and evaluating HFAs’ risk assessments, Treasury limits its ability to monitor the effectiveness of HFAs’ preventive activities, controls to detect fraud, and response efforts. In addition, Treasury is missing an opportunity to help ensure that risk levels are appropriate.

Documentation That Monitoring Plans Were Executed

Treasury’s documentation of its efforts to monitor HFAs is consistent with internal control standards, which state that management should establish and operate activities to monitor the internal control system and evaluate results and remediate deficiencies on a timely basis. More specifically, the standards cite as characteristics of these principles that management evaluate and document the results of ongoing monitoring and separate evaluations to identify internal control issues, and determine appropriate corrective actions for internal control deficiencies on a timely basis. Treasury addresses these criteria by documenting its monitoring findings through site visit reports, as previously discussed. Treasury requires HFAs to provide the agency with a plan to address any issue described in the site visit report within 30 days. In addition, Treasury addresses these criteria by documenting HFAs’ responses and assessing whether the issue has been addressed at the next site visit. Furthermore, Treasury sets deadlines for and documents receipt of HFAs’ annual internal control certifications, quarterly financial and performance reports, and annual audited financial statements. When underperforming HFAs are not responsive to Treasury’s attempts to work with them to improve their performance, Treasury documents the issues it has found and requires the HFAs to create and submit a corrective plan.

Treasury also directs HFAs to establish and execute their own internal control system, but it does not require HFAs to consistently document which of their staff are responsible for internal control execution. HFAs were required to submit staffing information within 90 days of joining HHF. However, HFAs are not required to regularly update this information. Further, Treasury’s written procedures for reviewing HFAs’ internal control programs during site visits do not include reviewing documentation of which HFA staff are responsible for responding to or reporting internal control issues. These practices are inconsistent with standards for internal control, which state that management should establish an organizational structure, assign responsibility, and delegate authority to achieve the entity’s objectives. The standards also note that effective documentation can assist management’s design of internal control by establishing the “who, what, when, where, and why” of internal control execution.

We asked Treasury if it encouraged HFAs to document which personnel are in charge of executing internal control procedures. Treasury staff referred us to the initial requirement that HFAs submit staffing information within 90 days of joining HHF and stated that there is no requirement that HFAs update this information. Further, Treasury staff said that during site visits they interview key HFA staff who execute internal controls and document these interviews. However, this practice does not help ensure that HFAs consistently provided updated information to their staff about which of their staff are responsible for internal control execution. Without requiring HFAs to routinely update their documentation, particularly as HFAs are winding down their HHF programs and staff begin to turn over, Treasury cannot be assured that HFAs are keeping their staff updated about who is responsible for monitoring issues and internal control execution.

Development of Relevant Output and Outcome Performance Indicators

Treasury and HFAs created quantitative output and outcome measures to assess HFAs’ performance. For example, Treasury created utilization thresholds to help ensure HFAs spend their HHF funds in a timely manner. Also, HFAs created performance targets to estimate the number of homeowners they could assist (or blighted properties they could demolish) through HHF. These activities are consistent with an attribute of successful performance measures—specifically, that measures should have a numerical goal.

However, some of Treasury’s performance measures are not clearly stated, and Treasury did not create consistent methodologies for HFAs to use to assess the performance of their HHF programs. In our previous work on attributes of successful measures, we identified that measures should be clearly stated and that the name and definition should be consistent with the methodology used to calculate them. While Treasury provided HFAs with a data dictionary to describe the information HFAs are required to report, Treasury defined the term “unique applicants” in a manner that allows HFAs to count applicants differently, leading to inconsistencies in HFAs’ methodologies for calculating some performance measures. As discussed later in this report, Treasury also allowed and sometimes required HFAs to self-define some data elements.

Additionally, performance measures should indicate how well different organizational levels are achieving goals. However, Treasury did not design a consistent methodology for HFAs to use to develop targets for the number of homeowners and properties their HHF programs may assist, and as discussed later in this report, HFAs we interviewed used different methodologies. Because some of Treasury’s performance measures are not clearly stated and because Treasury did not design consistent methodologies for HFAs to use in setting targets, as HFAs close down their HHF programs, Treasury has a limited ability to compare performance across HFAs or aggregate these data to evaluate how well the HHF program as a whole is achieving its goals.

Documentation of Goals and Measures and Their Relationship to Program Outputs

Treasury created goals and measures to assess HHF performance, consistent with a practice we previously identified of creating performance goals and measures that address important dimensions of program performance and balance competing priorities. Treasury addressed this practice by creating utilization thresholds for HFAs and inserting them in HFAs’ participation agreements. Treasury also addressed this practice by documenting its performance measures, using standardized spreadsheets through which HFAs regularly report on outputs and outcomes related to the services provided to distressed homeowners.

However, Treasury has not explicitly documented the relationship between program outputs and the overall goals of the HHF program, and it does not generally require HFAs to establish intermediate goals unless the HFA has not met Treasury’s performance expectations. This is inconsistent with practices we previously identified relating to results- oriented performance goals and measures. Among these practices are including explanatory information on goals and measures in performance plans and using intermediate goals to show progress or contributions toward intended results. The main goals of HHF are to prevent foreclosures and stabilize housing markets. However, Treasury has not documented the relationship between many of the program outputs it tracks and the main goals of the HHF program. According to Treasury, the relationship between its outputs and the goals of HHF can be inferred through various memorandums and materials it issued when HHF was created. However, these documents do not explicitly explain the rationale for the use of these output measures to assess HHF’s ability to stabilize neighborhoods and prevent foreclosures. By not documenting the relationship between HHF’s program outputs and services and the overall goals of the HHF program or requiring all HFAs to set intermediate goals, Treasury missed the opportunity to more proactively articulate a results- oriented focus for the HHF program.

Most HFAs Have Met Thresholds for Withdrawing Funds, but Inconsistent Targets and Outcome Measures Limit the Assessment of Program Performance Most Homeowners Participating in HHF Were Assisted through Mortgage Payment Assistance Programs

As of December 2017, the 19 participating HFAs had 71 active HHF programs. Active HHF programs fall under one of six Treasury-defined program types: mortgage assistance, reinstatement, transition assistance, principal reduction, down payment assistance, and blight elimination. Participating HFAs may have implemented additional HHF programs, but these programs had either stopped disbursing funds or had not received a total allocation from Treasury at the time of our review. Individual HFAs may implement multiple programs—for example, the Mississippi HFA had two active programs, and the South Carolina HFA had five.

The most common type of HHF program as of December 2017 was mortgage assistance, as shown in table 1. All 19 HFAs had active mortgage payment assistance programs as of December 2017. In contrast, 3 HFAs had active transition assistance programs.

As of December 2017, we found that the 71 active HHF programs had assisted approximately 400,000 homeowners and demolished almost 24,000 blighted properties. According to Treasury data, the majority of homeowners who received HHF assistance participated in a mortgage payment assistance program. Treasury data also indicate that transition assistance programs assisted the smallest number of homeowners relative to other HHF program types (see table 2).

HHF programs of the same program type can vary in a number of ways, including eligibility criteria, length of time implemented, and number of homeowners assisted. Within each program type, HFAs designed programs that sometimes varied based on specific housing needs. For example, while both the Nevada and Florida HFAs had active reinstatement programs as of December 2017, these programs had different eligibility criteria. The Nevada HFA’s reinstatement program targeted low-to-moderate income homeowners who had fallen behind on their mortgages. The Florida HFA offered a similar reinstatement program for delinquent mortgages but also offered a program for senior homeowners who had fallen behind on property taxes and other fees.

HHF programs also varied by duration and the amounts of assistance provided as of December 2017. For instance, since all HFAs initially launched mortgage payment assistance programs at the beginning of HHF, these programs have been active for an average of 7 years. In contrast, HFAs began implementing down payment assistance programs in 2015. Additionally, the median amount of assistance provided varied by program type. According to analysis of Treasury data from 2010 through 2017, assistance ranged from a median amount of $4,000 per household for transition assistance programs to over $42,000 per household for principal reduction programs.

The HHF program is beginning to wind down. As of September 2018, Treasury had disbursed $9.1 billion of the $9.6 billion obligated under HHF. According to Treasury officials, although HFAs may continue issuing new approvals through December 31, 2020, most states have already begun to close down HHF programs or will do so by the end of 2018 as they exhaust their available funds. These include California and Florida, the two largest states in the program.

Most HFAs Have Met Thresholds for Withdrawing Funds from Treasury

According to Treasury officials, during the fifth round of funding Treasury established new conditions for HFAs, called utilization thresholds, to help maximize the use of the $2 billion in newly available funds. According to documentation from Treasury, if an HFA does not meet its utilization threshold, Treasury will reallocate a portion of the unused funds to HFAs that did. The amount reallocated to each HFA is determined by state population, the percentage of funds drawn by HFAs, and other factors.

The utilization thresholds for 2016 and 2017 were structured as follows:

2016. If an HFA did not draw at least 70 percent of its funding from rounds one through four by December 31, 2016, 50 percent of its round five funding would have been reallocated.

2017. If an HFA did not draw at least 95 percent of its funding from rounds one through four by December 31, 2017, 75 percent of its round five funding would have been reallocated.

Most HFAs have met Treasury’s 2016 and 2017 utilization thresholds. More specifically, the 18 HFAs eligible for round five funding met the 2016 utilization threshold. As a result, Treasury did not reallocate any HHF funds for that year. As of December 2017, 17 of the 18 HFAs eligible for round five funding met the 2017 utilization threshold. The Nevada HFA drew 70 percent of its funding for rounds one through four as of December 31, 2017, and therefore did not meet the 2017 utilization threshold. As a result, Treasury reallocated approximately $6.7 million of the Nevada HFA’s unused fifth round HHF funds to the 17 other HFAs.

As of September 2018, all HFAs had met the 2018 utilization threshold, and Treasury had disbursed most of the funds obligated under HHF. If an HFA did not draw at least 80 percent of its participation cap by December 31, 2018, an amount equal to the portion of round five funding that had not been drawn from Treasury would have been reallocated.

Data on the Extent to Which HHF Programs Met Targets Are of Limited Use Because Treasury Did Not Develop a Consistent Methodology for Calculating Targets

The targets that HFAs set are of limited use for evaluating the performance of individual programs, program types, HFAs, or the HHF program overall. In their participation agreements, HFAs were required to estimate the number of homeowners they intended to assist and, if they had a blight elimination program, the number of blighted properties they intended to demolish for each of their HHF programs. Treasury refers to these estimates as targets.

HFAs that we spoke with used different methodologies to calculate these targets. For instance, one of the HFAs we spoke to calculated targets for the number of homeowners they could assist by dividing the program’s total allocation by the average amount of assistance it anticipated awarding to each homeowner. In contrast, another HFA calculated its target for assisting homeowners by dividing that program’s total allocation by the maximum amount of assistance homeowners could be awarded through the program. According to Treasury staff, they did not develop a consistent methodology for HFAs to use in setting these targets because, in their view, HFAs are most familiar with local conditions and should have flexibility in adjusting the program criteria or creating new programs based on these conditions.

Internal control standards state that management should define objectives clearly to enable the identification of risks and define risk tolerances. In particular, the standards note the importance of stating measurable objectives in a form that permits reasonably consistent measurement. Further, our guide to designing evaluations states that where federal programs operate through multiple local public or private agencies, it is important that the data agencies collect are sufficiently consistent to permit aggregation nationwide, which allows evaluation of progress toward national goals. Because Treasury did not develop a consistent methodology for HFAs to use when setting performance targets, the targets HFAs developed do not permit consistent measurement of program performance or an evaluation of how well the HHF program as a whole met its goals. However, with the program beginning to wind down, any changes going forward would not improve the consistency of previously collected data or Treasury’s ability to evaluate the program as a whole.

Treasury Collects Information on Outcomes for Some HHF Programs, but This Information Is of Limited Use

Treasury Requires HFAs to Report Some Outcome Information for Four Program Types

Treasury collects quarterly data on outcomes from HFAs that implement four of the six HHF program types: mortgage payment assistance, principal reduction, reinstatement programs, and transition assistance programs. HFAs must track outcomes, both intended and unintended, until a household is no longer involved with an HHF program. Intended outcomes include, for example, the number of homeowners who completed or transitioned out of an HHF program as a result of regaining employment. Unintended outcomes include the number of homeowners who transitioned out of an HHF program into a foreclosure sale. The type of outcomes Treasury requires HFAs to track depends on the program type.

Treasury did not design outcome measures in a way that would permit it to use these data to evaluate whether HFAs or the overall program are achieving the stated goals. More specifically, Treasury officials told us that the data they collect on outcomes cannot be used to compare the outcomes achieved by different HFAs or through different HHF program types. According to Treasury officials, HFAs have historically had different interpretations of Treasury’s outcome measures. Treasury revised its template for HHF reporting in 2015 and 2017 to clarify certain performance-related terms. However, Treasury officials told us that conclusions drawn from HHF data on some outcomes are of limited use because HFAs interpret Treasury’s guidance on these data differently. Additionally, after it made revisions to guidance on performance reporting in 2015, Treasury allowed—and in some cases required—HFAs to self- define certain data elements. For example, Treasury required HFAs to define how they calculate the median principal forgiveness awarded by an HHF program.

As previously discussed, a key attribute of effective performance measurement is clearly stated performance measures with names and definitions that are consistent with the methodology used to calculate the measure. Additionally, we have noted in our guide to designing evaluations that a program’s outcomes signal the ultimate benefits achieved by a program and should be considered when evaluating a program. Further, OMB has set the expectation that agencies should conduct evaluations of federal programs. However, because Treasury did not clarify certain outcome measures until 5 years into the program, or take steps to ensure that HFAs calculated alternative outcomes consistently, even after Treasury clarified its reporting guidance, the alternative outcomes data that Treasury collects are of limited use for evaluating the performance of HFAs, HHF programs by program type, or the HHF program overall. As many programs are closing, further clarification or changes would not capture the full scope of the program and would not improve such evaluations.

Treasury Requires HFAs with Blight Elimination and Down Payment Assistance Programs to Conduct Impact Studies

Treasury requires HFAs with blight elimination and down-payment assistance programs to identify indicators that are intended to track and quantify the HHF program’s impact on targeted areas, although HFAs are not required to report outcomes data to Treasury in their quarterly performance reports for these program types. According to Treasury, blight elimination and down payment assistance programs are focused on stabilizing housing markets in targeted distressed areas to prevent foreclosures, and therefore they are not required to report individual-level outcomes for HFAs to report in quarterly performance reports. Treasury officials told us that the impact of these program types upon neighborhoods, such as increases in the values of properties in neighborhoods where down-payment assistance or blight elimination programs were used, may not be observable immediately but may appear over time. As of August 2018, four of eight HFAs with blight elimination programs had submitted impact studies to Treasury. Also, all HFAs with down payment assistance programs have submitted studies to Treasury.

Three blight elimination program impact studies suggest that the programs had positive impacts on targeted areas, although two of the studies have important limitations. Studies on the programs in Michigan and Ohio found that home prices increased in communities where blighted properties were demolished. For example, the Ohio study found there was about a 4-dollar increase in home values for every dollar spent on the HHF-funded blight elimination program. However, this study examined only 1 of the 18 counties that were served by the Ohio HFA’s blight elimination program. A study on the Illinois program found that certain key economic indicators had improved over a 6-year period in areas targeted by the program. For example, the percentage of negative equity mortgages in 9 of the 10 areas studied declined by an average of 7 percent between 2010 and 2016. However, the findings of this study do not isolate the independent effect of the Illinois HFA’s blight elimination program because other factors, such as local economic conditions, could also affect the performance of key economic indicators.

Stakeholders Identified a Variety of Challenges in Implementing HHF Programs

Treasury, HFAs, and Mortgage Servicers Described Challenges Related to Implementing Programs

HHF stakeholders with whom we spoke described challenges in implementing HHF programs related to staffing and multiple funding rounds, program implementation, outreach to borrowers, program accessibility, the variety of programs and their status, and external factors. Both Treasury staff with responsibilities for monitoring HFAs’ implementation of HHF and stakeholders told us that these were the types of topics discussed during regular phone calls and annual meetings. Stakeholders included staff from four HFAs that are implementing HHF programs, mortgage servicers and housing counseling agencies that are involved with HHF, and other interested organizations, including those that work with HFAs.

Staffing and multiple funding rounds. All four HFAs and various stakeholders with whom we spoke told us that staff turnover at HFAs presents challenges. In some cases, turnover has been related to the way the HHF program has been funded. For example, staff from two HFAs mentioned that either they let staff go or their temporary staff found more permanent positions as the agencies spent down their initial HHF funds. When Congress authorized Treasury to make additional TARP funds available to HHF beginning in 2016, these HFAs had to hire and train new staff. Treasury officials told us that many HFAs encountered staffing challenges as a result of the program’s fifth funding round. Additionally, staff from two servicers and an organization that advocates for HFAs told us that HFA turnover presents challenges because it takes time for new staff to become familiar with the program and for programs to ramp back up.

Program implementation. Staff from most of the HFAs and servicers with whom we spoke, as well as Treasury staff and other stakeholders, told us that implementation of the HHF program was challenging. Specific implementation challenges mentioned by HFAs included creating an in- house information system to manage HHF data; managing refinancing requests from homeowners who have been awarded HHF funds (to help ensure the HFA’s place as a lien-holder); and sharing information with servicers. While Treasury helped to develop a system to facilitate the sharing of loan-level information for the HHF program, one HFA and some servicers noted that the system has not always worked smoothly. Additionally, Treasury staff told us that a challenge HFAs are currently facing is the wind-down of the HHF program. They stated that HFAs must determine how they should advertise to the public, internal staff, and external partners that programs are closing; when they should stop accepting applications; and what resources are available for activities related to program closeout.

Outreach to homeowners. All four HFAs and an advocacy organization told us that it can be challenging to effectively reach eligible homeowners. As an example, staff from one HFA told us that housing counseling agencies have been an effective tool for making homeowners aware of HHF programs but that there are fewer foreclosure counselors available to homeowners now compared to when the HHF program started in 2010. Staff from an HFA that closed its HHF programs to new applicants after the initial funding rounds told us that it was challenging to communicate to the public, and therefore to potential clients, that its HHF programs were reopening after they received additional funding. Additionally, a representative of a nonprofit organization that works to address challenges in the mortgage market told us that many people did not know about the HHF program and that program information was hard for consumers to find on many states’ websites.

Program accessibility. According to academic research and two stakeholders (an advocacy group and a housing counseling agency), the accessibility of an HFA’s program can affect program participation. A 2014 study of Ohio’s HHF program found that the design of the program hampered accessibility and therefore program participation. The program was designed to require registrants (those who started the application process) to continue the application process by working with a housing counseling agency. The study found that registrants who lived within 5 miles of their assigned housing counseling agency submitted a complete application almost 32 percent of the time, while those who lived over 50 miles away submitted a complete application about 18 percent of the time. Similarly, a representative for an organization that advocates on behalf of low-income homeowners noted that the design of one state HHF program requires applicants to meet with specific housing counseling agencies to complete the application process. However, the housing counseling agencies to which applicants are assigned may not be nearby. The representative stated that in some cases, homeowners are assigned to a housing counseling agency that is located 3 or 4 hours away from where the homeowners live. According to the advocacy group representative, this design is particularly challenging for elderly homeowners who may have trouble applying online and need personal help.

Additionally, representatives for a housing counseling agency told us that their state HFA stopped involving community organizations to guide applicants throughout the application process once the HFA received additional HHF funding in 2016 and instead chose to work with applicants directly. They said this design may hurt homeowners who do not live near the HFA and would benefit from in-person assistance that could be provided close to their homes. A representative from the state’s HFA confirmed that the HFA decided to work directly with applicants once it received additional HHF funds in 2016. The representative stated that while homeowners could also apply for HHF assistance online (after the HFA changed the program design in 2016), the HFA’s system did not accept electronic signatures. Thus, homeowners without the ability to print and scan documents would need to come to the HFA’s office to complete the application process.

Variety of programs and their status. Treasury officials noted that the wide variety of programs that HFAs are implementing can create operational challenges for HFAs. As an example, the officials explained that HFAs may encounter challenges when their programs require coordination with local partners. For example, land banks can encounter delays in acquiring properties for demolition, and contractors may not do demolition work properly or may attempt to increase the amounts that they charge for their work after winning a contract.

Five mortgage servicers with whom we spoke described similar challenges. For example, representatives from one servicer told us that it was challenging to work with the 19 different HFAs because they all implemented different HHF programs. The representative added that it was particularly challenging if an HFA had a change in either leadership or points of contact for the HHF program. Another servicer explained that servicers have to review each HFA’s participation agreement and subsequent updates. This servicer noted that updates to agreements can create challenges, as the servicer needs to determine whether it can provide what the HFA is requesting. Representatives from this and a third servicer told us that it would have been helpful for servicers to have an up-to-date list of active HHF programs. Further, one servicer told us that it is challenging to help homeowners understand that each HFA and program has different requirements and guidelines. As previously discussed, Treasury communicates information to stakeholders, such as servicers, through regular conference calls. However, Treasury expects HFAs to keep their servicers abreast of the status of HHF programs because HFAs contract directly with servicers.

Representatives from one HFA noted that it was challenging to keep servicers updated on changes to their HHF programs. For example, they reported that when the HFA made changes to its unemployment program, servicers confused the program with another of the agency’s HHF programs. The representatives also stated that they have had to make many phone calls to try to keep servicers up to date.

External factors. Treasury officials and other stakeholders noted that external factors such as changing market needs and natural disasters have created challenges for some HFAs. Treasury officials noted that some HFAs have had to change their HHF programs over time to respond to changes in local housing conditions. An organization that advocates for HFAs as well as an HFA similarly noted that changing housing markets present challenges for HFAs, which have to adjust their program offerings in an effort to continue to serve homeowners. As previously discussed, HFAs must obtain Treasury approval to add or revise their HHF programs, and they must document the changes by amending participation agreements. Treasury officials also noted that natural disasters can affect HHF programs because HFAs have to turn their attention to post-disaster housing needs. Additionally, Treasury officials stated that after a natural disaster it can become difficult to verify the eligibility of applicants, particularly if key documents have been lost or communication channels with homeowners or servicers are affected.

Treasury and SIGTARP Also Identified Challenges through Their Monitoring and Oversight Activities

Through its on-site monitoring efforts, Treasury has identified issues that participating HFAs must address for their HHF programs. During on-site reviews in 2016 and 2017, Treasury staff assessed selected HFAs’ efforts in one or more Treasury-identified areas. As previously noted, Treasury’s policy at the time of our review was to conduct on-site reviews of each participating HFA at least once every 2 years. In 2016 Treasury conducted on-site monitoring visits for 14 HFAs and identified issues that the HFAs needed to address to improve their HHF programs. Issues Treasury identified primarily fell into two areas. The first of these was monitoring processes and internal controls—for example, Treasury found that one HFA had not developed documentation of its compliance procedures for a down payment assistance program. The other primary area was homeowner eligibility—for example, Treasury found that an HFA had misclassified the reasons that some homeowners were not admitted into the state’s HHF program.

In 2017 Treasury conducted site visits to 15 HFAs. For this period, Treasury’s most common issues related to homeowner eligibility and administrative expenses. According to Treasury officials, the increase in issues related to administrative expenses between 2016 and 2017 was a result of greater agency focus on this topic. Treasury observed, for example, that one HFA lacked sufficient documentation to support some administrative expenses and that another HFA had misclassified some administrative expenses. As previously discussed, HFAs are required to provide Treasury with a written plan describing how they will address issues Treasury identifies and reimburse HHF for any impermissible expenses.

Through its oversight activities, SIGTARP reported that some participating HFAs have encountered challenges related to appropriate use of administrative expenses, management of their programs, and blight removal. In August 2017, SIGTARP reported that participating HFAs used $3 million in HHF funds for unnecessary expenses. The report maintained that some HFAs were using their administrative funds for expenses that were unnecessary. In a May 2018 hearing, SIGTARP testified that some HFAs were not following federal cost principles related to administrative expenses. Additionally, SIGTARP has issued reports describing mismanagement of the HHF program by specific HFAs, as well as challenges related to blight removal. While Treasury has disagreed with the dollar amount of administrative expenses used inappropriately by HFAs, it has also worked with HFAs and SIGTARP to address SIGTARP’s findings.

Conclusions

As HHF programs begin to close and participating HFAs take steps to ensure they spend all of their HHF funds before the program deadline, opportunities exist in two areas for Treasury to manage risk and improve program operation and closeout:

By not consistently and routinely collecting HFAs’ risk assessments, Treasury limits its ability to monitor and evaluate the effectiveness of HFAs’ preventive activities, controls to detect fraud, and response efforts. Further, by not evaluating these risk assessments, Treasury is missing an opportunity to help ensure that risk levels are appropriate.

As HFAs wind down their HHF programs and HFA staff are relieved of their HHF-related positions, maintaining updated and accurate staffing information can help ensure that HFA staff are informed of who in their own offices is responsible for internal control execution.

Because Treasury did not implement the HHF program in a manner that is consistent with standards for program evaluation design we previously identified, the performance data that Treasury collects do not provide significant insights into the program’s effectiveness. More specifically, Treasury did not clearly state some of its performance measures; lacks documentation of the relationship between program outputs and overall goals; did not design consistent methodologies for HFAs to use in setting did not require participating HFAs to use consistent methodologies to calculate outcomes.

As a result, Treasury cannot aggregate key performance data or compare performance data across HFAs or HHF program types to demonstrate the results of the HHF program. As we have previously reported, OMB has set the expectation that agencies should conduct evaluations of federal programs. Moreover, our guide to designing evaluations states that where federal programs operate through multiple local public or private agencies, it important to ensure the data these agencies collect are sufficiently consistent to permit aggregation nationwide in order to evaluate progress toward national goals. Although HHF programs must stop disbursing funds by December 31, 2021, many of the programs have already ended or are in the process of winding down, making it too late for changes to Treasury’s approach to performance measurement to have a meaningful impact. However, we note that if Treasury were to extend the current program, as it did after Congress provided additional funding in 2015, or if Congress were to establish a similar program due to a future housing crisis, it would be useful at that time for Treasury to develop a program evaluation design that would allow the agency to assess overall program performance, as well as assess performance across HFAs and program types.

Recommendations for Executive Action

We are making the following two recommendations to Treasury: The Assistant Secretary for Financial Institutions should annually collect and evaluate HFAs’ risk assessments, which include HFAs’ risk levels. (Recommendation 1)

The Assistant Secretary for Financial Institutions should ensure that the documentation listing the HFA staff responsible for internal control execution is updated routinely. (Recommendation 2)

Agency Comments

We provided a draft of this report to Treasury for review and comment. In its comments, reproduced in appendix IV, Treasury agreed with our recommendations and stated that it has already taken steps toward addressing them by enhancing the existing review procedures for HFA’s risk assessments and staffing updates. Treasury also provided a technical comment, which we incorporated.

We are sending copies of this report to the appropriate congressional committees, the Secretary of the Treasury, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. We will make copies available to others upon request. The report will also be available at no charge on our website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-8678 or ortiza@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs are listed on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

The objectives of this report were to (1) determine the extent to which the Department of the Treasury’s (Treasury) monitoring of the Hardest Hit Fund (HHF) addresses leading practices for program oversight, (2) provide information on housing finance agencies’ (HFA) active programs and the status of HFAs’ progress toward program targets, and (3) describe challenges in implementing HHF programs that HFAs and others identified.

To determine the extent to which Treasury’s monitoring of HHF addresses leading practices for program oversight, we used a scorecard methodology to compare Treasury’s monitoring policies and procedures, as implemented by 2016, against leading practices for an effective monitoring framework. To create the framework, we reviewed key reports and guidance related to monitoring, oversight, and performance management. In particular we reviewed relevant leading practices from internal control standards; previous GAO work on results-oriented performance goals and measures, key attributes for successful performance measures, characteristics for successful hierarches of performance measures, and managing fraud risk; and Office of Management and Budget guidance on oversight. Although Treasury is not required to follow all of the guidance that we identified, we determined that the guidance describes practices that are helpful for creating an effective monitoring framework.

To select the practices for the scorecard, we focused on practices relevant to the structure of an oversight framework (including fraud risk); performance measures; goal setting; and communication with external parties. We reviewed key reports and guidance and then vetted our selected practices with stakeholders knowledgeable about performance measurement, design methodology, fraud risk, and the law. Based on this review and input, we consolidated identified practices into 14 leading practices to apply to Treasury’s monitoring framework.

We then assessed Treasury’s policies and procedures against the framework. Specifically, we reviewed the agencies’ documented policies and procedures, reviewed documentation of how Treasury followed its policies and procedures, conducted interviews with Treasury staff responsible for overseeing HHF, and interviewed stakeholders, such as mortgage servicers, about Treasury’s monitoring of HHF. We also interviewed staff from four HFAs about Treasury’s monitoring of their programs; we selected the HFAs based on their mix of HHF programs, proportion of HHF funds disbursed, and geographic diversity. We also took into account whether stakeholders indicated that an HFA’s implementation of the program was particularly successful or challenging. With regard to the documentation Treasury collects as part of its monitoring, we limited our review to its 2016 and 2017 monitoring activities, and we limited our review of Treasury’s written policies and procedures to those implemented from January 2016 to September 2018. Two analysts independently reviewed agency policies and procedures to determine whether the policies were consistent with the 14 identified leading practices. Any disagreements in the determinations were resolved through discussion or with a third party, including the General Counsel’s office. We categorized each practice as follows:

Addressed: Treasury’s policies and procedures reflect each component of the leading practice.

Partially addressed: Treasury’s policies and procedures reflect some but not all components of the leading practice.

Not addressed: Treasury’s policies and procedures do not reflect any of the components of the leading practice To describe active HHF programs and the status of HFAs’ progress toward program goals, we reviewed program documents, administered a data collection instrument, and spoke with officials at four HFAs (selected as previously described) and Treasury. We defined active programs as those that had a total allocation approved by Treasury and were accepting applications and still disbursing funds to households or blight elimination projects as of December 2017. In order to identify which programs were active, we developed, collected, and reviewed a questionnaire in which HFAs provided information on when each of their HHF programs started and stopped disbursing funds. For each of the 71 active programs we identified, we reviewed quarterly performance reports as of December 2017 to compile descriptive information such as program outputs and outcomes.

Through the review of program documentation and interviews with knowledgeable officials, we found that Treasury’s output data were sufficiently reliable for our description of homeowners assisted and properties demolished. We also found that the data Treasury collected from HFAs on program outcomes were not reliable for the purpose of summarizing alternative outcomes by HFA or by program type. Treasury officials noted that the conclusions that can be drawn from alternative outcome data are inherently limited, particularly for the purpose of making comparisons between HFAs or program types, due to HFAs interpreting certain outcome measures differently, among other factors. Additionally, by comparing Treasury’s outcome measures to leading practices, we found that their definitions were not clearly stated.

We also identified four studies on the impact of HHF blight elimination programs and reviewed them for reliable methodology. We determined that one of the four studies was not reliable for the purpose of assessing the impact of blight programs on targeted areas. Two of the three studies that we determined to be reliable had important limitations. One study examined 1 of the 18 counties that were served by that HFA’s blight elimination program. The other study did not isolate the independent effect of the HFA’s blight elimination program because other factors, such as local economic conditions, could also affect the performance of key economic indicators. We reviewed each HFA’s contract with Treasury as of December 2017 to identify each program’s target for assisting homeowners or demolishing blighted properties. Through comparison with internal control standards, we found that these targets were not reliable for the purpose of describing HFAs’ progress toward program goals because they were not stated in a form that permitted reasonably consistent measurement.

To describe the factors Treasury identified as challenges for the HHF program, we analyzed Treasury’s on-site compliance monitoring reports for 2016 and 2017. As a part of our analysis, we identified the HFAs that Treasury visited in 2016 and 2017 and the extent to which Treasury had observations related to five Treasury-identified areas: monitoring processes and internal controls, eligibility, program expenses and income, administrative expenses, and reporting.

We also interviewed key stakeholders regarding their views of challenges related to implementation of the HHF program, particularly since 2012. We discussed challenges with Treasury staff with responsibilities for monitoring HFAs’ implementation of the program; staff from four HFAs that are implementing HHF programs; six mortgage servicers that are involved with the HHF program; and two housing counseling agencies that are involved with the HHF program. For two of the HFAs with blight elimination programs, we conducted site visits to observe activities related to blight elimination. Additionally, we discussed challenges with other interested organizations, including an association for HFAs and an organization that brings together housing counselors, mortgage companies, investors, and other mortgage market participants to help address challenges in the mortgage market. Further, we reviewed reports issued by the Special Inspector General for the Troubled Asset Relief Program. We summarized the challenges that stakeholders described.

We conducted this performance audit from November 2017 through December 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Monitoring Scorecard

To determine the extent to which the Department of the Treasury’s (Treasury) policies and procedures for monitoring and oversight address leading monitoring practices, we identified factors for an effective monitoring framework based on a review of key reports and guidance and input from stakeholders knowledgeable about performance measurement, design methodology, fraud risk, and the law. To select the practices for the scorecard, we focused on factors relevant to the structure of an oversight framework (including fraud risk); performance measures; goal setting; and communication with external parties. We consolidated identified factors into 14 leading practices to apply to Treasury’s oversight and monitoring framework. See Table 3 for the 14 leading practices and their underlying factors.

Appendix III: Homeowners Assisted through the Hardest Hit Fund

As shown in table 4, housing finance agencies (HFA) were implementing from one to seven Hardest Hit Fund (HHF) programs (excluding blight programs) as of the fourth quarter of 2017. We included programs for which HFAs were disbursing funds to homeowners. As of December 2017, individual HFAs had assisted from 807 to 86,220 homeowners.

Eight HFAs were implementing active blight elimination programs as of December 2017, as shown in table 5. The number of blighted properties demolished by individual HFAs ranged from 0 to 13,925.

The Department of the Treasury’s 2017 utilization threshold requires that HFAs draw at least 95 percent of their HHF funding from rounds one through four by December 31, 2017 (see table 6). As of December 2017, 17 of 18 HFAs had drawn 95 percent or more of their funding from rounds one through four. The Nevada HFA had drawn 70 percent of its funding from rounds one through four.

Appendix IV: Comments from the Department of the Treasury

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Jill Naamane, Assistant Director; Lisa Moore, Analyst in Charge; Vida Awumey; Farrah Graham; John Karikari; Moira Lenox; Benjamin Licht; Dan Luo; John McGrail; Marc Molino; Jennifer Schwartz; Shannon Smith; Estelle Tsay-Huang; and Erin Villas made key contributions to this report.