Mortgage Financing: HUD Could Realize Additional Benefits from Its Mortgage Scorecard
Published: 2006-04-13T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Along with private mortgage providers, the Department of Housing and Urban Development's (HUD) Federal Housing Administration (FHA) has been impacted by technological advances that began in the mid-1990s and that have significantly affected the way the mortgage industry works. As a result, in 2004, FHA implemented Technology Open to Approved Lenders (TOTAL) Scorecard--an automated tool that evaluates the majority of new loans insured by FHA. However, questions have emerged about the effectiveness of TOTAL. Given these concerns, you asked GAO to evaluate the way the agency developed and uses this new tool. This report looks at (1) the reasonableness of FHA's approach to developing TOTAL and (2) the potential benefits to HUD of expanding its use of TOTAL.
section_title: What GAO Found
paragraphs: Some of the choices that FHA made during the development process could limit TOTAL's effectiveness, although overall the process was reasonable. Like the private sector, FHA and its contractor used many of the same variables, as well as an accepted modeling process, to develop TOTAL. However, the data that FHA and its contractors used to develop TOTAL were 12 years old by the time FHA implemented the scorecard, and the market has changed significantly since then. Also, FHA, among other things, (1) did not develop a formal plan for updating TOTAL on a regular basis; (2) did not include all the important variables that could help explain expected loan performance; and (3) selected a type of model that limits how the scorecard can be used. Despite potential problems with TOTAL, HUD could still see added benefits from it. As a result of TOTAL, FHA lenders and borrowers have seen two new benefits--less paperwork and more consistent underwriting decisions. However, FHA could gain additional benefits if, like private lenders and mortgage insurers, it put TOTAL to other uses. These uses include relying on TOTAL to help inform general management decision making, price products based on risk, and launch new products. Adopting these scorecard uses from the private sector could potentially generate three other benefits for FHA, including the ability to react to changes in the market, more control over its financial condition, and a broader customer base. Additionally, HUD's Government National Mortgage Association, a government corporation that guarantees securities of federally insured or guaranteed mortgage loans, could use credit scores that are used by TOTAL to help improve the transparency of the secondary mortgage market.

section_title: Letter
paragraphs: Since its inception in 1934, the Department of Housing and Urban  Development’s (HUD) Federal Housing Administration (FHA) has provided  mortgage insurance for nearly 33 million properties, often for low-income,  minority, and first-time homebuyers. Along with private mortgage  providers, FHA has been impacted by technological advances that began in  the mid-1990s and that have significantly affected the way the mortgage  industry works. Among the most important of these innovations are the  automated underwriting systems that mortgage providers now use to  process loan applications. With automated underwriting, lenders enter  information on potential borrowers into electronic systems that contain an  evaluative formula, or algorithm, called a scorecard. The scorecard uses a  variety of variables that include the borrower’s characteristics (credit score  and cash reserves, for example) and loan characteristics to calculate the  applicants’ creditworthiness.
In the mid-1990s, Freddie Mac and Fannie Mae developed the first  automated underwriting systems and scorecards—Freddie Mac’s Loan  Prospector and Fannie Mae’s Desktop Underwriter—that could be used to  evaluate applications for FHA-insured loans and inform FHA’s underwriting  standards. However, these two systems’ scorecards sometimes generated  conflicting results for the same borrower. In part because FHA did not have  access to these systems’ proprietary scorecards, the agency chose to  replace them with its own. In addition, HUD wanted to modernize its  processes and improve its delivery to its business partners. Between 1998  and 2004, FHA worked with HUD’s contractor, Unicon Research  Corporation, to develop and implement the Technology Open to Approved  Lenders (TOTAL) scorecard. Since 2004, FHA and its lenders have used  TOTAL to evaluate applications for FHA-insured loans and inform  underwriting standards.
Recently, questions have emerged about the effectiveness of TOTAL  Scorecard, as well as concerns that FHA has not fully explored all possible  uses of this new tool. Given these concerns, you asked us to evaluate the  way the agency developed and uses this new tool. This report looks at (1)  the reasonableness of FHA’s approach to developing TOTAL and (2) the  potential benefits to HUD of expanding its use of TOTAL.
To assess the reasonableness of FHA’s approach to developing TOTAL, we  reviewed agency documents and interviewed officials from HUD and  Unicon Research Corporation to determine (1) the process used to develop  TOTAL, (2) the reliability of the analysis used to evaluate it, and (3) the  methods FHA used to establish policies on cut points (i.e., the points of  separation within a population of mortgage scores that divide applications  that are accepted from those that are not). To assess the benefits to FHA of  expanding its use of TOTAL, we reviewed existing research on the uses and  benefits of scorecards and interviewed private sector companies,  academics, and HUD officials about these issues. We compared FHA’s use  of TOTAL with the private sector’s use of scorecards in order to determine  whether FHA could benefit from any private sector practices. We also  examined the extent to which opportunities exist for FHA to extend the use  of TOTAL, and the data it utilizes, throughout HUD by sharing information  with other HUD offices that could benefit from it. Appendix I contains  details of our scope and methodology, and appendix II contains  information on the products that lenders can underwrite with TOTAL. We  conducted our work in Washington, D.C., between April 2005 and February  2006 in accordance with generally accepted government auditing  standards.
subsections: section_title: Background
paragraphs: Congress established FHA in 1934 under the National Housing Act (Pub. L.  No. 73-479) to broaden homeownership, protect and shore up lending  institutions, and stimulate employment in the building industry. FHA’s  single-family programs insure private lenders against losses from borrower  defaults on mortgages that meet FHA criteria and that are made primarily  to low-income, minority, and first-time homebuyers of properties with one  to four housing units. In 2004, some 77.5 percent of FHA loans went to first- time homebuyers, and 35 percent of these loans went to minorities. FHA  insures most of its single-family mortgages under its Mutual Mortgage  Insurance Fund (MMI Fund), which is supported by borrowers’ insurance  premiums.
FHA insures a variety of mortgages that cover initial home purchases,  construction and rehabilitation, and refinancing. Its primary program is  Section 203(b), the agency’s standard product for single-family dwellings.  As the mortgage industry has developed products such as adjustable-rate  mortgages (ARM), FHA has followed suit and now insures ARMs on single- family properties. FHA insures a variety of refinancing products, including  mortgages designed to promote energy efficiency. Finally, it insures  specialty mortgages, such as the Hawaiian Home Lands mortgage, which  enables eligible native Hawaiians to obtain insurance for a mortgage on a  homestead lease granted by the Department of Hawaiian Home Lands.
Despite the products it insures, the number of loans FHA insures each year  has fallen dramatically since 2000, largely because lending for conventional  mortgage products (i.e., mortgages with no federal insurance or guarantee)  has grown much more rapidly since the late 1980s than mortgages insured  by government entities such as FHA and the Department of Veterans  Affairs. As conventional markets have grown, so has the private sector’s  use of automated underwriting systems, which has streamlined the  application process and allowed lenders to more quickly assess the risk of  loans. FHA began approving specific automated underwriting systems for  lenders in 1996 in an effort to streamline its manual underwriting process.  When it began delegating underwriting tasks to approved lenders in the  1980s, lenders manually underwrote loans before submitting the loan  applications and required documentation to an FHA field office for  approval. Once automated underwriting systems for FHA lending came  into use, “direct endorsement lenders” (i.e., lenders certified by HUD to  underwrite loans and determine their eligibility for FHA mortgage  insurance without obtaining prior review) could streamline the loan  application process by bypassing some documentation requirements.  According to FHA officials, automated underwriting has allowed FHA to  reduce the amount of time needed to approve insurance for a loan from  several days to 1 day.
The key to automated underwriting is a mortgage scorecard algorithm that  attempts to objectively measure the borrower’s risk of default quickly and  efficiently by examining the data that has been entered into the system. To  underwrite a loan, lenders first enter into the electronic system data such  as application information and credit scores. A scorecard compares these  data with specific underwriting criteria (e.g., cash reserves and credit  requirements) using a mathematical formula. Because the scorecard  electronically analyzes each variable, it can quickly predict the likelihood  of default. According to FHA officials, this process not only reduces  underwriting time but also decreases the amount of documentation needed  to assess the borrower’s credit risk.
Private mortgage insurers, such as United Guaranty and Mortgage  Guaranty Insurance Corporation (MGIC), were among the first to develop  mortgage scorecards in the early 1990s. Beginning in the mid-1990s,  Freddie Mac and Fannie Mae began to create their own automated  underwriting systems and scorecards to evaluate conventional loans for  purchase. More specifically, Freddie Mac implemented its Loan Prospector  automated underwriting and scorecard tool by 1996, and Fannie Mae  implemented a similar tool, Desktop Underwriter, in 1997. Experience  with these scorecards prompted Freddie Mac in 1998 and Fannie Mae in  1999 to develop versions of these scorecards for FHA that lenders first used  to automatically underwrite FHA-insured loans. Both entities used  performance data on FHA-insured loans as part of the loan data used to  create the FHA versions of their scorecards.
However, while FHA cooperated in the development of Freddie Mac’s and  Fannie Mae’s scorecards for FHA-insured loans, they were nonetheless  proprietary to those entities, and some important details (e.g., the  weighting of the variables) were withheld from FHA. In addition, the two  scorecards sometimes yielded contradictory results for the same borrower.  As a result, FHA decided to replace the Loan Prospector and Desktop  Underwriter scorecards and develop its own scorecard that would provide  uniform outcomes.
Between 1998 and 2004, FHA contracted with Unicon Research  Corporation to develop TOTAL. Direct endorsement lenders now use  TOTAL in conjunction with automated underwriting systems that meet  FHA standards—Loan Prospector, Desktop Underwriter, and Countrywide  Loan Underwriting Expert System (CLUES)—to determine the likelihood  of default. Although TOTAL can determine the credit risk of a borrower, it  does not reject a loan; FHA requires lenders to manually underwrite loans  that are not accepted by TOTAL to determine if the loan should be accepted  or rejected.
FHA’s automated mortgage underwriting process starts at the time that the  borrower meets with and submits information to the direct endorsement  lender for loan prequalification (see fig.1). First, the direct endorsement  lender enters the application variables, such as the applicant’s loan-to-value  ratio (LTV) and debt, into the automated underwriting system. Second,  the automated underwriting system electronically “pulls” the additional  credit data required to score the loan, which includes any bankruptcy and  foreclosure information and credit scores. Third, the automated  underwriting system transmits the data to TOTAL, which evaluates the  information and recommends whether the loan should be “referred” or  “accepted.” A “refer” recommendation requires that the direct endorsement  lender manually underwrite the loan. An “accept” recommendation means  that the loan does not have to be manually underwritten to determine the  borrower’s creditworthiness and, accordingly, that less documentation will  be required to process it. For example, borrowers whose loans are  accepted do not have to verify their employment history if they have  already met certain conditions, such as providing confirmation of current  employment. An accepted application must go through an additional series  of credit checks, or overrides, to ensure that it meets all of FHA’s  underwriting standards. If the loan does not pass the series of additional  credit checks, the application can still be downgraded to a “refer” for  manual underwriting. Once the loan is processed through the credit  checks, the automated underwriting system then sends the decision in a  feedback document that the lender uses to continue processing the loan  application.
section_title: FHA’s Approach to Developing TOTAL Was Generally Reasonable, but Some of Its Choices Could Limit TOTAL’s Effectiveness
paragraphs: FHA’s approach to developing TOTAL was generally reasonable, but some  of the decisions made during the development process could ultimately  limit the scorecard’s effectiveness. Like the private sector, FHA and its  contractor followed an accepted process, using a variety of variables that  took into account such items as credit history and economic conditions. As  a result, TOTAL is similar to private sector scorecards. But TOTAL’s  effectiveness could be limited by some of the choices that were made  during the development process, including the fact that (1) the data FHA  and its contractor used were 12 years old by the time TOTAL was  implemented, (2) FHA has not developed policies and procedures for  updating TOTAL, and (3) the benchmark analysis for determining TOTAL’s  predictive capability may have been inadequate.
subsections: section_title: The Process FHA and Its Contractors Used to Develop TOTAL Was Generally Reasonable
paragraphs: Scorecards are typically developed and maintained using data with specific  characteristics and an accepted modeling process. The data—such as,  variables that reflect credit histories and loan information—are typically  several years old and are drawn from samples of borrowers whose  characteristics resemble those of the borrowers whom the scorecard will  assess. The process used in the private sector to develop the scorecard  itself typically has four components:     identifying the variables that best predict the likelihood of default,   choosing a scorecard model by conducting various tests,   validating the scorecard to ensure that it is stable (i.e., consistently  produces reasonable results), and   determining the appropriate cut point for separating loans that will be  accepted from those that will be referred for manual underwriting.
Once the scorecard is complete, many private sector organizations plan for  and conduct ongoing analyses and generate reports to monitor and update  their scorecards. Analyses that help in updating scorecards include  measuring changes in the population of borrowers, the quality of the  portfolio, and the scorecard’s effectiveness. Organizations may conduct  these analyses on a monthly and quarterly basis, and they may also  supplement these analyses with more in-depth reviews.
In developing TOTAL, FHA’s contractor Unicon followed the four-step  process. First, it identified variables using data primarily for loans that FHA  had endorsed (i.e., approved for mortgage insurance) in 1992. In 1998,  when Unicon began developing TOTAL, FHA chose to use 1992 loan data,  which would reflect the characteristics of FHA borrowers and be  “seasoned,” or old enough, to provide a sufficient number of defaults that  could be attributed to a borrower’s poor creditworthiness. The 1992 sample  of endorsed loans included 9,867 loans that did not result in a claim default  and 4,818 that did. Unicon tested the variables’ ability to predict claim  default. Unicon determined that a number of variables, such as credit, LTV  ratio, and cash reserves should be included in TOTAL. To determine the  best type of credit variable for FHA’s purposes to include in TOTAL, Unicon  and its subcontractor Fair Isaac Corporation used 1994 and 1996 credit  data to test various credit models and confirm the results. These models  included those that measured borrowers’ credit using only credit scores  and more complex models that were based on individual credit  characteristics rather than on a credit score. Based on this analysis, FHA  decided that the standard FICO credit score was a reasonable credit  variable to include in the scorecard.
Second, Unicon tested various versions of statistical models suitable for  developing scorecards. These were variations on two types of models,  “logit” and “hazard.” Both models predict the probability of default based  on predictive variables that are weighted according to their statistical  importance, although the hazard model can predict default over multiple  time periods. FHA officials stated that, based on Unicon’s analyses, both  models’ predictive capability were about equal. FHA chose the logit model,  claiming that it was easier to implement and that its estimates were easier  to interpret.
Third, Unicon tested the stability of the model by estimating it against a  sample of loans from 1992 that had not been included in the original 1992  data. In addition, Unicon tested the model’s stability over time by checking  whether the determinants of defaults occurring within 2 years were similar  for the 1992 and 1994 application years. Both stability tests, according to  documents provided by FHA, suggested that the model did not materially  change over the 2-year period. In addition, FHA performed a benchmark  analysis by comparing the performance of TOTAL with previously used  scorecards—the FHA versions of Freddie Mac’s Loan Prospector and  Fannie Mae’s Desktop Underwriter—to determine the model’s precision.  According to documents provided by FHA, TOTAL slightly outperformed  the other scorecards.
Finally, FHA worked with Unicon, Freddie Mac, and Fannie Mae to  determine a cut point for TOTAL that would enable the agency to quickly  accept the majority of loan applications so that lenders could focus their  manual underwriting on the marginal, potentially riskier borrowers. This  cut point was based partly on a 1996 analysis that Freddie Mac, in  consultation with FHA, conducted on the version of the Loan Prospector  scorecard developed for FHA. According to HUD officials, it was also  consistent with cut points that had previously been used before TOTAL was  implemented. The current cut point allows the agency to accept 65 to 70  percent of the loan applications automatically and refer the remainder.
In a 2001 report, a consulting firm—KPMG LLP—that reviewed documents  relating to the development of TOTAL concluded that FHA adequately  supported most of its development decisions. The report focused on the  data used, the type of model selected, the determination of cut points, and  FHA’s benchmark analysis.
section_title: Some Development and Implementation Choices Could Limit TOTAL’s Effectiveness
paragraphs: Although FHA and its contractor used a reasonable and generally accepted  practice for developing TOTAL, some of the choices made during that  process could affect FHA’s ability to maximize its use of the scorecard.
subsections: section_title: Data Not Current
paragraphs: By the time TOTAL was implemented in 2004, the loans in the development  sample were 12 years old. Best practices call for scorecards to be based on  data that are representative of the current mortgage market—specifically,  relevant data that are no more than several years old. FHA officials told us  that the relationship between TOTAL’s predictive variables and FHA  borrowers’ tendency to default had not changed significantly since 1992  and that they believed the data were still useful. However, since 1992,  significant changes have occurred in the mortgage industry that have  affected the characteristics of those applying for FHA-insured loans. These  changes include generally lower credit scores, increased use of down  payment assistance, and new mortgage products that have allowed  borrowers who would previously have needed an FHA-insured loan to seek  conventional mortgages. As a result, the relationships between borrower  and loan characteristics and the likelihood of default may also have  changed. For example, the statistical relationship between the LTV ratio  and the likelihood of default may be different for borrowers who receive  down payment assistance than for those who do not.
section_title: No Plan for Regular Updates
paragraphs: As noted earlier, when TOTAL was implemented in 2004, FHA officials  believed that the 1992 loan sample used to develop the scorecard still  provided an adequate basis for assessing new loan applications. The  agency’s subsequent analyses of TOTAL using samples of FHA-insured  loans throughout the 1990s indicate that, for years tested, the scorecard has  performed consistently in separating loans that resulted in insurance  claims from those that did not. As a result, HUD did not update TOTAL  either before it was deployed or subsequently. However, best practices  implemented by private entities and reflected in guidance from a bank  regulator call for having formal policies to ensure that scorecards are  routinely updated. Frequent updating of scorecards ensures that they  reflect changes in consumer behaviors and thus continue to accurately  predict the likelihood of default. In September 2004, FHA awarded another  contract to Unicon to, among other things, update TOTAL by 2007. In  addition, HUD indicated that, through its contractors, it has the capacity to  update TOTAL should the need arise and has contracts for acquiring credit  data to support an update of the scorecard. However, FHA has not  developed policies and procedures for updating TOTAL on a regular basis.
section_title: Limited Sample of Loans Used for Development and Testing
paragraphs: Another potential shortcoming that could affect TOTAL’s effectiveness is  the fact that FHA used only endorsed loans to develop TOTAL. Because the  data did not cover all of the possible outcomes of applying for a loan  (rejection, for example), the results could be biased. Therefore, TOTAL will  likely assess a population of applications with generally poorer overall  credit quality than the original population used to develop the scorecard  and thus may not be as effective in evaluating applicants with poorer  credit. In addition, because the sample of loans that was used to develop  TOTAL differed from the total population of loan applications, the selection  and weighting of the variables in the scorecard could be less than optimal.  For the riskier applications, the predictive variables and associated  weightings might differ from those TOTAL currently uses. FHA officials  stated that, at the time TOTAL was being developed, they did not have  another choice in the data used. However, updating TOTAL using  information on marginal loans that were referred by the scorecard, but  ultimately endorsed for FHA insurance, could help mitigate the bias  problem.
Similarly, using cut points that were based only on endorsed loans at the  time TOTAL was developed—in this case, loans that were originated using  the Loan Prospector scorecard—could mean that a higher percentage of  loans that are likely to default would be accepted rather than referred for  manual underwriting. That is, a sample of endorsed loans does not include  loans that have been rejected and thus does not represent the total  population of loans. As previously noted, the current cut point allows FHA  to accept 65 to 70 percent of the total population of loan applications and  that percentage could include riskier loans—riskier loans that the sample  did not represent because they were referred by Loan Prospector and  ultimately rejected. Furthermore, because FHA’s selection of cut points  was not based on analysis of loans accepted by TOTAL, but rather on loans  accepted by Loan Prospector, the cut points may prove to be less useful for  FHA as it attempts to manage and understand its risk. KPMG LLP—the  consulting firm that reviewed TOTAL’s development in 2001—raised similar  concerns.
We also found that, similar to the sample of loans used to develop TOTAL,  the sample FHA used to perform the 1996 benchmark analysis of TOTAL  consisted only of endorsed loans, rather than a broader sample that  included the riskiest loans. Partly because other loan data were not readily  available, Unicon benchmarked TOTAL against a sample of loans  originated using the Loan Prospector scorecard. This sample consisted  primarily of loans that had been accepted by the scorecard and endorsed  for FHA insurance. However, because all models perform slightly  differently (i.e., each scorecard will mistakenly accept certain high-risk, or  “bad” loans), using a prescreened sample of loans could limit the accuracy  of the benchmark analysis. The potential effect on the benchmark  analysis was to suggest that TOTAL outperformed Loan Prospector.  However, using a sample of loans that had not been prescreened by Loan  Prospector might have yielded somewhat different results that would have  more accurately represented TOTAL’s predictive capabilities.
section_title: Excluded Important Variables
paragraphs: While TOTAL includes many of the variables included in other mortgage  scoring systems, it does not include a number of important variables  included in other systems. For example, the systems used by Fannie Mae  and Freddie Mac may assign higher risks to adjustable rate loans than to  fixed-rate loans. ARMs are generally considered to be higher risk than  otherwise comparable fixed-rate mortgages, because borrowers are  subject to higher payments if interest rates rise. Further, other scoring  systems often include indicators for property type (single-family detached,  two- to four-unit, or condominiums, for example). FHA indicated that these  variables were not included in TOTAL because the risk associated with  them did not differ significantly in the 1992 data used to estimate the  model. However, the 1992 data set was fairly small—fewer than 15,000  loans—and only about 16 percent of it consisted of ARMs. In addition,  condominiums and multiunit properties are a small component of FHA’s  business. The modeling effort may have failed to find significant effects for  these variables simply because of the small numbers of loans with these  characteristics in the development sample. Previous research by FHA  contractors on larger samples of FHA loans found that ARMs from this  period were riskier than comparable fixed-rate mortgages. The fact that  FHA’s scoring system does not consider the extra risk inherent in ARMs or  distinguish between different types of properties, while competitors’  systems do, could have important consequences. If marginal applications  that are ARMs or multiunit properties are rejected by competitors’ systems,  but accepted by FHA’s, then FHA’s share of these riskier loans may  increase. Finally, FHA does not include the source of the down payment in  its scorecard. However, research by HUD contractors, HUD’s Inspector  General, and us have all identified the source of a down payment as an  important indicator of risk, and the use of down payment assistance in the  FHA program has grown rapidly over the last 5 years. For example, as we  reported in November 2005, FHA-insured loans with down payment  assistance have higher delinquency and insurance claim rates than do  similar loans without such assistance.
section_title: Limited Logit Model
paragraphs: FHA chose a logit rather than a hazard model as a basis for TOTAL and,  therefore, potentially limited the variety of uses to which the scorecard can  be put. While a logit model predicts the probability of default for a specific  point in time, a hazard model, as previously noted, predicts the probability  of default over multiple time periods. Because a hazard model captures the  dynamic between time and loan performance, HUD could use it to project  cash flows over time and estimate profitability. In addition, a hazard model  more readily accepts and analyzes recent data, and FHA could update a  scorecard developed from this model with recent origination data as often  as it needs. Moreover, with a relatively current scorecard, FHA could  monitor market changes and TOTAL’s effectiveness at predicting defaults in  the current climate. Despite the added capabilities of a hazard model, FHA  officials stated that the logit model was sufficient for TOTAL’s intended  purpose because TOTAL was only intended to be used to rank order  applications for FHA-insured loans based on the likelihood of default.
section_title: HUD Could Benefit Significantly More from TOTAL
paragraphs: FHA uses TOTAL Scorecard in much the same way as its two earlier  scorecards—to inform underwriting standards and assess loan applications  against those standards. TOTAL has produced more consistent  underwriting results and, for some lenders, has streamlined the approval  process and reduced paperwork. Private sector organizations use their  scorecards more broadly, relying on them to assess risk, help launch new  products, and broaden their customer base, as well as updating them  regularly. FHA could realize similar types of benefits from TOTAL to help  the agency serve low- and moderate-income borrowers while ensuring its  financial soundness. In addition, the credit data used by TOTAL could help  to improve the transparency of the secondary market for FHA-insured  loans.
subsections: section_title: FHA Could Realize Additional Benefits Using TOTAL
paragraphs: FHA used TOTAL to test variables and identify the most predictive ones,  which the agency then used to inform its underwriting standards.  Therefore, TOTAL enables FHA to adjust its underwriting standards, if  needed, based on analyses of current market conditions—something that  Desktop Underwriter and Loan Prospector did not readily allow because  FHA did not have direct access to them. In addition, FHA directs lenders to  use TOTAL to assess loan applications by entering information that  corresponds to certain variables.As with the previous scorecards, the  only lenders that can directly interface with TOTAL and input loan  application data into the scorecard via automated underwriting systems are  direct endorsement lenders. Direct endorsement lenders can assess most  FHA loan products with TOTAL (see app. II).
As described in table 1, FHA’s current use of TOTAL has provided  additional benefits over previous scorecards, such as less paperwork for  lenders and more consistent underwriting decisions. Loan Prospector and  Desktop Underwriter had, among other things, helped speed up the  application process and provided an opportunity to base approvals on  objectively determined variables. TOTAL continues these benefits and, in  addition, has generated two others. First, as noted earlier, the previous  scorecards did not always provide consistent underwriting decisions—that  is, at times the results of their assessments differed, which resulted in the  same loan being accepted by one scorecard and referred by the other. As a  result, certain loans had to be approved manually, through potentially  subjective decision making. TOTAL limits the number of loans that need to  be approved manually because it provides consistent automatic  underwriting decisions. Second, lenders that use TOTAL do not have to  provide as much documentation for the accepted loans they underwrite as  lenders that do not use TOTAL. For example, these lenders do not have to  obtain or submit verification of rent, and the requirements for proof of  income employment and assets are less stringent.
section_title: Private Sector Organizations Benefit from Using Scorecards in a Variety of Ways
paragraphs: As noted earlier, the key to successfully using a scorecard is ensuring that it  is updated so that it can provide accurate and useful information. Updated  scorecards can provide a number of benefits because of the variety of  potential uses. Private sector organizations we spoke with said that their  scorecards had produced the same benefits as TOTAL, including reducing  loan origination times, and enhancing consistency and objectivity in the  underwriting process. In addition, private sector organizations use their  scorecards to help inform general management decision making, set prices  based on risk, and launch new products. To inform general management  decision making, private sector organizations compare the scorecards’  actual results with its predictions to, for example, set cut points and  redirect underwriting resources from relatively low-risk cases to more  marginal borrowers. To set risk-based prices, private sector organizations  use scorecards to rank the relative risk of borrowers and price products  according to that ranking. For instance, mortgage insurers may use FICO  scores as a basis for reducing insurance premiums for low-risk borrowers.  Finally, to help launch new products, these lenders may use scorecards to  balance risk and compensating factors. For example, a product with a more  flexible LTV could be offered to borrowers with characteristics such as a  strong credit history.
As a result of these uses, private lenders have been able to broaden their  customer base and improve their financial performance. Expanding their  product offerings based on a greater understanding of risk allows lenders  to broaden their customer base. Lenders told us that their scorecards had  allowed them to underwrite some borrowers who would have been  rejected using manual underwriting and to develop products to better serve  borrowers who were at a greater risk of default. One official noted that the  scorecard had provided a greater understanding of the individual  borrower’s risk and that, as a result, borrowers who would previously have  been considered for subprime loans were now rated at a higher level of  eligibility. In addition, lenders reported being able to reduce personnel  costs because the organizations were writing fewer loans manually.  Ultimately, these lenders said that they were able to maximize their profits  because of the streamlining and cost reductions the scorecards provided.
section_title: Implementing Private Sector Scorecard Practices Could Provide Additional Benefits for FHA
paragraphs: FHA could see additional benefits from TOTAL if it implemented some  private sector practices. By routinely monitoring and updating TOTAL, for  instance, FHA could better anticipate, understand, and react to changes in  the marketplace. FHA could also exercise more control over its financial  condition by using the scorecard to help (1) project estimated insurance  claims and adjust cut points and (2) institute its proposal for risk-based  pricing of the agency’s mortgage insurance products. FHA could also use  TOTAL to aid its efforts to develop new products for underserved  borrowers.
FHA could better anticipate, understand, and react to changes in the  marketplace if, like the private sector, it routinely updated TOTAL.
Updating the scorecard as new data become available could help ensure  that changes in consumer behavior are reflected in the model, which can be  affected by changes in products and other trends. By routinely comparing  the scorecard’s actual results to its predictions, FHA could ascertain  whether TOTAL was effectively predicting default risk and make any  necessary changes to the variables. In addition, FHA could use TOTAL to  more accurately determine the performance of new loans, which HUD  currently monitors on an ad hoc basis, to inform policy discussions on the  creation and revision of FHA products.
FHA could exercise more control over its financial condition, specifically  its credit subsidy costs and financial soundness, by using the scorecard’s  default predictions to project estimated claims and adjust cut points if  necessary. In order to project estimated insurance claims, FHA would  need to combine the variables’ weights estimated in the scorecard  development process with projections of interest and house price  appreciation rates, as is done in FHA’s actuarial studies. Based on its  projections, FHA could then determine how much risk it could or should  tolerate and make adjustments, if necessary, to the cut points and thus to  the numbers and types of loans it automatically accepted and referred for  manual underwriting. For example, if FHA raised the cut point, TOTAL  would accept fewer high-risk loans (i.e., loans more likely to result in an  insurance claim), thereby lowering FHA’s claim rate. Conversely, by  lowering the cut point, TOTAL would accept more high-risk loans, and the  agency would experience a higher claim rate.
TOTAL could also aid HUD’s efforts to implement risk-based pricing of its  mortgage insurance products. In its fiscal year 2007 budget submission,  HUD proposed legislation that would allow the agency to replace its  current insurance premium structure, where most borrowers pay the same  premium regardless of their default risk, to a risk-based structure where  borrowers would pay higher or lower premiums depending on their default  risk. HUD believes that risk-based pricing would allow the agency to  charge more competitive mortgage insurance premiums, attract and retain  relatively low-risk borrowers, and exercise more control over its credit  subsidy costs. HUD plans to set premiums based on an assessment of  borrowers’ credit histories, LTVs, and debt-to-income ratios. However, it  has not fully explored the potential of using TOTAL—especially a version  that includes additional variables, such as down payment assistance— which is capable of evaluating risk in a more comprehensive way, for this  purpose.
In its budget submissions for fiscal years 2006 and 2007, HUD also  proposed legislative changes that would allow FHA to develop new  mortgage insurance products for low- and moderate-income borrowers  (loans with lower down payment requirements, for example). HUD  believes that its traditional customers would be better served by these new  products than some of the high-cost, nonprime products offered in the  conventional market. To the extent that FHA develops these products, it  could use TOTAL to help identify alternatives that it previously may have  believed posed too much risk, given the expected profit, when its lenders  manually underwrote loans.
section_title: Providing Data Used by TOTAL Could Offer Additional Benefits to Ginnie Mae
paragraphs: HUD’s Ginnie Mae—which guarantees the timely payment of principal and  interest on securities issued by private institutions and backed by pools of  federally insured or guaranteed mortgage loans—could benefit from the  credit data used by TOTAL. As we reported in October 2005, Ginnie Mae  has taken steps to disclose more information to investors about the FHA- insured loans that back the securities it guarantees. However, unlike  many conventional securitizers, Ginnie Mae does not disclose credit  information—for example, summarized credit score data—for its loan  pools. Disclosing such information is important because investors can use  it to more accurately model prepayment rates. According to a Ginnie Mae  official, prior to the implementation of TOTAL in 2004, the credit scores  associated with FHA-insured loans were not available within HUD.  Because borrowers’ credit scores are used by TOTAL, Ginnie Mae has  expressed interest in obtaining this information and summarizing it for  investors.
section_title: Conclusions
paragraphs: Although FHA has helped to provide financing for nearly 33 million  properties, its share of the single-family market has steadily decreased over  time. Many of these potential borrowers—typically, first-time homebuyers  with minimal cash for down payments and lower than average credit  scores—may have been lost to conventional lenders. These lenders have  been, in part, able to provide conventional mortgages to these borrowers  with the increased use of scorecards—the evaluative component of  automated underwriting systems—that have enabled them to target the  traditional FHA borrower that poses the least amount of risk. If that is the  case, the effect on FHA is that it has started to serve more high-risk  borrowers. To enhance its understanding of risk posed by its borrowers,  FHA has adopted automated underwriting and developed its own  scorecard.
FHA followed an accepted process in developing TOTAL and has already  seen significant benefits from the scorecard. Because TOTAL has the same  types of capabilities as private sector scorecards, FHA has the option to  use and benefit from TOTAL in many different ways as do private sector  organizations. Specifically, FHA could use TOTAL to help compete in the  marketplace, manage risk, and serve its mission for borrowers. TOTAL’s  capabilities are important to FHA, in part, because as it begins to insure  more inherently risky loans, such as loans with down payment assistance,  it needs to understand the risks they pose to the FHA insurance fund and  manage those risks.
However, the potential benefits of TOTAL cannot be realized without  ensuring that TOTAL is regularly updated and exploring additional uses of  TOTAL. For example, by not developing and implementing policies and  procedures for rountinely updating TOTAL, it may become less reliable  and, therefore, less effective at predicting defaults. In addition, as a result  of not exploring additional uses of TOTAL, FHA will not receive all of the  types of benefits seen by private sector organizations. These additional  uses include applying TOTAL to proposed initiatives—such as risk-based  pricing and the development of new products—which may help strengthen  the FHA insurance fund and reach additional borrowers. Finally, FHA has  not taken steps to share credit scores utilized by TOTAL with Ginnie Mae,  which could use the information to help improve the transparency of the  secondary mortgage market.
section_title: Recommendations for Executive Action
paragraphs: To improve how HUD uses and benefits from TOTAL, we recommend that  the Secretary of HUD take the following two actions:  develop policies and procedures for updating TOTAL on a regular basis,  including using updated data, testing additional variables, exploring  hazard model benefits, and testing other cut points; and  explore additional uses of TOTAL and the credit data it utilizes,  including to help adjust cut points, implement risk-based pricing,  develop new products, and enable Ginnie Mae to disclose more  information about securities backed by FHA-insured loans.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided HUD with a draft of this report for review and comment. HUD  provided comments in a letter from the Assistant Secretary for Housing- Federal Housing Commissioner (see app. III). HUD made two general  observations about the report and provided specific comments on our  recommendations. First, HUD said the report did not convey the fact that  developing TOTAL was a HUD initiative to modernize its processes and  improve its delivery to business partners. Our draft report did discuss  HUD’s rationale for implementing TOTAL and the scorecards that preceded  it. It also discussed the benefits of these scorecards to FHA lenders,  including less paperwork and quicker approval of mortgage insurance.  However, in response to HUD’s comments, we added language to the report  that further describes HUD’s motivation for developing TOTAL.
Second, HUD said that TOTAL was working exactly as envisioned (i.e.,  segregating loans requiring limited underwriting and documentation from  those requiring a full review by an individual underwriter) and that the  draft report presented no evidence that the scorecard had failed to perform  as expected. HUD also indicated that the agency had provided us with  information and analysis based on FHA loan data from the 1990s, showing  that TOTAL performed well in separating loans that resulted in insurance  claims from those that did not. Our draft report did not state or intend to  suggest that TOTAL was not fulfilling its intended function or was not  working as well as expected. In fact, the report pointed out that TOTAL had  continued the benefits of previous scorecards while generating others. At  the same time, our draft report identified opportunities for HUD to improve  TOTAL so that it could become a more effective tool for assessing and  managing risk. For example, HUD could improve TOTAL by updating it to  reflect recent changes in the mortgage market, such as the substantial  growth in the percentage of FHA-insured loans with down payment  assistance.
HUD did not explicitly agree or disagree with our recommendation that it  should develop policies and procedures for updating TOTAL, including  using updated data, testing additional variables, exploring hazard model  benefits, and testing other cut points. HUD indicated that it was taking  steps to address some aspects of our recommendation but not others, as  follows:   HUD said that it had a formal plan for updating TOTAL, access to  TOTAL’s development and implementation contractors to accommodate  updates should the need arise, and contracts for acquiring credit data to  support an update of the scorecard. As our draft report discussed, HUD  had a contract to update TOTAL by 2007. However, best practices  implemented by private entities and reflected in guidance from a bank  regulator call for having formal policies to ensure that scorecards are  routinely updated. HUD’s current plan calls for one update to be  completed by 2007 (7 years after HUD finalized the scorecard model)  and has no provision for subsequent updates. Accordingly, we continue  to believe that HUD should develop policies and procedures for  updating TOTAL on a regular basis.
HUD acknowledged that it had used 1992 data to develop TOTAL but  stated that the data spanned a wide range of credit scores and  application factors represented in greater or lesser numbers in later  cohorts of loans. We disagree that the 1992 loan data sufficiently  represents later cohorts of loans and thus continue to believe that HUD  should use more current loan data to update TOTAL. As our draft report  stated, significant changes have occurred in the mortgage industry since  1992 that have affected the characteristics of those applying for FHA- insured loans. These changes include generally lower credit scores,  increased use of down payment assistance, and new mortgage products  that have allowed borrowers who would have previously needed an  FHA-insured loan to seek conventional mortgages.
HUD said that in developing TOTAL, the agency and Unicon tested all  the available variables and included those that were empirically  important, consistent with Equal Credit Opportunity Act (ECOA)  regulations (which, among other things, set forth rules for evaluating  credit applications). HUD also said that it intends to re-analyze all  available variables, including, as our draft report suggested, the source  and amount of down payment assistance. We agree that HUD should re- analyze all available variables and incorporate them into TOTAL,  consistent with ECOA requirements. Our draft report stated that HUD’s  analysis of certain variables, such as loan and property type, may not  have found significant effects simply because of the small numbers of  loans in HUD’s sample that were ARMs or were for condominiums or  multiunit properties. HUD could conduct future analyses with greater  statistical reliability if it were to use larger samples of loans, as major  private lending organizations do.
HUD stated that because TOTAL was designed to assess the  creditworthiness of borrowers, the logit model was sufficient for that  purpose. However, HUD also acknowledged that a hazard model could  be used for the purposes enumerated in our draft report. Accordingly,  we continue to believe that HUD should explore the benefits of a hazard  model.
HUD said that it did not rely solely on a 1992 sample of loans in setting a  cut point for TOTAL and that it worked with Unicon, Fannie Mae, and  Freddie Mac, using recent distributions of loans, to obtain a cut point  that was consistent with the ones already in use for FHA lending. Our  draft report did not state that HUD relied solely on a 1992 sample of  loans. Rather, it indicated that the cut point was based partly on a 1996  analysis that Freddie Mac performed in consultation with FHA.  However, in response to this comment, we added additional language to  the report describing how HUD determined the cut point. HUD did not  address the fundamental issue raised in our draft report—that the  limitations of its original analysis suggest that the agency should test  additional cut points. We continue to believe that HUD should test other  cut points based on analysis of loans accepted by TOTAL.
HUD did not explicitly agree with our recommendation that it should  explore additional uses of TOTAL, such as using it to help adjust cut points,  implement risk-based pricing, develop new products, and enable Ginnie  Mae to disclose more information about securities backed by FHA-insured  loans. However, the actions HUD said it plans to take are consistent with  our recommendation. Specifically,   HUD said that while TOTAL was not intended for risk-based pricing, the  agency planned to explore how TOTAL might be used for that purpose.
HUD stated that it planned to determine the benefits that TOTAL could  present in developing new products, if given the authority from  Congress.
HUD said that it was exploring the legal ramifications of giving Ginnie  Mae the credit scores obtained using TOTAL. HUD also provided a  technical correction, which we addressed in our final report, concerning  how it stores these credit scores.
Finally, HUD stated that the draft report contained several errors and that  these errors had been previously pointed out in meetings with us. Where  appropriate, we made technical corrections and clarifications in response  to HUD’s written comments and comments provided by a HUD official at a  March 2006 meeting to discuss our findings. However, we found that many  of these comments, rather than correcting any errors, merely provided  additional levels of detail that were unnecessary for the purpose of this  report.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  date of this letter. At that time, we will send copies to the Chairman and  Ranking Member of the Senate Committee on Banking, Housing, and Urban  Affairs; the Chairman and Ranking Member of the House Committee on  Financial Services; and the Ranking Member of the Subcommittee on  Housing and Community Opportunity. We also will send copies to the  Secretary of Housing and Urban Development and other interested parties  and make copies available to others upon request. In addition, this report  will be available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-8678 or shearw@gao.gov. Contact points for our Office of  Congressional Relations and Public Affairs may be found on the last page  of this report. Key contributors to this report are listed in appendix IV.
section_title: Scope and Methodology
paragraphs: To assess the reasonableness of the Federal Housing Administration’s  (FHA) approach to developing Technology Open to Approved Lenders  (TOTAL), we reviewed agency documents and interviewed the Department  of Housing and Urban Development (HUD) and contractor officials to  determine (1) the process and data used to develop TOTAL, including how  FHA identified and evaluated scorecard variables; (2) the reliability of the  analysis used to evaluate TOTAL’s effectiveness in predicting defaults; and  (3) how FHA established policies on cut points and overrides. In addition,  we reviewed industry literature and interviewed private sector officials  from large (based on volume) lending and private mortgage insurance  organizations to determine the extent to which FHA’s development of  TOTAL is consistent with private sector practices.
To assess the benefits to FHA of expanding its use of TOTAL, we reviewed  existing research on the uses and benefits of scorecards and interviewed  private sector companies, academics, and HUD officials about these issues.  We also determined how FHA and lenders use TOTAL by reviewing  relevant agency guidance and reports and interviewing FHA officials and  private lenders. In doing this work, we looked for any ways that FHA and  lenders are using TOTAL differently than the scorecards TOTAL replaced.  We compared FHA’s use of TOTAL with the private sector’s use of  scorecards and determined whether FHA could benefit from any private  sector practices that it has not already adopted. We also identified any  opportunities that may exist for FHA to share information with other HUD  offices that could benefit from TOTAL.
We conducted our work in Washington, D.C., between April 2005 and  February 2006 in accordance with generally accepted government auditing  standards.
section_title: Products That Lenders Can Underwrite with TOTAL
section_title: Comments from the Department of Housing and Urban Development
section_title: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Steve Westley, Assistant  Director; Triana Bash; Austin Kelly; Mamesho MacCaulay; John McGrail;  Mitch Rachlis; Rachel Seid; and Grant Turner made key contributions to  this report.