Small Business Administration: New Service for Lender Oversight Reflects Some Best Practices, but Strategy for Use Lags Behind
Published: 2004-06-08T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Small Business Administration (SBA) has been challenged in the past in developing a lender oversight capability and a loan monitoring system to facilitate its oversight. While SBA has made progress in its lender oversight program, its past efforts to develop a loan monitoring system were unsuccessful. In 2003, SBA obtained loan monitoring services from Dun & Bradstreet. GAO evaluated SBA's loan monitoring needs, how well those needs are met by the new service, and the similarities and differences for the purposes of credit risk management between SBA and private sector best practices.
section_title: What GAO Found
paragraphs: Largely because SBA relies on lenders to make the loans it guarantees, the agency needs a loan and lender monitoring capability that will enable it to efficiently and effectively analyze its overall portfolio of loans, its individual lenders, and their portfolios of loans. SBA, along with Dun & Bradstreet, essentially identified these same needs as they obtained the loan monitoring service. In addition, they identified the importance of applying industry standards and best practices for loan and lender monitoring and the need to identify high-risk lenders. Based on our assessment of best practices, SBA's credit risk management efforts need to include a comprehensive infrastructure, appropriate methodologies, and policies. The loan monitoring service could enable SBA to conduct the type of monitoring and analyses typical of best practices among banks and recommended by financial institution regulators, if SBA develops and implements appropriate policies. SBA's newly obtained service provides a credit risk management infrastructure and methodology that appear to be on par with those of many private sector lenders. For example, the database affords analytical capabilities based on common financial models that are used by major financial institutions. Although SBA obtained a useful service, it does not have comprehensive policies needed to implement best practices and address its needs as an agency with a public mission, especially regarding its need to use enforcement actions to address noncompliance. In addition, SBA does not have a contingency plan in the event the Dun & Bradstreet service is discontinued. SBA, similar to private lenders, must determine the level of risk it will tolerate, but it must do so within the context of its mission and its programs' structures, which may consequently translate into different uses of its Dun & Bradstreet loan monitoring service. Since SBA is a public agency with a public mission, its mission obligations will drive its credit risk management policies. For example, different loan products in the 7(a) program have different levels of guarantees, and guarantees on 504 program loans have a different structure from 7(a) guarantees. These differences influence the mix of loans in SBA's portfolio and, consequently, would impact how SBA manages its credit risk. Furthermore, the structure of SBA's loan guarantee programs may also result in different credit risk management policies between SBA and major lenders. Private sector lenders manage credit risk at the loan level and the portfolio level. Since SBA relies on private lenders to originate and service the majority of the loans it guarantees, it also needs to manage the credit risk in its portfolio at the lender level.

section_title: Letter
paragraphs: In fiscal year 2003, private lenders reportedly made more than 57,000 loans  totaling almost $12 billion to small businesses through the Small Business  Administration’s (SBA) two major loan guarantee programs. These loans  are made to businesses for operating capital and other purposes under  SBA’s 7(a) program and for fixed assets under its 504 program. SBA  guarantees varying portions of these loans, depending on the loan program  and loan product, although the majority (75 percent) was approved by  banks and other private financial entities under authority delegated by  SBA. To efficiently and effectively carry out its mission of maintaining and  strengthening the nation’s economy by guaranteeing loans in an effort to  help small businesses create jobs, SBA must monitor its overall portfolio of  loans, its individual lenders, and their portfolios. At the end of fiscal year  2003, SBA’s portfolio of business loans totaled $45 billion. Our past work  documented that SBA has not had a successful lender monitoring program  or a loan monitoring system. From 1998 to 2001, at a cost of $9.6 million,  SBA attempted to improve its monitoring by independently developing its  own loan monitoring system. These efforts failed in part because the  agency did not plan properly. And in 2003, partly based on congressional  action to cut funding of its loan monitoring system, SBA awarded a  contract to Dun & Bradstreet to enable the agency to better monitor its  portfolio, its individual lenders, and their portfolios. In this report, we refer  to the loan monitoring service provided under the contract with Dun &  Bradstreet as “Dun & Bradstreet service” or “loan monitoring service.”
Due to the importance of acquiring a loan monitoring service and an  effective set of policies for its use, you asked us to review the agency’s  acquisition and use of the new Dun & Bradstreet service. Specifically, you  asked us to determine (1) SBA’s loan portfolio monitoring needs, (2) how  well the newly obtained Dun & Bradstreet service meets SBA’s loan  portfolio monitoring needs, and (3) the major differences and similarities  for the purposes of credit risk management between SBA and private  sector best practices.
To determine SBA’s loan portfolio monitoring needs, we reviewed and  analyzed agency documents, and discussed related issues with agency and  industry officials and contractor staff. In addition we analyzed SBA’s  intended purposes for the Dun & Bradstreet service. Furthermore, we  identified applicable industry best practices and federal guidance to banks  for loan portfolio monitoring. To determine how well the new Dun &  Bradstreet service meets SBA’s needs, we reviewed and analyzed agency  documents, and conducted interviews with agency officials and contractor  staff. We also analyzed the Dun & Bradstreet deliverables and the  capabilities of the Dun & Bradstreet service, as well as SBA’s use and  planned use of the service. To determine the major similarities and  differences between SBA and private sector best practices for the purposes  of credit risk management, we interviewed selected major small business  lenders and federal banking regulators. We conducted our work in  Washington, D.C., between August 2003 and May 2004 in accordance with  generally accepted government auditing standards. Appendix I contains a  full description of our objectives, scope, and methodology.
subsections: section_title: Background
paragraphs: In pursuing its mission of aiding small businesses, SBA provides small  businesses with access to credit, primarily by guaranteeing loans through  its 7(a) and 504 loan programs. SBA has a total credit portfolio of $45  billion, the majority of which consists of 7(a) and 504 loans. The 7(a) Loan  Program is intended to serve small business borrowers who could not  otherwise obtain credit under suitable terms and conditions from the  private sector without an SBA guarantee. Under the program, SBA provides  guarantees of up to 85 percent on loans made by participating lenders— often called certified or preferred lenders, which are subject to program oversight by SBA. Loan proceeds can be used for most business purposes,  including working capital, equipment, furniture and fixtures, land and  buildings, leasehold improvements, and debt refinancing. The 504 loan  program provides long-term, fixed-rate financing to small businesses for  expansion or modernization, primarily of real estate. The 504 financing is  delivered through Certified Development Companies (CDC), about 270  typically preexisting private nonprofit corporations, established to  contribute to the economic development of their communities. For a  typical 504 loan project, at least 10 percent of the loan proceeds are  provided by the borrower, at least 50 percent by an unguaranteed third- party lender loan, and the remainder by an SBA-guaranteed debenture  from a CDC. Although SBA’s 7(a) and 504 loan programs serve different  needs, both programs rely on third parties to originate loan guarantees  (participating lenders for 7(a) and CDCs for 504 loans). Because SBA  guarantees up to 85 percent of the 7(a) loans and 40 percent of 504 loan  projects, there is risk to SBA similar to that of a lender if the loans it makes  are not repaid.
Loan portfolio management (monitoring) is the process by which risks that  are inherent in the credit process (primarily credit risk) are managed and  controlled. Current best practices emphasize an understanding of (1) the  risk posed by each loan and (2) how the risks of individual loans and  portfolios are interrelated. To address individual credit risk, best-practice  lenders focus on controlling the quality of individual loans approved and  carefully monitoring loan performance over time. These efforts encompass  such activities as specifying underwriting criteria, analyzing financial data  at loan origination, maintaining loan documentation, routinely reviewing  loan performance, and monitoring the financial condition of the borrower.  Managing a loan portfolio to consider portfolio concentration risks—which  can result from concentration of loans in, for example, a particular  industry—requires a more holistic view. Here, better technology and  information systems have opened the door to better management methods.  Today’s loan portfolio managers frequently use software tools to identify  interrelationships among loans and rank risk within a portfolio. The goal is  to obtain early indications of increasing risk. Together, these two  conceptual approaches—an individual and an aggregate view of risk—form  the foundation of modern loan portfolio management.
The Small Business Programs Improvement Act of 1996 required SBA to  establish a risk management database that would provide timely and  accurate information to identify loan underwriting, collections, recovery,  and liquidation problems. In its fiscal year 1998 budget request, SBA  presented plans for increased reliance on lenders to service and liquidate  defaulted small business loans. SBA planned to use the new database to  manage its loan portfolios, identify and effectively mitigate risks incurred  through loans guaranteed by SBA, implement oversight of internal and  external operations, and calculate subsidy rates.
We reviewed SBA’s plans to develop its loan monitoring system and  reported that SBA had not undertaken the essential planning needed to  develop the proposed system. Congress subsequently enacted provisions in  the Small Business Reauthorization Act of 1997 that directed the agency to  complete certain necessary planning activities that would serve as the basis  for funding the development and implementation of its loan monitoring  system. From 1998 to 2001, SBA’s estimate for implementing the system  grew from $17.3 million to $44.6 million. By 2001, SBA had spent $9.6  million for developmental activities but had never completed the mandated  planning activities or developed a functioning loan monitoring system. We  have periodically reported on SBA’s progress in planning and developing the loan monitoring system since 1997. In 2001, Congress stopped  appropriating funds for the loan monitoring system and instead authorized  SBA to use reprogrammed funds, provided that SBA notify Congress in  advance of SBA’s use of the reprogrammed funds. Congress also directed  SBA to develop a project plan to serve as a basis for future funding and  oversight of the loan monitoring system. As a result, SBA suspended the  loan monitoring system development effort. Of the $32 million  appropriated for the loan monitoring system effort, about $14.7 million  remained and was deposited with the General Services Administration’s  (GSA) Federal Systems Integration and Management Center Program  (FEDSIM). In January 2002, SBA contracted for assistance to identify  alternatives and provide recommendations for further developing a loan  monitoring system. As a result, SBA chartered a loan monitoring system  project management board with overall leadership and responsibility for  the vision, direction, and results of the loan monitoring system effort. This  board subsequently made the decision to no longer pursue the  development of a loan monitoring system, and in February 2003, SBA,  through FEDSIM, prepared a task order request for loan management  services. A contract was awarded to Dun & Bradstreet in April 2003 to  obtain loan management services, including loan and lender monitoring  and evaluation and risk management tools; the contract includes four one- year options at an average cost of approximately $2 million a year.
Prior to contracting for the Dun & Bradstreet loan monitoring service, SBA  had made progress in developing its lender oversight program for 7(a)  lenders with the establishment of the Office of Lender Oversight (OLO)— the office within SBA that is charged with ensuring consistent and  appropriate supervision of its lending partners, with the development of  written guidance in the form of “Standard Operating Procedures” and  “Loan Policy and Program Oversight Guide for Lender Reviews,” and  through conducting reviews. However, our 2002 study of SBA’s preferred  lender review process found that it involved only a cursory review of  lenders’ processes rather than a qualitative assessment of their decisions  with regard to borrowers’ creditworthiness and eligibility. Preferred  lender reviews were not designed to evaluate future financial risk.
SBA’s preferred lender reviews were set up as strict compliance reviews  and were not designed to measure the lenders’ future financial risk. Lender  reviews were based on reviewers’ findings using a questionnaire and a  review checklist. Recent changes related to these reviews are discussed in  this report. As participants in the 7(a) program, SBLCs are subject to the  same review requirements as other 7(a) lenders, in addition to the required  safety and soundness reviews. We have made recommendations calling on  SBA to clarify its supervisory and enforcement powers over 7(a) lenders  since November 2000. Further, CDCs are subject to the same lender  reviews as those required by 7(a) lenders. As with SBLCs, SBA provides the  only oversight currently required for CDCs; therefore, lender oversight for  both SBLCs and CDCs is especially important in order for SBA to monitor  the risk they pose to the agency. In February 2003, SBA’s Office of Inspector  General (OIG) recommended that SBA develop separate review  procedures for the oversight of the 504 loan program and that the review  process be both a financial and a compliance review. SBA responded that a  redesigned approach to CDC lender reviews was under way.
While elements of SBA’s oversight program touched on the financial risk  posed by preferred lenders, including SBLCs, based on historical  information, weaknesses in the program limited SBA’s ability to focus on,  and respond to, current and future financial risk to the lenders’ portfolio. In  the past, neither the lender review process nor SBA’s off-site monitoring  efforts adequately focused on the financial risk posed by preferred lenders  to SBA. Previously, SBA used loan performance benchmarking and ad hoc  portfolio analysis as its primary tools for off-site monitoring. SBA officials  stated that loan performance benchmarks are based on financial risk and  serve as a measure to address a lender’s potential risk to the SBA portfolio.
section_title: Loan and Lender Monitoring Capability Is Necessary for SBA to Conduct Effective Portfolio and Lender Oversight
paragraphs: As SBA’s reliance on lenders to originate 7(a) and 504 loans has grown, so  has SBA’s need for an effective method to monitor its portfolio and its  individual lenders’ performances. A credit risk loan and lender monitoring  system—based on industry best practices for infrastructure,  methodologies, and policies—would be an effective way to address credit  risk in the SBA portfolio and to facilitate the oversight of SBA’s lending  partners. Although SBA has not articulated its specific information and  analytical requirements needed to monitor credit risk, it has over several  years developed some general requirements for its loan monitoring needs.  Based on our assessment of best practices and our understanding of SBA’s  oversight and programmatic responsibilities, SBA needs a credit risk loan  and lender monitoring service that will enable the agency to efficiently and  effectively analyze various aspects of its overall portfolio, its individual  lenders, and their portfolios. Although specific credit risk management  practices may differ among banks, depending on the nature and complexity  of their credit activities, a bank’s credit risk management program will  likely include a comprehensive infrastructure, appropriate methodologies,  and policies.
subsections: section_title: Continued Efforts within SBA Have Yielded General Requirements for Its Loan Monitoring Needs
paragraphs: Although SBA recognized the need for a credit risk loan and lender  monitoring system and tried for years to build a system, SBA did not  specify the information and analytical requirements to meet its needs. In its  request for proposals to obtain loan management services, SBA officials  stated that they did not include a needs assessment because they did not  want to dictate the solution to be provided but to have vendors bring  innovative risk management solutions to SBA. However, SBA reported in  its fiscal year 2003-2008 strategic plan that, in general, it planned to allocate  resources for a loan monitoring capability to provide effective oversight of  its portfolio, its lending partners, and their portfolios in its 7(a) and 504  loan programs. In April 2003, SBA contracted with Dun & Bradstreet, which  worked in conjunction with Fair Isaac, to obtain such services. In the  interim, SBA collaborated with Dun & Bradstreet to identify more specific  requirements. According to the statement of work prepared by FEDSIM,  SBA wanted a loan monitoring capability that would apply monitoring and  evaluation services to existing loan data, apply industry standards and best  practices for loan and lender monitoring, and enable SBA to identify high- risk lenders. These requirements applied to both the 7(a) loan program and  the 504 loan program.
section_title: SBA’s Loan Monitoring Capability Should Be Based on Industry Best Practices for Infrastructure, Methodologies, and Policies
paragraphs: Based on our analysis of guidance published by financial regulators and  on interviews with risk management professionals, it would be appropriate  for SBA’s loan monitoring capability to be based on best practices for  infrastructure, methodologies, and policies. Figure 1 illustrates this  concept. The Office of the Comptroller of the Currency (OCC), the federal  regulator of national banks, requires regulated lenders to practice basic  loan portfolio monitoring/risk management. However, OCC notes that the  sophistication of an institution’s risk management policies and processes  will depend on the size of the institution, the complexity of its portfolio,  and the types of credit risks it has assumed. Accordingly, no single credit  risk rating system is ideal for every bank. In practice, a bank’s risk rating  system should reflect the complexity of its lending activities and the overall  level of risk involved.
Despite customization of risk management systems, financial regulators  and practitioners we spoke with are in general agreement about the  characteristics associated with effective credit risk management. Similar to  private lenders that focus on individual loans and their overall portfolio,  SBA must monitor its overall portfolio, its individual lenders, and their  portfolios. As such, it is important for SBA to have an effective monitoring  capability based on best-practice infrastructure, methodologies, and  policies.
subsections: section_title: Infrastructure
paragraphs: The infrastructure comprises the elements within an effective monitoring  system that makes the methodologies and policies work. Financial  regulators report that an infrastructure based on best practices will consist  of skilled personnel who are well-trained and properly motivated with the  ability to make professional judgments based on complex analytical data;  strong management information systems that provide accurate, timely,  complete, consistent, and relevant information; and functioning internal  controls related to data quality.  SBA has been especially challenged, and  did not succeed, in creating a loan monitoring management information  system on its own.
section_title: Methodologies
paragraphs: Best-practice methodologies refer to the application of analytic models to  measure credit risk. Financial institution regulators agree that internal risk  rating systems are becoming increasingly important in credit risk  management at large banks in the United States and are an essential  ingredient in effective credit risk management. They also agree that  methodologies based on best practices will consist of the following  elements:   sound statistical and financial modeling assumptions;   scenario approaches such as (1) back testing to see if the models’  projected default probabilities or expected loss rates are largely  confirmed by experience and (2) stress testing to see how loan  performance is affected by changes in one or more financial, structural,  or economic variables; and  concentration management techniques.
section_title: Policies
paragraphs: Policies based on best practices will consist of the establishment of a risk  management function consistent with the nature, size, and complexity of  the portfolio. According to financial regulators and practitioners,  successful risk management functions work under the guidance of a clear  credit strategy and risk profile (i.e., an institution’s tolerance for risk)  established by senior management. Policies and procedures also help staff  apply the institution’s credit strategy in a consistent manner to help ensure  that management’s risk profile objectives are met. Standard management  reporting—such as various forms of segmentation (i.e., various data  analyses based on variables such as geography, industry, and loan type),  trend, and purchase/default rate analyses—is one such element within the  policy framework, which facilitates compliance with management’s  objective of a clear and transparent credit strategy and risk profile. Risk  management professionals we talked with meet frequently, often weekly or  monthly, in order to review these standard management reports and to  discuss their action plans. Further, policies should be in place to ensure  risk management information systems are continuously updated in an ever- changing business environment and internal controls are enforced to  ensure that exceptions to policies and procedures are reported and  handled appropriately in a timely manner.
Together, infrastructure, methodologies, and policies form the foundation  of a best-practices risk management framework, as illustrated in figure 1.  The sophistication of the individual framework components varies and is  correlated with the complexity and risk profile of the portfolio. The goal is  to understand and manage credit risk such that a reasonable risk-adjusted  profit is generated, or in SBA’s case, to ensure compliance with its program  goals while staying within its congressionally approved budget. Table 1  describes these credit risk management best practices in more detail.
section_title: The Dun & Bradstreet Loan Monitoring Service Appears to Provide Appropriate Infrastructure and Methodologies, but SBA’s Lack of Comprehensive Policies Could Hamper Effective Oversight
paragraphs: Combined with appropriate SBA policies, the Dun & Bradstreet service  could enable the agency to conduct the type of monitoring and analyses  typical among major lenders and recommended by financial regulators.  SBA now has access to a risk management infrastructure and methodology  that appear to have characteristics similar to those of many private sector  lenders, including a functioning Web-accessible “data mart” that will  provide the agency with the information necessary to manage its loan  portfolio. Furthermore, the Dun & Bradstreet service provides SBA with an  independent risk management team of contractor staff dedicated to  managing the service and associated portfolio analysis. Although SBA has  obtained a useful service, it does not yet have comprehensive policies on  par with industry best practices to support the loan monitoring service.  SBA has implemented certain key elements, such as an internal risk rating  system, but it has not yet adopted other critical policy-related best  practices. The policies, for example, should set explicit risk limits and  steps to take when the limits are violated.
subsections: section_title: The Dun & Bradstreet Service Appears to Provide an Infrastructure and Methodology on Par with Best Practices
paragraphs: The loan monitoring service SBA obtained under contract from Dun &  Bradstreet includes an infrastructure that appears to be on par with best  practices, including a strong management information system, quality data,  and human capital. The comprehensive data mart hosted by Dun &  Bradstreet, referred to as RAM (Risk Assessment Manager), is a password- protected, Web-accessible data mart that SBA staff can query at any time.  The sources for the RAM data are SBA’s 7(a) and 504 databases, Dun &  Bradstreet corporate information, and commercial scoring data (e.g., Small  Business Predictive Score (SBPS) and Financial Stress Score (FSS)).   Each month, SBA staff electronically send Dun & Bradstreet updated loan  data files. After Dun & Bradstreet staff process the SBA loan data, they add  the corporate and scoring data, which are updated quarterly.
Ensuring the integrity of data used in the RAM is critical to the value of the  loan monitoring service and is considered a best practice. Routine quality  control and reconciliation processes are fundamental to ensuring data  integrity. We analyzed the processes SBA, Dun & Bradstreet, and Fair Isaac  have to manage the integrity of data associated with the service. We found  through our own testing and other analyses that SBA’s controls to ensure  the integrity of both the 7(a) and the 504 program data appear reasonable,  as a whole, to ensure that misstatements or inaccuracies are detected and  corrected on a timely basis. These controls were adequate to help ensure  the quality of the underlying SBA data used in the data mart. Although we  did not test the Dun & Bradstreet and Fair Isaac’s processes for data  quality, we reviewed their established procedures for data integrity and  found them generally reasonable. Appendix II contains a full discussion of  our review of data integrity.
There are several contractor staff that manage and assist SBA staff with  using the loan monitoring service. SBA has a risk management team within  the Office of Lender Oversight (OLO) dedicated to managing the Dun &  Bradstreet contract as part of its lender oversight responsibilities.  Furthermore, SBA can contact Dun & Bradstreet staff to fulfill ad hoc  analysis requests and for consultation regarding best practices. The Dun &  Bradstreet staff also provide SBA with monthly status reports about the  progress of their obligations under the contract and current trends in best  practices related to the small business lending industry.
Similar to the loan monitoring service infrastructure, the associated  methodology appears to be consistent with private sector best practices  since it appears to be based on sound financial models. The financial  models used to score the loans and lenders are based on data managed by  Dun & Bradstreet and commercial-off-the-shelf risk scoring models  developed by Fair Isaac. Dun & Bradstreet has over 160 years of data  management experience, including current relationships with over 90  percent of the top 1,000 companies worldwide, whereas Fair Isaac has over  50 years of experience as the leading provider of financial services  analytics. Fair Isaac’s suite of solutions is used by 22 of the top 25 U.S.  small business lenders. Fair Isaac conducts statistical analysis on its  products, including stress testing during its model development.
In addition to using the widely used statistical and financial models, Dun &  Bradstreet and Fair Isaac conduct continuous process improvement  through back testing to ensure that the models are working correctly for  SBA. The modeling and SBPS and FSS scores undergo evaluation on a  regular basis, including analyses to determine whether the models predict  outcomes in a stable manner as the population of loans changes (called  population stability) and loan characteristics change (called character  analysis). These analyses and reports can help determine when the models  require redevelopment to maintain accurate predictive risk information.  Since SBA is solely dependent on the Dun & Bradstreet service to provide  them with infrastructure and methodologies consistent with best practices,  without the service it is unlikely, at this time, that SBA would be able to  continue the same level of risk management of its overall portfolio, its  individual lenders, and their portfolios.
section_title: SBA Does Not Have Comprehensive Policies for Its New Loan Monitoring Capability on Par with Industry Best Practices
paragraphs: Unlike best practices, SBA has not fully developed or implemented  comprehensive loan monitoring-related policies and procedures to improve  its lender oversight. However, SBA has implemented certain key elements  of policy-related best practices.  For instance, SBA established a risk  management function when it created the Office of Lender Oversight in  1999. In addition, SBA officials have implemented an internal risk rating  process (i.e., lender rankings) and receive standard quarterly reports, or  tools, provided by Dun & Bradstreet. According to SBA’s own broad time  line for developing policy related to the new loan monitoring capability,  while some key oversight standard operating procedures are scheduled to  be completed by September 2004, its policies will remain incomplete until  at least April 30, 2005, about 1.5 years after Dun & Bradstreet began  providing its service to SBA in September 2003. Comprehensive policies  based on best practices would enable the agency to effectively carry out its  public mission, especially regarding its need to address any findings of  noncompliance with enforcement actions.
SBA has, through the Dun & Bradstreet service, an internal risk rating  process that includes lender rankings and associated risk scoring. Dun &  Bradstreet ranks SBA lenders each quarter based on their risk level. To do  this, Dun & Bradstreet consolidates each lender’s loans and then scores, or  quantifies, the risk by calculating the projected purchase rate (i.e., the price  SBA pays a lender for a loan when a borrower defaults on the loan and SBA  determines the lender has complied with the loan program requirements)  for each loan portfolio against the total SBA dollars at risk. Subsequently,  Dun & Bradstreet staff rank lenders for review based on their score. On  September 30, 2003, Dun & Bradstreet provided OLO with the first round of  lender rankings.
Dun & Bradstreet staff also provide SBA with standard lender performance  reports each quarter. These reports are based on profiles Dun & Bradstreet  staff develop of each loan and lender portfolio. These include high-level  profiling, such as demographic profiles and segmentation profiling and  analysis. The lender-level profiling also includes aggregating each loan  portfolio into lender portfolios and comparing lenders based on high-level  performance analysis and reporting. The variables used to do this include  dollar value of loans, distribution of 90-plus days past due by SBPS, average  SBPS, and dollars at risk.
However, SBA falls short on other key elements of policy-related best  practices. Best practices dictate the need for a clear and transparent  understanding of how a risk management service and the tools it provides  will be used. Comprehensive policies are fundamental to developing and  implementing a shared understanding of tools associated with the Dun &  Bradstreet service. Best practices state that agency stakeholders should  meet frequently and routinely to review the loan portfolios and the  resulting analyses, and discussion should occur within the context of the  comprehensive policies, notably the institution’s credit strategy and risk  profile. According to major-lender officials, internal stakeholders  (companywide) meet at least once a month to analyze and review the  standard management reporting packages to understand the major trends  within the portfolio and identify possible policies that need to be revised or  adopted to ensure they are consistent with the credit strategy and risk  profile. At SBA, according to OLO officials, agencywide stakeholders meet  periodically to discuss overall portfolio performance trends. These  portfolio reviews, often occurring monthly, incorporate the quarterly Dun  & Bradstreet reports, and according to SBA officials, additional internal  SBA management reporting in their discussions. This process of meeting  routinely to review standardized reporting is consistent with major-lender  best practices, although SBA’s lack of a clear credit strategy and risk profile  may impact the efficacy of this portfolio review process.
Additionally, SBA states in its fiscal year 2005 Performance Plan that it will  continue to use and enhance its new loan monitoring capability to improve  financial accountability and management, to improve the content of and  processes involving the agency’s financial statements, and the subsidy  models used for estimating the cost of SBA’s loan programs. Although  selected offices within the agency currently receive monthly portfolio  management reporting and analytics, including quarterly Dun & Bradstreet  reports, stakeholders agencywide do not yet routinely use Dun &  Bradstreet reports to support their mission activities. For example, the  Chief Financial Officer’s (CFO) office, which is one of the offices that does  not routinely use these reports, may benefit from the data and analytic  capabilities provided by the Dun & Bradstreet service in fulfilling its budget  and financial management responsibilities. In addition, other offices might  use performance reports to better inform SBA district office staff about  specific lender activity in order to enhance their outreach efforts to both  businesses and lenders and their technical support services to businesses.  For example, performance reports could be used to monitor lending to  special groups of eligible small businesses like veterans, Native Americans,  women, and disadvantaged businesses.
Although SBA recognizes that it needs to revise its lender review process, it  has yet to fully implement a review process that enables it to ensure that its  lending partners are complying with agency regulations and policies and  that it has found any prospective financial risks. In 2003, the agency  planned to begin conducting new strategic on-site operational reviews with  those lenders whose risk profiles indicate a high level of financial risk to  the agency. SBA reviewers intend to assess a lender’s SBA origination,  servicing, and liquidation practices. These risk-based reviews should  provide the SBA with better information to both improve lender loan  management processes and SBA loan programs, as well as develop useful  information regarding lender and portfolio risk. In a related effort, the  agency performance plan has a goal to expand its safety and soundness  examinations of certain state-chartered nondepository financial entities.  SBA officials stated that there are only a small number of these entities  making 7(a) loans and that these entities are currently overseen by state  regulators. The SBA Administrator testified in February 2004 that the new  loan monitoring capability, coupled with a redesigned lender review  process, would result in a risk-based approach to oversight, providing the  agency with more meaningful information about SBA’s lenders.   According to the Administrator’s testimony, the approach would also be  more streamlined and efficient, allowing SBA to better deploy resources in  areas where the agency has the most exposure, while being less intrusive to  the lenders. Pilot testing of the new review process began in May 2003.
Tables 2 and 3 compare SBA’s credit risk management capability to key  elements of best practices. SBA relied solely on Dun & Bradstreet to  provide the infrastructure and methodologies consistent with best  practices. The service, which is owned and operated by Dun & Bradstreet,  provides SBA with many key best-practice elements, including a strong  management information system based on apparent sound statistical and  financial models. Although the Dun & Bradstreet service is consistent with  key elements of best practices associated with infrastructure and  methodologies, without contingency plans SBA would not have the  capability on its own to duplicate the loan monitoring service. SBA officials  shared general ideas about what they might be able to do without the Dun  & Bradstreet service, but they have no specific contingency plans.  Moreover, while SBA has incorporated selected best-practice policies, such  as a functioning internal risk rating system and more frequent and relevant  standardized risk management reporting, the agency has yet to develop a  clear credit strategy and risk profile for its credit portfolio or to define  enforcement actions against its lenders in cases of noncompliance.
section_title: SBA’s Mission and Loan Program Structure Would Affect Its Use of Credit Risk Management Tools
paragraphs: SBA, similar to private lenders, must determine the level of risk it will  tolerate but do so within the context of the public purposes of its loan  guarantee programs, their budget constraints, and their structures.   Nevertheless, many private sector risk management best practices are  relevant to SBA.
subsections: section_title: SBA’s Mission and Loan Guarantee Program Structure Would Affect How SBA Uses the New Loan Monitoring Capability
paragraphs: Although SBA, similar to private lenders, must determine the level of risks  it will tolerate in the loans it guarantees, its mission obligations will drive  its credit risk management policies. For example, different loan products in  the 7(a) program have different levels of guarantees, and guarantees on 504  program loans have a different structure from 7(a) guarantees. These  differences influence the mix of loans in SBA’s portfolio and, consequently,  would impact how SBA manages its credit risk. Accordingly, SBA may  require policies and management reporting that are different from what  lenders require. For example, while lenders manage credit risk by  determining which loans to make and the mix of loans made, SBA, as a  federal agency and advocate for small business, may not be able to manage  its risk in the same ways. SBA’s exclusion of, or imposition of,  concentration limits on selected loan sectors based on risk limits could  conflict with congressional, public, or industry interpretations of its  mission obligations. Similarly, changing underwriting standards for certain  classes of loans could be difficult to implement because it would compel its  lending partners to change their underwriting criteria as needed due to  economic conditions. Additionally, SBA may permit its lenders to offer  greater forbearance (e.g., time to repay the loan) than private lenders  would in the absence of an SBA guarantee. Also, SBA could offer  assistance, such as counseling and technical help, to struggling borrowers  through its partnerships with private entities. These kinds of broad,  mission-related issues may influence the policies and business practices  governing SBA’s use of the Dun & Bradstreet loan monitoring service and  related tools.
The structures of SBA’s loan guarantee programs may also account for  some of the differences in risk management policies and practices between  SBA and major lenders. This lender-level emphasis contrasts with how  major private sector lenders manage credit risk, which is at the loan level.  Because SBA relies on private lenders to originate and service the majority  of the loans it guarantees, SBA is primarily managing the credit risk in its  portfolio at the lender level. As a result, much of the agency’s risk rating  processes and management reporting—while conceptually similar to the  processes associated with loan-level analysis—focuses on lenders, or a  lender’s portfolio of loans. Here, the Dun & Bradstreet loan monitoring  service supports lender oversight functions, such as SBLC examinations.  These lender oversight responsibilities, and the associated interest in  lender risk, contrast with how SBA, compared with private lenders, might  use its risk management tools.
section_title: Conclusions
paragraphs: In acquiring the loan monitoring service under contract with outside  experts, SBA has taken an important step that should help it meet the  needs it identified for monitoring its lending partners, and their portfolios,  and in managing the risk inherent in its $45 billion loan portfolio. The  service provided by Dun & Bradstreet reflects many best practices,  particularly those related to infrastructure and methodology, and can  facilitate a new level of sophistication in SBA’s oversight efforts. It will  afford SBA a means to obtain various measures of financial risk posed by  its lending partners and the opportunities to analyze loans and lending  patterns efficiently and effectively. These functions are important to  managing risk and to strengthening both SBA’s on-site reviews and off-site  monitoring of its lending partners—functions of the Office of Lender  Oversight (OLO). In addition, the Dun & Bradstreet service, its related  tools, and its potential for developing other tools could aid SBA offices  with other responsibilities. These include certifying preferred lenders,  identifying lenders against which enforcement actions might be taken,  ensuring that its lending programs are providing credit to special groups of  eligible small businesses (veterans, disadvantaged businesses, etc.), and  estimating the cost of its loan programs. However, the potential benefits of  the service, for OLO and other offices, cannot be realized without  comprehensive policies that reflect best practices appropriate to SBA’s  responsibilities to guide the use of the loan monitoring service. SBA’s time  line for developing such policies stretches into 2005, more than a year and a  half after the contractor delivered the capability to SBA. Moreover, SBA  officials have not yet begun to explore the potential uses of the service for  purposes other than lender oversight and portfolio monitoring, such as  creating budget projections for its loan programs. Notably, SBA’s continued  risk management capability is solely contingent on the continuation of the  Dun & Bradstreet contract. In the event that the Dun & Bradstreet contract  is discontinued, SBA would not have the capability on its own to duplicate  the loan monitoring service provided by Dun & Bradstreet.
section_title: Recommendations for Executive Action
paragraphs: We are making five recommendations to the SBA Administrator. First, we  recommend that in developing policies for the use of the Dun & Bradstreet  loan monitoring service, SBA consider the applicability of best practices,  including specific policy elements identified in this report. Practices that  should be considered include plans for continuous improvement in the  service and its tools, frequent and routine portfolio reviews, and active  involvement of senior SBA managers in reviewing the use of output.
Second, the Administrator should expedite the development of policies for  taking enforcement actions against all lending partners to address  noncompliance issues identified through the loan monitoring service and to  address safety and soundness issues among SBLCs and CDCs, for whom  SBA is the only regulator. We have made recommendations calling on SBA  to clarify its supervisory and enforcement powers since November 2000.  Although SBA has taken some incremental planning steps to address the  issue, its current time line estimates finalizing enforcement regulations in  April 2005.
Third, ensure that resources within SBA are devoted to developing policies  for the use of the loan monitoring service, so that the overall time line for  completion—April 2005—is met.
Fourth, establish an agencywide task force to explore the potential for  applying the capabilities of the Dun & Bradstreet service to SBA business  processes and responsibilities other than lender oversight, such as overall  portfolio risk management or budget projections. Programmatic offices  and the Office of the Chief Financial Officer should be included.
Fifth, develop contingency plans that would enable SBA's continued risk  management of the 7(a) and 504 portfolio overall, individual lenders, and  their portfolios in the event that the Dun & Bradstreet contract is  discontinued.
section_title: Agency Comments and Our Evaluation
paragraphs: We requested SBA’s comments on a draft of this report. The Associate  Deputy Administrator for Capital Access provided written comments that  are presented in appendix III. The Associate Deputy Administrator  generally agreed with the overall findings and recommendations, especially  the need to develop and fully implement policies for using the Dun &  Bradstreet service. However, the letter stated that SBA should receive more  credit for the progress it has made in developing these policies.
In contrast to SBA’s Associate Deputy Administrator, we think that we have  given SBA sufficient credit for its progress. In particular, we give credit for  obtaining the service, and we documented the significant progress made in  how the service provides SBA with best-practice infrastructure and  methodologies. However, SBA has not detailed how it has devoted  resources to the development of needed policies. In addition, based on our  analysis, it appears that SBA has not taken actions that are important to  successfully develop needed policies. The Associate Deputy Administrator  stated, “The development of policies is progressing logically following the  acquisition of the loan and lender monitoring services.”  In contrast, we  think that the development of policies for using such a service is an integral  part of strategic planning, including planning during the time period before  such a service is obtained. In our view, SBA could have developed more  specific policies for using the service before it was obtained. For example,  we have not seen evidence that SBA has developed policies addressing the  level of risk it will tolerate within the context of its mission and its  programs’ structures.
In response to our recommendation on considering the applicability of best  practices for risk management as it develops policies for using the Dun &  Bradstreet service, SBA’s Associate Deputy Administrator stated that it is  committed to fully implementing the service based on best practices  consistent with those that were identified in the report.
In comments regarding our recommendation to expedite the development  of policies, especially as they relate to enforcement, SBA’s Associate  Deputy Administrator stated that the agency has made progress in  developing its enforcement-related policies. SBA submitted legislative  proposals for specific enforcement authorities, but in the absence of  specific legislation, SBA intends to go forward with proposed enforcement  regulations under its general oversight authority. However, the final rule for  enforcement actions will not be completed until April 2005. We support  SBA’s intent to go forward with proposed enforcement regulations under  SBA’s general oversight authority, consistent with our earlier  recommendations.
Concerning our recommendation that SBA should ensure that resources  already within the agency are devoted to developing policies for the use of  the Dun & Bradstreet service, SBA’s Associate Deputy Administrator stated  that the agency is committed to fully implementing the service, including  the associated policies and procedures, and will make every effort to meet  the established time line of April 2005 for the policies’ completion.  However, the Associate Deputy Administrator did not specifically detail  what resources would be devoted to the development of the policies.
The Associate Deputy Administrator agreed with our recommendation that  SBA establish an agencywide task force to explore the potential for  applying capabilities of the Dun & Bradstreet service to various offices  within SBA and stated that the agency should leverage this resource to the  maximum extent possible. He acknowledged that while some information  provided by the Dun & Bradstreet service has far-ranging uses that could  benefit other program areas within SBA, the agency must recognize that  the service provides confidential business information. Therefore, uses of  the service by other offices remain unresolved.
In response to our recommendation that SBA develop contingency plans  that would enable SBA’s continued risk management of the 7(a) and 504  portfolio overall, individual lenders, and their portfolios in the event that  the Dun & Bradstreet contract is discontinued, SBA’s Associate Deputy  Administrator noted that the agency has begun to consider various options  to continue its approach to loan and lender monitoring, should the contract  be discontinued. SBA has identified several nationally recognized vendors  that offer possible replacement services, but the Associate Deputy  Administrator stated, and we agree, that it is impractical to run concurrent  contracts as a contingency plan. However, SBA does not have a formal  contingency plan in place.
The Associate Deputy Administrator stated in his comment letter that he  identified a number of inaccuracies in our draft report. However, these  were mostly technical corrections, which we incorporated, as appropriate,  in this report. SBA’s letter is reprinted in appendix III.
Unless you publicly announce its contents earlier, we plan no further  distribution until 30 days after the date of this report. At that time, we will  send copies of this report to the Ranking Minority Member of the Senate  Committee on Small Business and Entrepreneurship, the Chairman and  Ranking Minority Member of the House Committee on Small Business,  other appropriate congressional committees, and the Administrator of the  Small Business Administration. We also will make copies available to  others upon request. In addition, the report will be available at no charge  on the GAO Web site at http://www.gao.gov.
If you have any questions about this report, please contact me at (202) 512- 8678 or shearw@gao.gov; or Katie Harris, Assistant Director, at (202) 512- 8415 or harrism@gao.gov. Key contributors to this report are listed in  appendix IV.
section_title: Objectives, Scope, and Methodology
paragraphs: To evaluate the Small Business Administration’s (SBA) loan portfolio  monitoring needs, we first identified SBA’s loan portfolio monitoring  strategy and the intended purpose of the Dun & Bradstreet service. Then,  we identified best practices from federal guidance to banks and generally  accepted industry practices and explored how these practices might apply  to SBA. To identify SBA’s loan portfolio monitoring strategy, we analyzed  agency and contractor files. In addition, we interviewed SBA Office of  Lender Oversight (OLO) officials and Dun & Bradstreet contractors who  were providing the loan monitoring service during our review. We also  interviewed Farm Credit Administration (FCA) officials responsible for  conducting the Small Business Lending Corporation (SBLC) reviews during  the last few years and reviewed their summary report for fiscal year 2002.  To identify industry best practices for loan portfolio monitoring, we  analyzed guidance published by the Office of the Comptroller of the  Currency, the Basel Committee, the Federal Deposit Insurance  Corporation, and the Federal Reserve and consolidated all like practices.  We also consulted relevant literature related to financial markets and risk  management. Lastly, we interviewed officials at several large private banks  that make 7(a) and 504 loans as well as other loans to small businesses and  selected SBLCs.
To determine how well the new Dun & Bradstreet service and associated  tools meet SBA’s needs, we reviewed and analyzed agency and contractor  documents and conducted interviews. We analyzed the Dun & Bradstreet  contract files to identify the contract deliverables and the service’s  capabilities. We also verified the contractor’s implemented and planned  actions and interviewed relevant contractor staff. In addition, we obtained  and analyzed SBA planning documents, including its 2003-2008 Strategic  Plan, and its 2004 and 2005 Annual Performance Plans, and we interviewed  agency officials to determine SBA's use and planned use of the loan  monitoring service. Moreover, we compared SBA’s current and planned use  of the service to industry best practices we identified in analyzing SBA’s  loan portfolio monitoring needs.
To determine the major differences and similarities for the purposes of  credit risk management between SBA and private sector best practices, we  analyzed industry documents and interviewed risk management  professionals employed at several of SBA’s largest and most active small  business lending partners. We analyzed banking regulator publications  related to risk management, primarily credit risk, as well as position papers  from the Basel Committee, and considered various academic studies, and  selected books and papers recommended by the Global Association of Risk  Management Professionals. Furthermore, we interviewed bank examiners  and relevant employees of the Office of the Comptroller Currency and the  Federal Deposit Insurance Company.
To determine what steps SBA took to ensure the integrity of the data used  in the Dun & Bradstreet RAM (Risk Assessment Manager) data mart, we  analyzed agency and contractor documents and interviewed SBA and  contractor officials. To document SBA controls over its 7(a) program data,  we relied on the findings of our recent audit of SBA’s 7(a) program subsidy  model, in which we assessed the integrity of the data in SBA’s database. To  determine the data integrity processes for the 504 program, we analyzed  agency documents and 504 LAMP (the SBA-developed customized Access  database tool) data samples, and interviewed SBA officials. However, we  did not conduct independent tests of the 504 program data integrity  process. To determine the data integrity processes of the Dun & Bradstreet  and Fair Isaac data, we interviewed company officials. Although we did not  test the Dun & Bradstreet and Fair Isaac processes for ensuring data  quality, we reviewed their established procedures for quality and found  them generally reasonable. A summary of our related findings is contained  in appendix II.
We conducted our work in Washington, D.C., between August 2003 and May  2004 in accordance with generally accepted government auditing  standards.
section_title: SBA Data Integrity Processes for the Dun & Bradstreet RAM Data Mart
paragraphs: Controls to help ensure the integrity of the data entered in the Dun &  Bradstreet RAM data mart appear reasonable, as a whole, to ensure that  misstatements or inaccuracies are detected and corrected on a timely  basis, and the level of data errors in the system would not significantly  affect the loan monitoring service’s risk profiling capabilities. The RAM  database includes information related to SBA’s entire loan portfolio,  roughly 5,000-plus lenders and 230,000 outstanding loans, combining SBA  data with commercial data, consumer data, and credit scores to produce  risk metrics to facilitate lender oversight. The RAM receives data from four  different sources—SBA’s 7(a) and 504 databases, and Dun & Bradstreet and  Fair Isaac. We found that SBA’s controls over its 7(a) program data, which  represent approximately 70 percent of the data entered into the RAM, were  adequate to help ensure the quality of the underlying data. Our review of  504 program database data integrity procedures showed generally adequate  controls, as well. Although we did not test the Dun & Bradstreet and Fair  Isaac’s processes for data quality, we reviewed their established procedures  for data integrity and found them generally reasonable.
subsections: section_title: SBA Has Adequate Controls over 7(a) Program Data Integrity
paragraphs: In our report on SBA’s 7(a) program subsidy model, we found that SBA’s  monthly 7(a) reconciliation process, combined with lender incentives and  loan sales, helped ensure the quality of the underlying data. Although some  errors existed in SBA’s database at the time of the review, the nature and  magnitude of these errors were unlikely to significantly affect the  usefulness of the database. The 7(a) program data represent 70 percent of  the data entered into the RAM. Therefore, reasonableness of data integrity  over the 7(a) program data helps to provide assurance that the quality of  the data used is sufficiently reliable to monitor the performance of SBA’s  lenders and the risk exposure of SBA.
The primary method SBA used to identify and correct data errors in its 7(a)  program is its Form 1502 reconciliation process. Reconciliations are an  important internal control established to ensure that all data inputs are  received and are valid and that all outputs from a particular system are  correct. This process, in which an SBA contractor every month matches  borrower data submitted by 7(a) program lenders on SBA’s Form 1502 to  information in the agency’s portfolio management system, helps ensure the  completeness and accuracy of the agency’s data. SBA district office staff  work with lenders to correct errors identified by this match process. We  did not independently test the data match conducted by SBA’s contractors  or the field office staff. However, we reviewed summary reports of the  errors for each district office over a 4-month period during fiscal year 2003  and found that most of the errors reported were resolved during the month  the errors were identified.
In addition to the monthly loan data reconciliation process, lender  incentives also helped ensure the integrity of the underlying data. In  accordance with current SBA policy, the agency can reduce or completely  deny a lender’s claim for payment of the SBA guarantee if the defaulted  loan data are not correct. According to SBA officials, this policy gives the  7(a) program lenders an incentive to correct data errors because it helps  ensure they will be paid the full guarantee amount if the borrower  subsequently defaults on the loan. Further, an ancillary benefit of SBA’s  loan sales program was to help ensure data integrity. Prior to a sale, SBA  district office staff, as well as contractors, reviewed loan files as part of the  “due diligence” reviews to provide accurate information about the loans  available for sale, so that potential investors could make informed bids.  According to SBA officials, discrepancies between the lender’s data and  SBA data had to be resolved prior to selling a loan.
section_title: Processes for SBA 504 Data Integrity Appear Adequate
paragraphs: Unlike the 7(a) loan program, SBA does not currently have a formal  reconciliation process in place for 504 program data, but testing we  conducted found no major errors in the data. The informal process that  SBA uses to ensure the integrity of its 504 data is based on a series of  checks and balances, notably: (1) processing all payments through the  federal government’s automated clearinghouse (ACH); (2) electronically  uploading data; and (3) evaluating and certifying approved 504 lenders  based on accounting reports by a third party—Colson Services  Corporation, a unit of JP Morgan Chase. In addition, Certified Development  Companies (CDC) have an incentive to review the monthly reports and  notify SBA of any discrepancies.
The aggregated 504 data come from three sources, but only one source’s  data are inputted into the RAM database. The three sources for aggregated  data are current loan status and payment history, which is provided by  Colson—the same contractor that performs similar loan payment and  accounting for SBA’s 7(a) program; semiannual dividend disbursements to  investors, which is provided by the Bank of New York; and loan approval  and default loan information that resides in SBA’s mainframe. Colson and  the Bank of New York transmit data monthly to SBA. SBA developed a  customized Access database tool, referred to as the 504 LAMP, which  aggregates the data following a set of procedures. Dun & Bradstreet’s RAM  database will input only the Colson data for lender oversight purposes  since it is concerned only with the current loan data.
The processes used to collect and input the Colson data into the 504 LAMP  appear to minimize errors. Initially, Colson collects the majority of loan  payments electronically via ACH and credits the payments within one  business day of receipt. For payments not made, Colson is immediately  notified by ACH and contacts the CDCs to collect the payments. For those  late payments, checks or money orders are sent to Colson, and it enters the  payments into its database. Colson electronically sends the payment  information each month to SBA. Finally, SBA electronically inputs the  Colson data into the 504 LAMP database.
Another informal check on the integrity of the 504 LAMP data is the CDCs’  incentives to ensure that the current status of loans is accurate. CDCs’  continued participation in making 504 loans is contingent upon adequate  financial performance and accountability. Therefore, CDCs have strong  incentives to contact SBA to have any data errors corrected, or risk losing  further participation in the program. Selected CDC performance data are  uploaded monthly onto SBA’s password protected Web site. CDC directors  in the field can log in and receive a monthly report on their loan  performance. SBA officials stated that CDC staff are diligent about finding  errors and contacting SBA to remedy them.
section_title: Dun & Bradstreet and Fair Isaac Data Integrity Processes Appear Adequate
paragraphs: The quality control processes of Dun & Bradstreet and Fair Isaac appear to  be reasonable to help ensure the validity of the data used to produce risk  management information for SBA, based on our review of their  documentation and interviews with company officials. Due to the  proprietary nature of the processes, we were unable to independently test  the Dun & Bradstreet and Fair Isaac processes. However, Dun & Bradstreet  officials explained their proprietary quality control process, referred to as  DUNSRight, to validate the commercial data they provide to SBA.  Additionally, Fair Isaac officials discussed the sources of their consumer  data and how they ensure data quality.
The commercial and consumer data that Dun & Bradstreet staff input into  the RAM is used to analyze SBA loan data. More specifically, Dun &  Bradstreet staff use the data to create predictive models and decision tree  methodologies, and to group accounts with specific behaviors and risk  profiles. The predictive models include a suite of five different models  using Dun & Bradstreet and principal owner data, built using Fair Isaac  proven analytic methodologies. According to Dun & Bradstreet officials,  the models and decision trees are reviewed periodically to test and fine- tune strategies, based on changing market conditions. Dun & Bradstreet  officials also stated they have a continual improvement process whereby  the models used to analyze SBA loan and lender data are validated.
The commercial data that Dun & Bradstreet collects go through a five-step  quality assurance process. First, Dun & Bradstreet collects data from more  than 80 million businesses and continuously updates its databases more  than 1 million times daily based on real-time business transactions. Second,  it matches SBA records with its records and achieves at least 95 percent  match of the data on seven critical pieces of information used to identify  the borrower. Third, Dun & Bradstreet assigns a unique identifier to each  company. Fourth, Dun & Bradstreet identifies the corporate linkage of a  business’s branches/subsidiaries with their parent entity to help the SBA  understand their complete corporate exposure between borrowers and  their parent entities. Finally, Dun & Bradstreet generates predictive  indicators of a business’s potential inability to repay a loan. Dun &  Bradstreet officials refer to this process as the DUNSRight process.
Fair Isaac uses the commercial data from Dun & Bradstreet and consumer  data from a credit bureau to develop its credit scores. The consumer data  that Fair Isaac gathers from Trans Union Credit Bureau go through a less  detailed cleansing process, but the process still appears to be reasonable.  Initially, Fair Isaac provides the credit bureau with identifier information  (i.e., name and address) from SBA, so it can match the entity with its  associated credit report. Credit bureaus then send a report to Fair Isaac if  there is a match (or a “hit”). Fair Isaac officials told us that the match rate  is 95 percent. After Fair Isaac receives the credit reports, it electronically  files the multiple credit reports for each business and transforms them into  predictable variables. Finally, Fair Isaac creates predictive characteristics  from the blended Trans Union consumer and Dun & Bradstreet commercial  data, resulting in a Small Business Predictive Score (SBPS) intended to  predict the likelihood of severe loan delinquency. Fair Isaac sends the  SBPS score to Dun & Bradstreet, so it can load it into the RAM. Dun &  Bradstreet officials stated that controls are in place to verify that all data  merges in the RAM are successful.
According to Fair Isaac officials, its SBPS model will likely remain the  same because it is stable. The process Fair Isaac staff use to determine the  stability of its model starts with the development of a population stability  report. If the report states that the models are unstable, Fair Isaac then  creates a characteristics analysis report. This report determines if the  characteristics (or variables) have changed and by how much over time. In  addition, each year the models are revalidated. Third parties do not  routinely ensure the reliability or integrity of the models, but Fair Isaac’s  clients, such as SBA, inform the company if the models are not reasonably  predicting borrower behavior.
section_title: Comments from the Small Business Administration
section_title: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to the individuals above, Triana Bash, Dan Blair, Jamey Collins,  Jordan Corey, Dave Gill, Fred Jimenez, Mitch Rachlis, Carl Ramirez, and  Rhonda Rose made key contributions to this report.
section_title: GAO’s Mission
paragraphs: The General Accounting Office, the audit, evaluation and investigative arm of  Congress, exists to support Congress in meeting its constitutional responsibilities  and to help improve the performance and accountability of the federal government  for the American people. GAO examines the use of public funds; evaluates federal  programs and policies; and provides analyses, recommendations, and other  assistance to help Congress make informed oversight, policy, and funding  decisions. GAO’s commitment to good government is reflected in its core values of  accountability, integrity, and reliability.
section_title: Obtaining Copies of GAO Reports and Testimony
paragraphs: The fastest and easiest way to obtain copies of GAO documents at no cost is  through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older  products. The Web site features a search engine to help you locate documents  using key words and phrases. You can print these documents in their entirety,  including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and  correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site  daily. The list contains links to the full-text document files. To have GAO e-mail this  list to you every afternoon, go to www.gao.gov and select “Subscribe to   e-mail alerts” under the “Order GAO Products” heading.
subsections: section_title: Order by Mail or Phone
section_title: To Report Fraud, Waste, and Abuse in Federal Programs
section_title: Public Affairs