SBA Disaster Loan Program: Accounting Anomalies Resolved but Additional Steps Would Improve Long-Term Reliability of Cost Estimates
Published: 2005-04-14T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In response to a January 2003 GAO report that identified significant anomalies in the Small Business Administration's (SBA) disaster loan accounts and raised serious concerns about its ability to account for loan sales and estimate program costs, SBA conducted an extensive analysis to identify causes of the anomalies and implemented a number of corrective actions. In light of SBA's actions, GAO undertook a follow-up review to (1) describe the nature of the deficiencies SBA identified, (2) determine whether its corrective actions resolved the deficiencies, and (3) assess whether its procedures provide a reasonable basis for future credit estimates.
section_title: What GAO Found
paragraphs: SBA took prompt action with a comprehensive review of its financial records and systems to identify the deficiencies related to accounting for its disaster loans and loan sale program. SBA's review found (1) the cash flow model used to estimate the cost of the disaster loan program was unreliable and underestimated the cost, (2) the model used to determine whether sales were beneficial had errors and incorrectly indicated that loans were sold at gains, (3) incorrect loan values used to calculate the results of loan sales led to inaccurate reporting in SBA's financial statements, and (4) incomplete tools provided by OMB to calculate interest payments on borrowings from Treasury resulted in excess payments to Treasury and an insufficient balance in SBA's financing account and subsidy allowance. To resolve these deficiencies, SBA implemented a number of corrective actions during fiscal years 2003 and 2004. To address the first three, SBA developed a new cash flow model to estimate the costs and loan values for the disaster loan program. This improved the agency's ability to prepare more reliable cost estimates and determine the gain or loss on prior loan sales. To address the fourth deficiency, SBA analyzed its interest payments to Treasury and found that it had overpaid by about $134 million. SBA included this amount in its reestimates for the disaster loan program to correct prior interest payments and also implemented a different approach to update or "reestimate" its cost estimates, which will adjust its transactions with Treasury going forward. However, until OMB updates its tools for computing these interest payments, other credit agencies may also be over- or underpaying interest to Treasury. Further, SBA improved its policies and procedures to help ensure that future loan program cost estimates will be reasonable. For example, SBA implemented new standard operating procedures for calculating reestimates and prepared documentation to support the rationale and basis for key aspects of the cash flow model. However, because of the complexities associated with estimating loan program costs, additional actions by SBA would help improve the long-term reliability of cost estimates. These include (1) further documentation of the model and disaster data to readily provide for knowledge transfer between staff and contractors to help ensure proper maintenance, updating, and running of the model; (2) periodic assessments of the model's ability to predict loan performance; and (3) additional procedures to ensure the disaster data used in the model are tested to verify and document that they are reliable. In addition, there may be opportunities to improve the model with additional variables, such as financial strength of borrowers, as well as revisions to simplify the estimation process that warrant further consideration by SBA.

section_title: Letter
paragraphs: In 1999, the Small Business Administration (SBA) began a loan asset sales  program at the direction of the Office of Management and Budget (OMB) to  reduce the amount of loans the agency owned and serviced. A primary  objective of the sales was to maximize proceeds, with a goal for each sale  to be financially beneficial to the government. The loans eligible for sale  were disaster assistance and other direct loans and defaulted business loan  guarantees.  Between fiscal years 1999 and 2003, SBA conducted seven  sales, divesting itself of about 166,000 loans with an outstanding balance of  about $5.7 billion.  Approximately 86 percent of the amount sold was  disaster assistance loans.
Our January 2003 report on SBA’s first five loan sales identified significant  anomalies in SBA’s disaster loan accounts and raised serious concerns  about its ability to properly account for its loan sales and to estimate the  costs associated with its remaining disaster loan portfolio for budget and  accounting purposes. In response to our findings and recommendations,  SBA and its contractors (hereinafter referred to as SBA) conducted an  extensive analysis of its accounting and budgeting for its loan sales and  disaster loan program and implemented a number of corrective actions  during fiscal years 2003 and 2004, including the development of a new cash  flow model to estimate the cost of the disaster loan program for budget and  financial reporting purposes.
In light of these events, we were asked to conduct a follow-up review to  assess SBA’s corrective actions.  Specifically, our objectives were to (1)  describe the nature of the deficiencies SBA’s analysis identified that  contributed to the disaster loan accounting anomalies, (2) identify  corrective actions taken by SBA and assess whether these actions resolved  the identified deficiencies, and (3) determine whether SBA’s new cash flow  model and procedures for the disaster loan program provide a reasonable  basis for future credit subsidy estimates.
To address these objectives, we reviewed SBA’s analysis of its accounting  and budgeting for its disaster and loan sales programs and analyzed SBA’s  corrective actions, including the new cash flow model to estimate costs for  the disaster loan program. We also assessed the sufficiency of its policies  and procedures to estimate program costs and its corrective actions based  on applicable guidance.  We provided SBA a draft of this report and OMB a  draft of applicable sections of this report for review and comment. SBA and  OMB provided written comments, which are reprinted in appendix II and  III, respectively. We performed our work in accordance with generally  accepted government auditing standards in Washington, D.C. from April  2004 through March 2005. Our scope and methodology are discussed in  greater detail in appendix I.
subsections: section_title: Background
paragraphs: SBA provides small businesses with access to credit, primarily by  guaranteeing loans through its 7(a) and 504 programs. SBA also makes  loans directly to businesses and individuals trying to rebuild in the  aftermath of a disaster, and it primarily services these loans directly.   Substantially all of the disaster assistance loans have below-market interest  rates and repayment terms of up to 30 years.  Interest rates on disaster  loans vary, depending on the borrower’s ability to obtain credit in the  private sector.
The President’s fiscal year 1998 budget proposed that SBA begin selling  disaster and business loans that the agency was servicing and transition  from servicing loans directly to overseeing private-sector servicers. Before  its loan asset sales program began, SBA was servicing approximately  300,000 loans, with a principal balance of over $9 billion. About 286,000 of  these loans, with a principal balance of $7 billion, were disaster assistance  loans.
SBA, as well as other credit agencies, is required to account and budget for  its credit programs in accordance with the Federal Credit Reform Act of  1990 (FCRA).  FCRA was enacted to require agencies to more accurately  measure the government’s cost of federal credit programs and to permit  better cost comparisons, both among credit programs and between credit  and noncredit programs. The act gave OMB responsibility for coordinating  credit program estimates required by the act.  Authoritative guidance on  preparing cost estimates for the budget and conducting loan sales is  contained in OMB Circular A-11, Preparation, Submission, and Execution  of the Budget.  The Federal Accounting Standards Advisory Board  developed accounting standards for credit programs.  This guidance is  generally found in Statement of Federal Financial Accounting Standards  No. 2, Accounting for Direct Loans and Loan Guarantees, which became  effective in fiscal year 1994.  This standard, which generally mirrors FCRA  and budget guidance, established accounting guidance for estimating the  subsidy cost of loan programs, as well as recording loans and loan sales for  financial reporting purposes.
According to FCRA, the actual and expected costs of federal credit  programs should be recognized in budgetary reporting. The accounting  standard also requires these costs to be recognized for financial reporting.  To determine the expected cost of a credit program, agencies are required  to predict or estimate the future performance of the program on a cohort  basis. This cost, known as the subsidy cost, is the net present value of  disbursements by the government minus estimated payments to the  government over the life of the loan or loan guarantee, excluding  administrative costs. Figure 1 presents the cash flows included in the  subsidy cost calculation for direct and guaranteed loans.
FCRA established a special budgetary accounting system to record the  budget information necessary to implement credit reform.  Loans and loan  guarantees made on or after October 1, 1991—the effective date of credit  reform—use the (1) program and (2) financing accounts to handle credit  transactions. The program account is included in budget totals, receives  separate appropriations for the administrative and subsidy costs of a credit  program, and records the budget authority and outlays for these costs. The  program account is used to pay the associated subsidy cost to the financing  account when a direct or guaranteed loan is disbursed.  The financing  account, which is nonbudgetary, is used to collect the subsidy cost from  the program account, borrow from Treasury to provide financing for loan  disbursements, and record the cash flows associated with direct loans or  loan guarantees over their lives, including loan disbursements, default  payments to lenders, loan repayments, interest payments, recoveries on  defaulted loans, and fee collections.  Figure 2 shows the flow of program  and financing accounts transactions for a direct loan program.
FCRA requires that the rate of interest charged by Treasury on lending to  financing accounts be the same as the final discount rate used to calculate  the net present value of cash flows when estimating the subsidy cost of a  credit program. The final discount rate for a cohort of loans is determined  based on interest rates prevailing during the period that the loans are  disbursed.  Once the loans for a cohort are substantially disbursed (at least  90 percent), the final discount rate for that cohort is determined, and this  rate is to be used for financing account interest calculations. The same rate  is required to be used to calculate subsidy costs and interest on the  financing account, so that the financing account will break even over time  as it uses its collections to repay its Treasury borrowing. OMB provides  tools for agencies to use to calculate interest on the financing account.
To estimate the cost of credit programs, agencies first estimate the future  performance of direct and guaranteed loans, using cash flow models, when  preparing their annual budgets. The data used for these budgetary  estimates are generally updated or “reestimated” annually as of the end of  the fiscal year to reflect any changes in loan performance since the  estimates were prepared, as well as any expected changes in assumptions  related to future loan performance. Increases in subsidy costs that are  recognized through reestimates are funded through permanent indefinite  budget authority.
Before SBA could proceed with a loan sale, OMB had to approve it.  This  approval was based primarily on whether or not the sale was expected to  be financially beneficial to the government, meaning that the estimated  proceeds were expected to be greater than the estimated value of holding  the loans.  SBA estimated the current value to the government of holding  the loans, also known as the “hold value,” in accordance with OMB Circular  A-11.  The hold value is the expected net cash flows from the loans,  discounted at current Treasury rates. This differs from the net book value  recorded on SBA’s books, which is the expected net cash flows from the  loans discounted using Treasury rates in effect when the loans were  disbursed. Therefore, the hold value takes into account changes in interest  rates since the loans were disbursed, whereas the net book value does not.
Our January 2003 report on SBA’s first five loan sales highlighted  accounting anomalies related to its disaster loans and loan sales program.   Specifically, SBA incorrectly calculated the accounting losses on the loan  sales it disclosed in its financial statements and lacked reliable financial  data to determine the overall financial impact of the sales. Further, because  SBA did not analyze the effect of loan sales on its remaining portfolio, its  reestimates of loan program costs for the budget and financial statements  may have contained significant errors. In addition, SBA could not explain  significant declines in its subsidy allowance for disaster loans.
In response to our findings, SBA took immediate action to begin the  process of identifying the deficiencies that contributed to the disaster loan  accounting anomalies, including unexplained significant declines in its  subsidy allowance. A team of financial experts, including contractors and  staff from the Office of the Chief Financial Officer, was assembled to  conduct detailed reviews of financial records and systems related to the  disaster and loan sales programs. Several diagnostic-type analyses were  performed, including detailed reconciliations of the subsidy allowance and  testing of alternative versions of the cash flow model used to estimate the  cost of the disaster loan program. In January 2003, SBA hired IBM Business  Consulting Services (IBM) to help determine reasons for the abnormal  balance in the disaster loan program’s subsidy allowance and to identify  recommendations for correcting any deficiencies noted. IBM assisted SBA  in a detailed review of SBA’s accounting and budgeting for the disaster loan  program and its loan sale procedures. IBM summarized the results of this  review in a March 2003 report. According to SBA officials, the core  diagnosis of the problems was completed by April 2003 with the  submission and analysis of the report from IBM.
section_title: SBA Identified Major Deficiencies in Models and Methodologies Used to Account for the Disaster Loan Program
paragraphs: SBA and its contractors identified four key deficiencies related to the  disaster loan program accounting anomalies. These were (1) major flaws in  the cash flow model used to estimate the cost of the disaster loan program,  (2) errors and inconsistencies in the model used to determine whether  sales were beneficial, (3) incorrect loan values used to calculate the results  of loan sales, and (4) inconsistencies between the interest rates used to  estimate subsidy costs and the interest rates used to determine interest  payments to Treasury.
subsections: section_title: Cash Flow Model Used to Estimate Costs for the Disaster Loan Program Was Flawed
paragraphs: The methodology SBA’s cash flow model used to estimate costs for the  disaster loan program assumed that a single illustrative loan with  characteristics based on overall portfolio averages could serve as a proxy  for all loans and reasonably estimate cash flows for an entire cohort of  loans, which included sold and unsold loans. This methodology could have  produced reasonable cash flow estimates, even considering loan sales, if all  loans had similar characteristics.  However, this was not the case, and  flaws in this methodology became apparent once SBA began substantial  loan sales and the loans sold had different characteristics than the loans  not sold.  For example, the sold loans tended to have longer borrower  repayment periods, or loan terms, than the loans not sold. Therefore, the  sold loans would have had subsidized interest for a longer period of time  and would have cost more.  Because the single illustrative loan did not take  into consideration these differences when estimating cash flows, the model  had problems reestimating the cost of the program.
In addition to the basic flaw in the methodology, SBA incorrectly calculated  the single illustrative loan’s average loan-term assumptions used in the cash  flow model.  SBA estimated costs of the disaster loan program using  average loan-term assumptions of 16 years for business disaster loans and  17 years for home disaster loans. In our January 2003 report, we raised  concerns about the validity of these assumptions, as our review of disaster  loans sold indicated an average loan term of about 25 years.  SBA’s loan- term assumptions were based on the number of loans, rather than the  dollar value of loans disbursed, and therefore was a straight average rather  than a weighted average. However, SBA found that, for all disaster loans,  the average loan term, based on the dollar value of loans disbursed, was 23  years. Therefore, the model, which used loan terms of 16 and 17 years, did  not consider cash flows for the full term of the loans. Given that the  borrower interest rates on these loans were generally below market rates  and less than SBA’s cost of borrowing from Treasury to finance its loans,  understating the loan term also understated the program costs. According  to SBA, this caused the model to underestimate the cost of the disaster loan  program by 6 to 7 percent.
Further, during the reestimate process, SBA did not update the estimated  principal and interest with actual collection amounts, which resulted in  inaccurate data being used to calculate costs. These flaws related to the  methodology, loan-term assumptions, use of inaccurate data, and other  problems, resulted in unreliable subsidy cost estimates and reestimates for  the disaster loan program. Collectively, the problems with SBA’s cash flow  model resulted in significant underestimates of the cost of the program,  which was ultimately reflected in the negative balance in the disaster loan  subsidy allowance. The negative balance would occur for programs that  are expected to be profitable, which would not be expected for the disaster  loan program, or when the allowance is overspent, meaning that the  program cost far more than estimated.
section_title: The Model Used to Determine Whether Loan Sales Were Beneficial Was Also Flawed
paragraphs: When SBA sold loans, it used another model, called the hold model, to  estimate the value to the government of holding the loans scheduled for  sale until they were repaid, either at or before maturity.  The hold model  considered the same possible cash flows as the cash flow model used to  estimate the cost of the program, including principal and interest  collections, prepayments, delinquencies, defaults, and recoveries.   However, the hold model differed from the cash flow model because it was  constructed using a different methodology. The hold model measured loans  individually whereas the cash flow model, as previously discussed, used a  single-loan approach.  In addition, expected defaults were determined  differently, which caused the hold model to produce higher default rates  than the cash flow model.  Further, the hold model used economic  variables and performance indicators and the cash flow model did not.  As  a result of these differences, the two models produced different results.   While the hold model’s conceptual design was superior to the cash flow  model, it too contained serious flaws that produced misleading results.
For example, the hold model erroneously used assumptions in determining  the amount of recoveries expected on defaulted loans. The recovery  assumptions were taken from the cash flow model used to estimate the  cost of the program.  These assumptions were calculated based on actual  recoveries as a percentage of the value of loans disbursed and, therefore,  should have been applied based on the value of loans disbursed. The hold  model, however, erroneously applied this percentage to estimated  defaulted loan amounts, therefore calculating a far lower amount of  estimated recoveries than was appropriate, which made the value of  holding the loans seem much lower. Collectively, problems with SBA’s hold  model caused it to undervalue the loans it sold by about 30 percent,  according to disclosures in SBA’s fiscal year 2003 financial statements.  As a  result, at the time of the sales the hold model indicated that it was  financially beneficial to sell the loans when in fact it was not.
section_title: Incorrect Loan Values Were Used to Calculate the Results of Loan Sales Disclosed in SBA’s Financial Statements
paragraphs: While SBA’s hold model indicated at the time of the sales that SBA had  gains on loan sales, SBA concurrently disclosed losses on loan sales in its  financial statements based on yet another set of flawed calculations. As  reported by us in our January 2003 report and in the report issued by IBM  in March 2003, when SBA calculated the results of loan sales for purposes  of its financial statements, it incorrectly estimated the portion of the  subsidy allowance to allocate to each loan sold in order to calculate the  value on its books for the loans it had sold (net book value).  For example,  when calculating the net book value for the disaster loans that were sold,  SBA did not allocate a portion of the subsidy allowance for financing costs  associated with lending to borrowers at below-market interest rates. Given  that a large portion of the subsidy cost was related to providing below- market borrower interest rates, this omission resulted in a significant  overestimate of the net book value of the loans sold and, therefore, a  significant overestimate of the losses SBA disclosed in its financial  statements related to the sale of its disaster loans.  Even though SBA  calculated losses for the financial statements, it still operated on the  premise that loans were sold at gains when considering changes in interest  rates, which the hold model was purportedly designed to do.
section_title: Interest Rates Used to Calculate Interest Payments to Treasury Were not Consistent with Those Used to Estimate Subsidy Costs
paragraphs: The final deficiency that SBA identified in its disaster loan program  accounting related to inconsistencies in the interest rates it used to  estimate its subsidy costs versus those used to calculate its interest  payments to Treasury.  A direct loan program, including the disaster loan  program, funds its lending to borrowers with the subsidy cost it receives  through appropriations and from borrowing from Treasury. Because the  borrowing is expected to be repaid with collections from borrowers,  borrowing is not a budgeted cost to the program and is accounted for in the  program’s financing account. FCRA requires that the rate of interest  charged to the financing account on the agency’s borrowing be the same as  the interest rate used to discount cash flows (discount rate) when  estimating the subsidy cost for a program.  The equality of these rates is  fundamental to achieving the proper balance in the financing account. If  subsidy cost calculations are accurate and the proper interest rates used,  the financing account will break even over time as it uses collections from  borrowers to repay Treasury borrowings.
SBA, in coordination with OMB, found that the tools provided to agencies  to calculate interest for the financing account did not adjust the amount of  interest paid by the financing account while the loans were disbursing. The  discount rate used to estimate the subsidy cost is not final until the loans in  a cohort are substantially disbursed (at least 90 percent), which, for the  disaster program, generally may take at least 2 years.  When the loans are  substantially disbursed, and the final discount rate is fixed, the reestimate  process retroactively adjusts the subsidy costs to reflect the final discount  rate.
SBA and other agencies must make annual interest payments to Treasury  while the loans are disbursing, although the final interest rate has not yet  been determined.  Thus in the early years of a cohort, before the loans are  substantially disbursed, an interim interest rate is used to calculate interest  payments.  However, the tools provided by OMB to calculate interest  between the financing account and Treasury do not retroactively adjust  prior interest earnings or payments to reflect the final interest rate.  This  failure to adjust prior interest payments to reflect the final interest rate  resulted in excess payments to Treasury and an insufficient balance in  SBA’s financing account and subsidy allowance, since the interest  payments impact both.  This omission in the tools OMB provides to all  agencies that disburse or guarantee loans could result in a disconnect  between the amounts required to be earned from or paid to Treasury to  make the financing account whole, and the actual amounts earned or paid.  Consequently, agencies’ financing account balances and subsidy allowance  may be over- or understated.
section_title: SBA Has Taken Corrective Actions to Resolve Identified Deficiencies
paragraphs: Following its analysis and identification of deficiencies, SBA developed a  new cash flow model to estimate the cost of the disaster loan program. This  improved the reliability of the disaster program cost estimates and  corrected the abnormal balance in the subsidy allowance for the disaster  loan program. In addition, SBA analyzed its prior interest payments to  determine the effect of using inconsistent interest rates to calculate its  estimated subsidy costs and interest payments to Treasury, and  implemented a different approach to reestimate program costs. These  corrective actions helped SBA achieve an improved audit opinion on its  fiscal year 2004 and restated fiscal year 2003 financial statements.
subsections: section_title: New Cash Flow Model Improves SBA’s Ability to Reasonably Estimate Program Costs and Determine Loan Sale Results
paragraphs: In fiscal year 2003, SBA’s contractor developed a new cash flow model to  calculate subsidy cost estimates and reestimates for the disaster loan  program. In contrast to the prior model’s flawed single loan approach to  estimate the cash flows, the new model was designed to estimate cash  flows individually for each loan. This design facilitates calculating loan  values for loans sold to determine gains or losses on loan sales and, if SBA  schedules additional loan sales, could also be used to calculate loan values  for determining whether these sales would be financially beneficial to the  government, thus negating the need for a separate hold model. Application  of the model enabled SBA to retroactively determine the results of its prior  loan sales and to correct the abnormal balance in its subsidy allowance.
During the development of the new cash flow model, SBA analyzed the  available disaster loan data, including loan performance information, loan  terms, disaster type and magnitude, and regional information, as well as  certain economic data, such as unemployment, gross domestic product,  and interest rates. Based on these analyses, the data that best predicted  default and prepayment behavior—two important cash flows for the  disaster loan program—were selected to use as variables in the model. The  model segments the loan portfolio into groups of loans based on the final  variables selected, which were (1) the age of the loan, (2) the type of  borrower (home or business), (3) the size of the loan, (4) the type of  disaster loan (economic injury or physical damage), and (5) the length of  the grace period. Based on these variables, there are a total of 162 groups of  loans used to segment the disaster loan portfolio.
On a loan-by-loan basis, the cash flow model estimates the expected  principal and interest payments based on loan contract terms. Then the  model estimates deviations from these expected payments for  delinquencies, charge-offs, and prepayments. These deviations are  calculated based on historical averages of loan performance for each group  of loans. Lastly, the model estimates recoveries on charged-off loans based  on historical averages. The model’s methodology is based on the  assumption that the behavior of loans in the future, taking several  important loan characteristics into account, will be similar to loans in the  past.  However, as discussed later, if future loans are made to substantially  different types of borrowers, such as those with better or worse financial  strength, or have substantially different loan terms, changes to the model  would be required to correctly consider these new characteristics in the  cash flow estimates.
Throughout the development of the model, SBA documented several  analyses of the performance and characteristics of its disaster loans and  the model’s ability to predict loan performance. In addition to its own  analyses, SBA contracted with Ernst & Young LLP (E&Y) to conduct an  independent review of the model.  E&Y reviewed the model documentation  and computer code, and reviewed SBA’s testing and validation analyses.  E&Y summarized its observations and findings in two reports issued in  November and December 2003. E&Y noted that the model can be  expected to perform reasonably well for reestimates of existing loans and  to produce stable estimates over time given the model’s emphasis on long- term averages. In addition, E&Y stated that given the limitation of what can  be known about future loans, the model takes a reasonable approach to  estimating costs of future loans for budget purposes. E&Y also noted that  the model achieves SBA’s objective to consistently value individual loans  for reestimates and loan sales.
Based on our review of the model, its documentation, and the reports  issued by E&Y, we concluded that the new model provides a sound basis to  estimate costs and improved SBA’s ability to prepare more reliable and  reasonable cost estimates for the disaster loan program. When SBA used  this new model for the first time to reestimate the cost of the disaster loan  program, it resulted in a reestimate indicating increased costs of over  $1 billion as of the end of fiscal year 2003. As shown in figure 3, the  adjustment to increase these costs on SBA’s books helped bring the disaster  loan program’s subsidy allowance to a positive balance and more in line  with expectations for this type of subsidized program. In SBA’s fiscal year  2004 financial statements, the subsidy allowance was reported to be about  $613 million, or about 20 percent, of the $3 billion outstanding balance of  the disaster loan program.  Given that the estimated cost of the program  generally ranges from $16 to $36 for every $100 that SBA lends, this balance  is within the expected range.  SBA also used the new model to recalculate  the results of its prior disaster loan sales and determined that the sales  resulted in losses, or increased budgetary costs, of over $900 million.
section_title: Analysis of Interest Payments and Change in Approach to Reestimate Costs Addressed Interest Rate Inconsistency
paragraphs: To resolve the inconsistency in the interest rates used to calculate interest  on its financing account borrowing from Treasury and the interest rates  used to discount cash flows when estimating subsidy costs, SBA completed  a detailed analysis of its interest transactions with Treasury. SBA  recalculated what its interest payments would have been based on the final  (rather than the interim) interest rates and determined that it overpaid  Treasury by about $128.6 million and $5.6 million as of the end of fiscal  years 2003 and 2004, respectively. These amounts were included in SBA’s  reestimates for the disaster loan program and corrected SBA’s interest  transactions for the fiscal years 1992 through 2003 cohorts.
Also in fiscal year 2004, SBA implemented a new approach to reestimate  costs, called the balances approach. Because the balances approach  determines the amount of the reestimate based on a comparison of  resources in the financing account and expected future cash flows, the  approach automatically adjusts the financing account and subsidy  allowance for any inconsistency in interest rates going forward.
Other agencies that disburse or guarantee loans would also be affected by  this interest rate inconsistency, which could result in misstatements in their  accounts. However, the significance of this issue cannot be determined  without extensive analyses similar to SBA’s analysis because a number of  factors influence how the inconsistency would impact other agencies’  accounts, including balances in financing accounts and the length of time a  program takes to substantially disburse its loans.  OMB has notified  agencies of the flaw in the tools and outlined plans to issue a  comprehensive revised reestimate tool that resolves this problem. Until  updated tools are provided, agencies will continue to make incorrect  interest payments that could result in financing accounts having excess or  insufficient funds and misstatements in financial statement reporting  accounts for credit programs.
section_title: SBA’s Corrective Actions Helped It Achieve an Improved Audit Opinion
paragraphs: SBA’s corrective actions helped it achieve an improved audit opinion for its  fiscal year 2004 financial statements. Earlier, SBA’s auditor withdrew its  unqualified opinions on SBA’s fiscal years 2000 and 2001 financial  statements and issued a disclaimer of opinion for fiscal year 2002, in part,  because of issues identified in our January 2003 report. While progress was  made in addressing these issues during fiscal year 2003, the auditor also  issued a disclaimer of opinion on SBA’s fiscal year 2003 financial  statements. The auditor reported that because SBA was late in completing  reestimates and preparing its financial statements, among other things, the  auditor did not have adequate time to resolve reservations related to SBA’s  disaster loan program, including abnormal balances in the subsidy  allowance and a difference in interest rates SBA used to estimate subsidy  costs and calculate interest payments to Treasury.
Subsequently, SBA continued to implement its corrective actions, which  the auditor assessed as part of the fiscal year 2004 financial statement  audit. SBA received a mixed opinion—a combination of unqualified and  qualified opinions—on its fiscal year 2004 financial statements, which  represented an improvement over the disclaimer it received for fiscal year  2003. In addition, SBA received an unqualified opinion on its restated fiscal  year 2003 balance sheet.  SBA’s auditor did not cite any issues related to  previously identified problems with the disaster program or the new  disaster cash flow model in these audit opinions.
section_title: New Policies and Procedures Will Help Ensure Future Estimates Are Reasonable, but Additional Procedures and Documentation Are Needed
paragraphs: In addition to implementing the corrective actions to resolve the  accounting anomalies, SBA also implemented new policies and procedures  to help ensure that future loan program cost estimates will be reasonable,  including (1) the development and implementation of new standard  operating procedures for calculating reestimates; (2) the preparation of  documentation to support the rationale and basis for key aspects of the  cash flow model; (3) a process to coordinate the preparation of cost  estimates between budget, accounting, and program staff; and (4) a revised  reestimate approach. However, additional documentation of the new cash  flow model would help ensure proper operation and maintenance of the  model.  Further, over time it will be important for SBA to continue to assess  the model’s ability to predict loan performance.  In addition, there may be  opportunities to improve the model, as well as simplify the estimation  process, that warrant further consideration by SBA. Lastly, additional  procedures to test the disaster data used in the model will help ensure their  reliability.
subsections: section_title: New Policies and Procedures Strengthened Internal Controls
paragraphs: During fiscal years 2003 and 2004, SBA enhanced its policies and  procedures by implementing several of the internal control practices  identified in federal accounting guidance that will help it ensure that future  cost estimates are reliable and reasonable. SBA developed and  implemented standard operating procedures for calculating its reestimates  based on federal accounting guidance.  These procedures established an  internal review process and standardized steps that must be performed and  documented as part of the reestimate process. Steps included ensuring that  the correct cash flow model files are used, verifying that the model  appropriately reflects the program’s structure, documenting any technical  changes to the model, updating actual data and estimated cash flows in the  model, and reviewing the reasonableness of the estimated cash flows. The  procedures also call for the cash flow model to be reviewed by an outside  party. The supporting documentation for the reestimates was provided to  SBA’s financial statement auditor during the fiscal year 2004 audit.  The  auditor noted in its report on internal controls that the adherence to a set of  standard operating procedures for calculating reestimates, along with  improved documentation and an effective internal review process, were  critical to SBA’s success in meeting key milestone dates and completing the  audit process within accelerated financial reporting deadlines.
SBA also established other important internal control practices identified  in the guidance.  During the development of the new cash flow model, SBA  documented key analyses and decisions regarding the model’s  methodology. For example, SBA compared the characteristics of the loans  sold to the loans kept and developed an approach within the new model to  take those differences into account when estimating loan performance.  It  also documented the basis for selecting the model’s methodology and  variables, the assumptions and calculations in the model, and results of  testing the model’s ability to predict cash flow estimates. This  documentation helps support the rationale and basis for key aspects of the  model that provide important cost information for budgets and financial  statements.
SBA has also established procedures to coordinate the preparation of cost  estimates among budget, accounting, and program offices. This will help  ensure that the estimates are reviewed and prepared with the proper  information. Further, as previously discussed, SBA implemented the  balances approach for reestimates. This approach will help ensure that  SBA’s account balances are in line with expected future cash flows.  These  practices and the other practices discussed above will help ensure that  anomalies such as those we identified during our last review do not go  undetected or uncorrected.
section_title: Additional Documentation, Analysis, and Testing Would Improve Long-Term Reliability of Cost Estimates
paragraphs: While SBA resolved its accounting anomalies related to the disaster loan  program and made important improvements to its policies and procedures,  we found that additional enhancements to internal controls would help  ensure the long-term reliability of future cost estimates. Further,  strengthening internal controls will help SBA identify potential problems in  the future and sustain the progress it has already made.
Even though SBA completed substantial documentation for the new cash  flow model, we found that this documentation was not sufficient to readily  provide for knowledge transfer between staff and contractors to help  ensure proper maintenance and updating of the model. For example, the  documentation does not specify what is done to prepare the data for use in  the model and does not always clearly indicate the data sources. In  addition, SBA’s documentation to explain the files used to run the model  and update the data used in the model was not complete.  For example, in  SBA’s documentation of the files and steps used to run the cash flow model,  out of a total of 19 steps, 6 steps had no explanations of the process and  another 2 were not complete and indicated that someone who was no  longer employed at SBA was to provide the information.
Improved documentation is particularly important because SBA relied on a  contractor to help develop the new cash flow model for the disaster loan  program and has recently experienced significant turnover in staff  responsible for preparing cost estimates. Without complete and detailed  documentation on how to maintain the model, update it with additional  data, and run it, it will be more difficult for current SBA staff to fully  understand the model, which could result in future errors in the cash flow  estimates.
Thorough documentation of the model is even more important given the  complexity associated with its calculation process. E&Y noted in its review,  and we agree, that the model’s complexity creates an ongoing challenge  related to transparency and maintenance.  Because complexity increases  the risk of errors occurring, SBA could benefit from continuing to evaluate  whether there are opportunities to simplify the estimation process with  model revisions or alternative estimation methodologies.
There may also be opportunities to improve the model with additional  variables. When estimating loan performance, the new model does not use  data related to the financial strength of borrowers.  Because this kind of  information has been shown to be useful in predicting loan performance,  such as defaults and prepayments, incorporating this type of information  could improve the model’s estimates. Further, additional detailed data on  borrower financial strength and loan collateral, among other things, may  improve the model’s effectiveness for supporting any future loan sales.  According to SBA officials, beginning in fiscal year 2003 SBA began  collecting credit scores for disaster loan borrowers. Once these newer  loans have sufficient historical data, SBA will be able to evaluate the  usefulness of these data as a potential variable to predict loan  performance.
In addition to opportunities to improve the model, SBA could also enhance  its procedures to ensure that the model’s estimates reasonably predict  future loan performance. While SBA has completed testing of the model’s  ability to predict loan performance, it is important that SBA establish a  process to help ensure this testing continues routinely and that causes of  any significant variances are identified and addressed.  For example, as  stated earlier, if future loans are made to substantially different types of  borrowers or have substantially different loan terms, changes to the model  would be required to correctly consider these new characteristics in the  cash flow estimates. Routine testing would help identify this type of  change.
While the new cash flow model provides SBA with a sound approach to  estimate costs for the disaster loan program, additional verification  procedures would provide better assurance that data used by the model are  reliable.  Federal accounting guidance requires agencies to accumulate  sufficient, relevant, and reliable supporting data that provide a reliable  basis for estimates of future loan performance. Because SBA’s old cash  flow model used data from SBA’s Main On-Line System for Tracking  Evaluation and Response (MONSTER) database which contains summary  information, most of its detailed data reliability assessments and  reconciliation practices revolved around MONSTER. For example, SBA  maintains a reconciliation that tracks loans in MONSTER with its general  ledger at a cohort level. However, the new model uses data from MONSTER  and loan-level data from SBA’s Electronic Loan Information Processing  System (ELIPS) database.  SBA officials indicated that plans are to  continue to move away from using MONSTER. While SBA routinely  reconciles its ELIPS database at a high level, as it moves toward using  ELIPS data for estimating its disaster program costs, it is important that  SBA reconcile and test the data at the level used in the model.
section_title: Conclusions
paragraphs: SBA took prompt action to identify the deficiencies related to its disaster  and loan sale programs with a comprehensive review of its financial  records. The corrective actions it then took established a basis for reliable  and reasonable cost estimates. At the same time, the complexities  associated with estimating costs for these programs will require continued  attention.  Without enhancements to the model’s documentation, additional  procedures to test data reliability, and continued testing and analysis of the  model, SBA may find it difficult to fully sustain the progress it has made.  Further, improved tools from OMB would help SBA and other agencies  ensure proper calculation of interest costs related to their credit programs.
section_title: Recommendations for Executive Action
paragraphs: We are making five recommendations to SBA and two to OMB.  To help  ensure that future subsidy cost estimates are reliable, we recommend that  the SBA Administrator take the following five actions.
Develop additional documentation of the new disaster cash flow model  to help facilitate proper operation, maintenance, and updating of the  model.
Study the value of incorporating additional variables in the new disaster  cash flow model, such as detailed information on the financial strength  of borrowers.
Establish policies and procedures to routinely test the new disaster cash  flow model’s ability to predict loan performance by comparing the  model’s predictions to actual loan performance and to identify and  address the causes of any significant variances.
Consider possible revisions to the model and/or alternative  methodologies that would simplify the estimation process.
Establish additional procedures to test and document the reliability of  the data used in the new cash flow model for the disaster loan program.
To help ensure that agencies make correct interest calculations for  financing accounts, we recommend that the OMB Director take the  following two actions.
Update the tools provided to agencies for adjusting financing account  interest transactions once a final interest rate is determined for a cohort.
Provide instructions to agencies on making retroactive corrections to  financing account interest transactions based on final interest rates for a  cohort.
section_title: Agency Comments
paragraphs: In written comments reprinted in appendix II, SBA stated that these were  appropriate recommendations and that it already has work underway to  address several of them. In written comments reprinted in appendix III,  OMB agreed with our recommendations and stated that it would work with  agencies to correct interest transactions with Treasury. SBA and OMB also  provided technical comments, which we have incorporated as appropriate.
As agreed with your offices, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days from its  date.  At that time, we will send copies of this report to the Ranking  Minority Member of the Senate Committee on Small Business and  Entrepreneurship, other appropriate congressional committees, the  Administrator of the Small Business Administration, and the Director of the  Office of Management and Budget. Copies will also be made available to  others upon request. In addition, the report will be available at no charge  on the GAO Web site at http://www.gao.gov.
If you or your staffs have any questions about this report, please contact  me at (202) 512-9508 or calboml@gao.gov. Major contributors to this report  are acknowledged in appendix IV.
section_title: Objectives, Scope, and Methodology
paragraphs: To describe the nature of the deficiencies SBA identified that contributed  to the disaster loan program accounting anomalies, we reviewed the report  prepared by IBM that summarized the detailed review of SBA’s accounting  and budgeting for its disaster loan program and its loan sale procedures  performed by SBA and IBM, and a variety of documents prepared by SBA  that summarized the issues found. We also reviewed OMB Circular A-11  and FCRA to determine the criteria for calculating interest payments to  Treasury on borrowing. We also interviewed SBA and OMB officials, SBA’s  financial statement auditor, and a contractor with SBA.
To identify the corrective actions taken and assess whether these actions  resolved the identified deficiencies, we:   obtained and assessed the new cash flow model used to estimate the  cost of the disaster loan program and its various supporting  documentation;  obtained an understanding of how the model works by reviewing SBA’s  summary of the behavior equations, the production system  documentation, and assumptions made in the model;   analyzed the model’s methodology, including choice of statistical  technique and variables included in the analysis, and determined that  they were appropriate and reasonably related to the prediction of cash  flows of disaster loans;   replicated certain components of the model, such as the process used to  segment the portfolio into groups of loans and predict future loan  performance;   assessed SBA’s logistic regression used to determine and support the  variables used in the model to verify that the variables selected were  statistically significant;   reviewed (1) SBA’s testing of the cash flow model for biases; (2) SBA’s  comparison of the loans sold and the loans kept; (3) E&Y’s reports  summarizing its independent reviews of the model, including  procedures performed, findings, and observations; and (4) SBA’s  financial statement auditor’s assessment of SBA’s model, its reestimates,  and its revised loan sale loss calculation;   obtained SBA’s analysis of its interest payments to Treasury and verified   reviewed SBA’s fiscal year 2004 financial statements summarizing the  implementation of the balances approach to reestimate costs;   interviewed SBA officials, an SBA contractor, and SBA’s financial   interviewed OMB officials to obtain an understanding of their efforts to  update the tools agencies use to calculate interest on financing account  balances.
To determine whether SBA’s new cash flow model and procedures for the  disaster loan program provide a reasonable basis for future subsidy cost  estimates, we interviewed SBA officials and a contractor to obtain an  understanding of SBA policies and procedures for estimating subsidy costs.   We reviewed supporting documentation related to its procedures, including  the standard documentation template used to support its reestimates for its  fiscal year 2004 financial statements, the various documentation prepared  to support the model, and the reliability of data from SBA’s computer  systems. Based on SBA’s procedures and documentation, we assessed the  sufficiency of SBA’s estimation process based on federal accounting  guidance that identifies internal control practices that help ensure that  future cost estimates are reliable and reasonable. We also reviewed SBA’s  financial statement auditor’s reports on internal controls for fiscal years  2003 and 2004.  The checking of key components, along with our review of  SBA’s documentation and E&Y’s evaluation of the model, provided a  sufficient level of understanding to conclude on its approach and ability to  produce more reliable and reasonable cost estimates for the disaster loan  program. To identify any additional steps SBA could take to improve the  long-term reliability of its model, we considered additional types of  variables that might enhance SBA’s approach. As part of this analysis, we  reviewed academic literature on default modeling and discussed  alternative variables and modeling techniques with the contractor SBA  used to develop the model. Based on these assessments, our assessment of  the model, and E&Y’s findings and observations on the cash flow model, we  identified opportunities SBA could explore to enhance its procedures to  improve the long-term reliability of its cost estimates.
We provided SBA a draft of this report and OMB a draft of applicable  sections of this report for review and comment. SBA and OMB provided  written comments, which are reprinted in appendix II and III, respectively.  They also provided technical comments, which we have incorporated as  appropriate. We performed our work in accordance with generally  accepted government auditing standards in Washington, D.C. from April  2004 through March 2005.
section_title: Comments from the Small Business Administration
section_title: Comments from the Office of Management and Budget
section_title: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the above, Marcia Carlsen, Lisa Crye, Austin Kelly, Beverly  Ross, Kara Scott, and Brooke Whittaker made key contributions to this  report.