SBA Disaster Loan Program: Accounting Anomalies Resolved but Additional Steps Would Improve Long-Term Reliability of Cost Estimates
Released: 2005-05-16T00:00:00

Letter

Background

SBA provides small businesses with access to credit, primarily by guaranteeing loans through its 7(a) and 504 programs. SBA also makes loans directly to businesses and individuals trying to rebuild in the aftermath of a disaster, and it primarily services these loans directly. Substantially all of the disaster assistance loans have below-market interest rates and repayment terms of up to 30 years. Interest rates on disaster loans vary, depending on the borrower’s ability to obtain credit in the private sector.

The President’s fiscal year 1998 budget proposed that SBA begin selling disaster and business loans that the agency was servicing and transition from servicing loans directly to overseeing private-sector servicers. Before its loan asset sales program began, SBA was servicing approximately 300,000 loans, with a principal balance of over $9 billion. About 286,000 of these loans, with a principal balance of $7 billion, were disaster assistance loans.

SBA, as well as other credit agencies, is required to account and budget for its credit programs in accordance with the Federal Credit Reform Act of 1990 (FCRA). FCRA was enacted to require agencies to more accurately measure the government’s cost of federal credit programs and to permit better cost comparisons, both among credit programs and between credit and noncredit programs. The act gave OMB responsibility for coordinating credit program estimates required by the act. Authoritative guidance on preparing cost estimates for the budget and conducting loan sales is contained in OMB Circular A-11, Preparation, Submission, and Execution of the Budget. The Federal Accounting Standards Advisory Board developed accounting standards for credit programs. This guidance is generally found in Statement of Federal Financial Accounting Standards No. 2, Accounting for Direct Loans and Loan Guarantees, which became effective in fiscal year 1994. This standard, which generally mirrors FCRA and budget guidance, established accounting guidance for estimating the subsidy cost of loan programs, as well as recording loans and loan sales for financial reporting purposes.

According to FCRA, the actual and expected costs of federal credit programs should be recognized in budgetary reporting. The accounting standard also requires these costs to be recognized for financial reporting. To determine the expected cost of a credit program, agencies are required to predict or estimate the future performance of the program on a cohort basis. This cost, known as the subsidy cost, is the net present value of disbursements by the government minus estimated payments to the government over the life of the loan or loan guarantee, excluding administrative costs. Figure 1 presents the cash flows included in the subsidy cost calculation for direct and guaranteed loans.

FCRA established a special budgetary accounting system to record the budget information necessary to implement credit reform. Loans and loan guarantees made on or after October 1, 1991—the effective date of credit reform—use the (1) program and (2) financing accounts to handle credit transactions. The program account is included in budget totals, receives separate appropriations for the administrative and subsidy costs of a credit program, and records the budget authority and outlays for these costs. The program account is used to pay the associated subsidy cost to the financing account when a direct or guaranteed loan is disbursed. The financing account, which is nonbudgetary, is used to collect the subsidy cost from the program account, borrow from Treasury to provide financing for loan disbursements, and record the cash flows associated with direct loans or loan guarantees over their lives, including loan disbursements, default payments to lenders, loan repayments, interest payments, recoveries on defaulted loans, and fee collections. Figure 2 shows the flow of program and financing accounts transactions for a direct loan program.

FCRA requires that the rate of interest charged by Treasury on lending to financing accounts be the same as the final discount rate used to calculate the net present value of cash flows when estimating the subsidy cost of a credit program. The final discount rate for a cohort of loans is determined based on interest rates prevailing during the period that the loans are disbursed. Once the loans for a cohort are substantially disbursed (at least 90 percent), the final discount rate for that cohort is determined, and this rate is to be used for financing account interest calculations. The same rate is required to be used to calculate subsidy costs and interest on the financing account, so that the financing account will break even over time as it uses its collections to repay its Treasury borrowing. OMB provides tools for agencies to use to calculate interest on the financing account.

To estimate the cost of credit programs, agencies first estimate the future performance of direct and guaranteed loans, using cash flow models, when preparing their annual budgets. The data used for these budgetary estimates are generally updated or “reestimated” annually as of the end of the fiscal year to reflect any changes in loan performance since the estimates were prepared, as well as any expected changes in assumptions related to future loan performance. Increases in subsidy costs that are recognized through reestimates are funded through permanent indefinite budget authority.

Before SBA could proceed with a loan sale, OMB had to approve it. This approval was based primarily on whether or not the sale was expected to be financially beneficial to the government, meaning that the estimated proceeds were expected to be greater than the estimated value of holding the loans. SBA estimated the current value to the government of holding the loans, also known as the “hold value,” in accordance with OMB Circular A-11. The hold value is the expected net cash flows from the loans, discounted at current Treasury rates. This differs from the net book value recorded on SBA’s books, which is the expected net cash flows from the loans discounted using Treasury rates in effect when the loans were disbursed. Therefore, the hold value takes into account changes in interest rates since the loans were disbursed, whereas the net book value does not.

Our January 2003 report on SBA’s first five loan sales highlighted accounting anomalies related to its disaster loans and loan sales program. Specifically, SBA incorrectly calculated the accounting losses on the loan sales it disclosed in its financial statements and lacked reliable financial data to determine the overall financial impact of the sales. Further, because SBA did not analyze the effect of loan sales on its remaining portfolio, its reestimates of loan program costs for the budget and financial statements may have contained significant errors. In addition, SBA could not explain significant declines in its subsidy allowance for disaster loans.

In response to our findings, SBA took immediate action to begin the process of identifying the deficiencies that contributed to the disaster loan accounting anomalies, including unexplained significant declines in its subsidy allowance. A team of financial experts, including contractors and staff from the Office of the Chief Financial Officer, was assembled to conduct detailed reviews of financial records and systems related to the disaster and loan sales programs. Several diagnostic-type analyses were performed, including detailed reconciliations of the subsidy allowance and testing of alternative versions of the cash flow model used to estimate the cost of the disaster loan program. In January 2003, SBA hired IBM Business Consulting Services (IBM) to help determine reasons for the abnormal balance in the disaster loan program’s subsidy allowance and to identify recommendations for correcting any deficiencies noted. IBM assisted SBA in a detailed review of SBA’s accounting and budgeting for the disaster loan program and its loan sale procedures. IBM summarized the results of this review in a March 2003 report. According to SBA officials, the core diagnosis of the problems was completed by April 2003 with the submission and analysis of the report from IBM.

SBA Identified Major Deficiencies in Models and Methodologies Used to Account for the Disaster Loan Program

SBA and its contractors identified four key deficiencies related to the disaster loan program accounting anomalies. These were (1) major flaws in the cash flow model used to estimate the cost of the disaster loan program, (2) errors and inconsistencies in the model used to determine whether sales were beneficial, (3) incorrect loan values used to calculate the results of loan sales, and (4) inconsistencies between the interest rates used to estimate subsidy costs and the interest rates used to determine interest payments to Treasury.

Cash Flow Model Used to Estimate Costs for the Disaster Loan Program Was Flawed

The methodology SBA’s cash flow model used to estimate costs for the disaster loan program assumed that a single illustrative loan with characteristics based on overall portfolio averages could serve as a proxy for all loans and reasonably estimate cash flows for an entire cohort of loans, which included sold and unsold loans. This methodology could have produced reasonable cash flow estimates, even considering loan sales, if all loans had similar characteristics. However, this was not the case, and flaws in this methodology became apparent once SBA began substantial loan sales and the loans sold had different characteristics than the loans not sold. For example, the sold loans tended to have longer borrower repayment periods, or loan terms, than the loans not sold. Therefore, the sold loans would have had subsidized interest for a longer period of time and would have cost more. Because the single illustrative loan did not take into consideration these differences when estimating cash flows, the model had problems reestimating the cost of the program.

In addition to the basic flaw in the methodology, SBA incorrectly calculated the single illustrative loan’s average loan-term assumptions used in the cash flow model. SBA estimated costs of the disaster loan program using average loan-term assumptions of 16 years for business disaster loans and 17 years for home disaster loans. In our January 2003 report, we raised concerns about the validity of these assumptions, as our review of disaster loans sold indicated an average loan term of about 25 years. SBA’s loan- term assumptions were based on the number of loans, rather than the dollar value of loans disbursed, and therefore was a straight average rather than a weighted average. However, SBA found that, for all disaster loans, the average loan term, based on the dollar value of loans disbursed, was 23 years. Therefore, the model, which used loan terms of 16 and 17 years, did not consider cash flows for the full term of the loans. Given that the borrower interest rates on these loans were generally below market rates and less than SBA’s cost of borrowing from Treasury to finance its loans, understating the loan term also understated the program costs. According to SBA, this caused the model to underestimate the cost of the disaster loan program by 6 to 7 percent.

Further, during the reestimate process, SBA did not update the estimated principal and interest with actual collection amounts, which resulted in inaccurate data being used to calculate costs. These flaws related to the methodology, loan-term assumptions, use of inaccurate data, and other problems, resulted in unreliable subsidy cost estimates and reestimates for the disaster loan program. Collectively, the problems with SBA’s cash flow model resulted in significant underestimates of the cost of the program, which was ultimately reflected in the negative balance in the disaster loan subsidy allowance. The negative balance would occur for programs that are expected to be profitable, which would not be expected for the disaster loan program, or when the allowance is overspent, meaning that the program cost far more than estimated.

The Model Used to Determine Whether Loan Sales Were Beneficial Was Also Flawed

When SBA sold loans, it used another model, called the hold model, to estimate the value to the government of holding the loans scheduled for sale until they were repaid, either at or before maturity. The hold model considered the same possible cash flows as the cash flow model used to estimate the cost of the program, including principal and interest collections, prepayments, delinquencies, defaults, and recoveries. However, the hold model differed from the cash flow model because it was constructed using a different methodology. The hold model measured loans individually whereas the cash flow model, as previously discussed, used a single-loan approach. In addition, expected defaults were determined differently, which caused the hold model to produce higher default rates than the cash flow model. Further, the hold model used economic variables and performance indicators and the cash flow model did not. As a result of these differences, the two models produced different results. While the hold model’s conceptual design was superior to the cash flow model, it too contained serious flaws that produced misleading results.

For example, the hold model erroneously used assumptions in determining the amount of recoveries expected on defaulted loans. The recovery assumptions were taken from the cash flow model used to estimate the cost of the program. These assumptions were calculated based on actual recoveries as a percentage of the value of loans disbursed and, therefore, should have been applied based on the value of loans disbursed. The hold model, however, erroneously applied this percentage to estimated defaulted loan amounts, therefore calculating a far lower amount of estimated recoveries than was appropriate, which made the value of holding the loans seem much lower. Collectively, problems with SBA’s hold model caused it to undervalue the loans it sold by about 30 percent, according to disclosures in SBA’s fiscal year 2003 financial statements. As a result, at the time of the sales the hold model indicated that it was financially beneficial to sell the loans when in fact it was not.

Incorrect Loan Values Were Used to Calculate the Results of Loan Sales Disclosed in SBA’s Financial Statements

While SBA’s hold model indicated at the time of the sales that SBA had gains on loan sales, SBA concurrently disclosed losses on loan sales in its financial statements based on yet another set of flawed calculations. As reported by us in our January 2003 report and in the report issued by IBM in March 2003, when SBA calculated the results of loan sales for purposes of its financial statements, it incorrectly estimated the portion of the subsidy allowance to allocate to each loan sold in order to calculate the value on its books for the loans it had sold (net book value). For example, when calculating the net book value for the disaster loans that were sold, SBA did not allocate a portion of the subsidy allowance for financing costs associated with lending to borrowers at below-market interest rates. Given that a large portion of the subsidy cost was related to providing below- market borrower interest rates, this omission resulted in a significant overestimate of the net book value of the loans sold and, therefore, a significant overestimate of the losses SBA disclosed in its financial statements related to the sale of its disaster loans. Even though SBA calculated losses for the financial statements, it still operated on the premise that loans were sold at gains when considering changes in interest rates, which the hold model was purportedly designed to do.

Interest Rates Used to Calculate Interest Payments to Treasury Were not Consistent with Those Used to Estimate Subsidy Costs

The final deficiency that SBA identified in its disaster loan program accounting related to inconsistencies in the interest rates it used to estimate its subsidy costs versus those used to calculate its interest payments to Treasury. A direct loan program, including the disaster loan program, funds its lending to borrowers with the subsidy cost it receives through appropriations and from borrowing from Treasury. Because the borrowing is expected to be repaid with collections from borrowers, borrowing is not a budgeted cost to the program and is accounted for in the program’s financing account. FCRA requires that the rate of interest charged to the financing account on the agency’s borrowing be the same as the interest rate used to discount cash flows (discount rate) when estimating the subsidy cost for a program. The equality of these rates is fundamental to achieving the proper balance in the financing account. If subsidy cost calculations are accurate and the proper interest rates used, the financing account will break even over time as it uses collections from borrowers to repay Treasury borrowings.

SBA, in coordination with OMB, found that the tools provided to agencies to calculate interest for the financing account did not adjust the amount of interest paid by the financing account while the loans were disbursing. The discount rate used to estimate the subsidy cost is not final until the loans in a cohort are substantially disbursed (at least 90 percent), which, for the disaster program, generally may take at least 2 years. When the loans are substantially disbursed, and the final discount rate is fixed, the reestimate process retroactively adjusts the subsidy costs to reflect the final discount rate.

SBA and other agencies must make annual interest payments to Treasury while the loans are disbursing, although the final interest rate has not yet been determined. Thus in the early years of a cohort, before the loans are substantially disbursed, an interim interest rate is used to calculate interest payments. However, the tools provided by OMB to calculate interest between the financing account and Treasury do not retroactively adjust prior interest earnings or payments to reflect the final interest rate. This failure to adjust prior interest payments to reflect the final interest rate resulted in excess payments to Treasury and an insufficient balance in SBA’s financing account and subsidy allowance, since the interest payments impact both. This omission in the tools OMB provides to all agencies that disburse or guarantee loans could result in a disconnect between the amounts required to be earned from or paid to Treasury to make the financing account whole, and the actual amounts earned or paid. Consequently, agencies’ financing account balances and subsidy allowance may be over- or understated.

SBA Has Taken Corrective Actions to Resolve Identified Deficiencies

Following its analysis and identification of deficiencies, SBA developed a new cash flow model to estimate the cost of the disaster loan program. This improved the reliability of the disaster program cost estimates and corrected the abnormal balance in the subsidy allowance for the disaster loan program. In addition, SBA analyzed its prior interest payments to determine the effect of using inconsistent interest rates to calculate its estimated subsidy costs and interest payments to Treasury, and implemented a different approach to reestimate program costs. These corrective actions helped SBA achieve an improved audit opinion on its fiscal year 2004 and restated fiscal year 2003 financial statements.

New Cash Flow Model Improves SBA’s Ability to Reasonably Estimate Program Costs and Determine Loan Sale Results

In fiscal year 2003, SBA’s contractor developed a new cash flow model to calculate subsidy cost estimates and reestimates for the disaster loan program. In contrast to the prior model’s flawed single loan approach to estimate the cash flows, the new model was designed to estimate cash flows individually for each loan. This design facilitates calculating loan values for loans sold to determine gains or losses on loan sales and, if SBA schedules additional loan sales, could also be used to calculate loan values for determining whether these sales would be financially beneficial to the government, thus negating the need for a separate hold model. Application of the model enabled SBA to retroactively determine the results of its prior loan sales and to correct the abnormal balance in its subsidy allowance.

During the development of the new cash flow model, SBA analyzed the available disaster loan data, including loan performance information, loan terms, disaster type and magnitude, and regional information, as well as certain economic data, such as unemployment, gross domestic product, and interest rates. Based on these analyses, the data that best predicted default and prepayment behavior—two important cash flows for the disaster loan program—were selected to use as variables in the model. The model segments the loan portfolio into groups of loans based on the final variables selected, which were (1) the age of the loan, (2) the type of borrower (home or business), (3) the size of the loan, (4) the type of disaster loan (economic injury or physical damage), and (5) the length of the grace period. Based on these variables, there are a total of 162 groups of loans used to segment the disaster loan portfolio.

On a loan-by-loan basis, the cash flow model estimates the expected principal and interest payments based on loan contract terms. Then the model estimates deviations from these expected payments for delinquencies, charge-offs, and prepayments. These deviations are calculated based on historical averages of loan performance for each group of loans. Lastly, the model estimates recoveries on charged-off loans based on historical averages. The model’s methodology is based on the assumption that the behavior of loans in the future, taking several important loan characteristics into account, will be similar to loans in the past. However, as discussed later, if future loans are made to substantially different types of borrowers, such as those with better or worse financial strength, or have substantially different loan terms, changes to the model would be required to correctly consider these new characteristics in the cash flow estimates.

Throughout the development of the model, SBA documented several analyses of the performance and characteristics of its disaster loans and the model’s ability to predict loan performance. In addition to its own analyses, SBA contracted with Ernst & Young LLP (E&Y) to conduct an independent review of the model. E&Y reviewed the model documentation and computer code, and reviewed SBA’s testing and validation analyses. E&Y summarized its observations and findings in two reports issued in November and December 2003. E&Y noted that the model can be expected to perform reasonably well for reestimates of existing loans and to produce stable estimates over time given the model’s emphasis on long- term averages. In addition, E&Y stated that given the limitation of what can be known about future loans, the model takes a reasonable approach to estimating costs of future loans for budget purposes. E&Y also noted that the model achieves SBA’s objective to consistently value individual loans for reestimates and loan sales.

Based on our review of the model, its documentation, and the reports issued by E&Y, we concluded that the new model provides a sound basis to estimate costs and improved SBA’s ability to prepare more reliable and reasonable cost estimates for the disaster loan program. When SBA used this new model for the first time to reestimate the cost of the disaster loan program, it resulted in a reestimate indicating increased costs of over $1 billion as of the end of fiscal year 2003. As shown in figure 3, the adjustment to increase these costs on SBA’s books helped bring the disaster loan program’s subsidy allowance to a positive balance and more in line with expectations for this type of subsidized program. In SBA’s fiscal year 2004 financial statements, the subsidy allowance was reported to be about $613 million, or about 20 percent, of the $3 billion outstanding balance of the disaster loan program. Given that the estimated cost of the program generally ranges from $16 to $36 for every $100 that SBA lends, this balance is within the expected range. SBA also used the new model to recalculate the results of its prior disaster loan sales and determined that the sales resulted in losses, or increased budgetary costs, of over $900 million.

Analysis of Interest Payments and Change in Approach to Reestimate Costs Addressed Interest Rate Inconsistency

To resolve the inconsistency in the interest rates used to calculate interest on its financing account borrowing from Treasury and the interest rates used to discount cash flows when estimating subsidy costs, SBA completed a detailed analysis of its interest transactions with Treasury. SBA recalculated what its interest payments would have been based on the final (rather than the interim) interest rates and determined that it overpaid Treasury by about $128.6 million and $5.6 million as of the end of fiscal years 2003 and 2004, respectively. These amounts were included in SBA’s reestimates for the disaster loan program and corrected SBA’s interest transactions for the fiscal years 1992 through 2003 cohorts.

Also in fiscal year 2004, SBA implemented a new approach to reestimate costs, called the balances approach. Because the balances approach determines the amount of the reestimate based on a comparison of resources in the financing account and expected future cash flows, the approach automatically adjusts the financing account and subsidy allowance for any inconsistency in interest rates going forward.

Other agencies that disburse or guarantee loans would also be affected by this interest rate inconsistency, which could result in misstatements in their accounts. However, the significance of this issue cannot be determined without extensive analyses similar to SBA’s analysis because a number of factors influence how the inconsistency would impact other agencies’ accounts, including balances in financing accounts and the length of time a program takes to substantially disburse its loans. OMB has notified agencies of the flaw in the tools and outlined plans to issue a comprehensive revised reestimate tool that resolves this problem. Until updated tools are provided, agencies will continue to make incorrect interest payments that could result in financing accounts having excess or insufficient funds and misstatements in financial statement reporting accounts for credit programs.

SBA’s Corrective Actions Helped It Achieve an Improved Audit Opinion

SBA’s corrective actions helped it achieve an improved audit opinion for its fiscal year 2004 financial statements. Earlier, SBA’s auditor withdrew its unqualified opinions on SBA’s fiscal years 2000 and 2001 financial statements and issued a disclaimer of opinion for fiscal year 2002, in part, because of issues identified in our January 2003 report. While progress was made in addressing these issues during fiscal year 2003, the auditor also issued a disclaimer of opinion on SBA’s fiscal year 2003 financial statements. The auditor reported that because SBA was late in completing reestimates and preparing its financial statements, among other things, the auditor did not have adequate time to resolve reservations related to SBA’s disaster loan program, including abnormal balances in the subsidy allowance and a difference in interest rates SBA used to estimate subsidy costs and calculate interest payments to Treasury.

Subsequently, SBA continued to implement its corrective actions, which the auditor assessed as part of the fiscal year 2004 financial statement audit. SBA received a mixed opinion—a combination of unqualified and qualified opinions—on its fiscal year 2004 financial statements, which represented an improvement over the disclaimer it received for fiscal year 2003. In addition, SBA received an unqualified opinion on its restated fiscal year 2003 balance sheet. SBA’s auditor did not cite any issues related to previously identified problems with the disaster program or the new disaster cash flow model in these audit opinions.

New Policies and Procedures Will Help Ensure Future Estimates Are Reasonable, but Additional Procedures and Documentation Are Needed

In addition to implementing the corrective actions to resolve the accounting anomalies, SBA also implemented new policies and procedures to help ensure that future loan program cost estimates will be reasonable, including (1) the development and implementation of new standard operating procedures for calculating reestimates; (2) the preparation of documentation to support the rationale and basis for key aspects of the cash flow model; (3) a process to coordinate the preparation of cost estimates between budget, accounting, and program staff; and (4) a revised reestimate approach. However, additional documentation of the new cash flow model would help ensure proper operation and maintenance of the model. Further, over time it will be important for SBA to continue to assess the model’s ability to predict loan performance. In addition, there may be opportunities to improve the model, as well as simplify the estimation process, that warrant further consideration by SBA. Lastly, additional procedures to test the disaster data used in the model will help ensure their reliability.

New Policies and Procedures Strengthened Internal Controls

During fiscal years 2003 and 2004, SBA enhanced its policies and procedures by implementing several of the internal control practices identified in federal accounting guidance that will help it ensure that future cost estimates are reliable and reasonable. SBA developed and implemented standard operating procedures for calculating its reestimates based on federal accounting guidance. These procedures established an internal review process and standardized steps that must be performed and documented as part of the reestimate process. Steps included ensuring that the correct cash flow model files are used, verifying that the model appropriately reflects the program’s structure, documenting any technical changes to the model, updating actual data and estimated cash flows in the model, and reviewing the reasonableness of the estimated cash flows. The procedures also call for the cash flow model to be reviewed by an outside party. The supporting documentation for the reestimates was provided to SBA’s financial statement auditor during the fiscal year 2004 audit. The auditor noted in its report on internal controls that the adherence to a set of standard operating procedures for calculating reestimates, along with improved documentation and an effective internal review process, were critical to SBA’s success in meeting key milestone dates and completing the audit process within accelerated financial reporting deadlines.

SBA also established other important internal control practices identified in the guidance. During the development of the new cash flow model, SBA documented key analyses and decisions regarding the model’s methodology. For example, SBA compared the characteristics of the loans sold to the loans kept and developed an approach within the new model to take those differences into account when estimating loan performance. It also documented the basis for selecting the model’s methodology and variables, the assumptions and calculations in the model, and results of testing the model’s ability to predict cash flow estimates. This documentation helps support the rationale and basis for key aspects of the model that provide important cost information for budgets and financial statements.

SBA has also established procedures to coordinate the preparation of cost estimates among budget, accounting, and program offices. This will help ensure that the estimates are reviewed and prepared with the proper information. Further, as previously discussed, SBA implemented the balances approach for reestimates. This approach will help ensure that SBA’s account balances are in line with expected future cash flows. These practices and the other practices discussed above will help ensure that anomalies such as those we identified during our last review do not go undetected or uncorrected.

Additional Documentation, Analysis, and Testing Would Improve Long-Term Reliability of Cost Estimates

While SBA resolved its accounting anomalies related to the disaster loan program and made important improvements to its policies and procedures, we found that additional enhancements to internal controls would help ensure the long-term reliability of future cost estimates. Further, strengthening internal controls will help SBA identify potential problems in the future and sustain the progress it has already made.

Even though SBA completed substantial documentation for the new cash flow model, we found that this documentation was not sufficient to readily provide for knowledge transfer between staff and contractors to help ensure proper maintenance and updating of the model. For example, the documentation does not specify what is done to prepare the data for use in the model and does not always clearly indicate the data sources. In addition, SBA’s documentation to explain the files used to run the model and update the data used in the model was not complete. For example, in SBA’s documentation of the files and steps used to run the cash flow model, out of a total of 19 steps, 6 steps had no explanations of the process and another 2 were not complete and indicated that someone who was no longer employed at SBA was to provide the information.

Improved documentation is particularly important because SBA relied on a contractor to help develop the new cash flow model for the disaster loan program and has recently experienced significant turnover in staff responsible for preparing cost estimates. Without complete and detailed documentation on how to maintain the model, update it with additional data, and run it, it will be more difficult for current SBA staff to fully understand the model, which could result in future errors in the cash flow estimates.

Thorough documentation of the model is even more important given the complexity associated with its calculation process. E&Y noted in its review, and we agree, that the model’s complexity creates an ongoing challenge related to transparency and maintenance. Because complexity increases the risk of errors occurring, SBA could benefit from continuing to evaluate whether there are opportunities to simplify the estimation process with model revisions or alternative estimation methodologies.

There may also be opportunities to improve the model with additional variables. When estimating loan performance, the new model does not use data related to the financial strength of borrowers. Because this kind of information has been shown to be useful in predicting loan performance, such as defaults and prepayments, incorporating this type of information could improve the model’s estimates. Further, additional detailed data on borrower financial strength and loan collateral, among other things, may improve the model’s effectiveness for supporting any future loan sales. According to SBA officials, beginning in fiscal year 2003 SBA began collecting credit scores for disaster loan borrowers. Once these newer loans have sufficient historical data, SBA will be able to evaluate the usefulness of these data as a potential variable to predict loan performance.

In addition to opportunities to improve the model, SBA could also enhance its procedures to ensure that the model’s estimates reasonably predict future loan performance. While SBA has completed testing of the model’s ability to predict loan performance, it is important that SBA establish a process to help ensure this testing continues routinely and that causes of any significant variances are identified and addressed. For example, as stated earlier, if future loans are made to substantially different types of borrowers or have substantially different loan terms, changes to the model would be required to correctly consider these new characteristics in the cash flow estimates. Routine testing would help identify this type of change.

While the new cash flow model provides SBA with a sound approach to estimate costs for the disaster loan program, additional verification procedures would provide better assurance that data used by the model are reliable. Federal accounting guidance requires agencies to accumulate sufficient, relevant, and reliable supporting data that provide a reliable basis for estimates of future loan performance. Because SBA’s old cash flow model used data from SBA’s Main On-Line System for Tracking Evaluation and Response (MONSTER) database which contains summary information, most of its detailed data reliability assessments and reconciliation practices revolved around MONSTER. For example, SBA maintains a reconciliation that tracks loans in MONSTER with its general ledger at a cohort level. However, the new model uses data from MONSTER and loan-level data from SBA’s Electronic Loan Information Processing System (ELIPS) database. SBA officials indicated that plans are to continue to move away from using MONSTER. While SBA routinely reconciles its ELIPS database at a high level, as it moves toward using ELIPS data for estimating its disaster program costs, it is important that SBA reconcile and test the data at the level used in the model.

Conclusions

SBA took prompt action to identify the deficiencies related to its disaster and loan sale programs with a comprehensive review of its financial records. The corrective actions it then took established a basis for reliable and reasonable cost estimates. At the same time, the complexities associated with estimating costs for these programs will require continued attention. Without enhancements to the model’s documentation, additional procedures to test data reliability, and continued testing and analysis of the model, SBA may find it difficult to fully sustain the progress it has made. Further, improved tools from OMB would help SBA and other agencies ensure proper calculation of interest costs related to their credit programs.

Recommendations for Executive Action

We are making five recommendations to SBA and two to OMB. To help ensure that future subsidy cost estimates are reliable, we recommend that the SBA Administrator take the following five actions.

Develop additional documentation of the new disaster cash flow model to help facilitate proper operation, maintenance, and updating of the model.

Study the value of incorporating additional variables in the new disaster cash flow model, such as detailed information on the financial strength of borrowers.

Establish policies and procedures to routinely test the new disaster cash flow model’s ability to predict loan performance by comparing the model’s predictions to actual loan performance and to identify and address the causes of any significant variances.

Consider possible revisions to the model and/or alternative methodologies that would simplify the estimation process.

Establish additional procedures to test and document the reliability of the data used in the new cash flow model for the disaster loan program.

To help ensure that agencies make correct interest calculations for financing accounts, we recommend that the OMB Director take the following two actions.

Update the tools provided to agencies for adjusting financing account interest transactions once a final interest rate is determined for a cohort.

Provide instructions to agencies on making retroactive corrections to financing account interest transactions based on final interest rates for a cohort.

Agency Comments

In written comments reprinted in appendix II, SBA stated that these were appropriate recommendations and that it already has work underway to address several of them. In written comments reprinted in appendix III, OMB agreed with our recommendations and stated that it would work with agencies to correct interest transactions with Treasury. SBA and OMB also provided technical comments, which we have incorporated as appropriate.

As agreed with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its date. At that time, we will send copies of this report to the Ranking Minority Member of the Senate Committee on Small Business and Entrepreneurship, other appropriate congressional committees, the Administrator of the Small Business Administration, and the Director of the Office of Management and Budget. Copies will also be made available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-9508 or calboml@gao.gov. Major contributors to this report are acknowledged in appendix IV.

Objectives, Scope, and Methodology

To describe the nature of the deficiencies SBA identified that contributed to the disaster loan program accounting anomalies, we reviewed the report prepared by IBM that summarized the detailed review of SBA’s accounting and budgeting for its disaster loan program and its loan sale procedures performed by SBA and IBM, and a variety of documents prepared by SBA that summarized the issues found. We also reviewed OMB Circular A-11 and FCRA to determine the criteria for calculating interest payments to Treasury on borrowing. We also interviewed SBA and OMB officials, SBA’s financial statement auditor, and a contractor with SBA.

To identify the corrective actions taken and assess whether these actions resolved the identified deficiencies, we: obtained and assessed the new cash flow model used to estimate the cost of the disaster loan program and its various supporting documentation; obtained an understanding of how the model works by reviewing SBA’s summary of the behavior equations, the production system documentation, and assumptions made in the model; analyzed the model’s methodology, including choice of statistical technique and variables included in the analysis, and determined that they were appropriate and reasonably related to the prediction of cash flows of disaster loans; replicated certain components of the model, such as the process used to segment the portfolio into groups of loans and predict future loan performance; assessed SBA’s logistic regression used to determine and support the variables used in the model to verify that the variables selected were statistically significant; reviewed (1) SBA’s testing of the cash flow model for biases; (2) SBA’s comparison of the loans sold and the loans kept; (3) E&Y’s reports summarizing its independent reviews of the model, including procedures performed, findings, and observations; and (4) SBA’s financial statement auditor’s assessment of SBA’s model, its reestimates, and its revised loan sale loss calculation; obtained SBA’s analysis of its interest payments to Treasury and verified reviewed SBA’s fiscal year 2004 financial statements summarizing the implementation of the balances approach to reestimate costs; interviewed SBA officials, an SBA contractor, and SBA’s financial interviewed OMB officials to obtain an understanding of their efforts to update the tools agencies use to calculate interest on financing account balances.

To determine whether SBA’s new cash flow model and procedures for the disaster loan program provide a reasonable basis for future subsidy cost estimates, we interviewed SBA officials and a contractor to obtain an understanding of SBA policies and procedures for estimating subsidy costs. We reviewed supporting documentation related to its procedures, including the standard documentation template used to support its reestimates for its fiscal year 2004 financial statements, the various documentation prepared to support the model, and the reliability of data from SBA’s computer systems. Based on SBA’s procedures and documentation, we assessed the sufficiency of SBA’s estimation process based on federal accounting guidance that identifies internal control practices that help ensure that future cost estimates are reliable and reasonable. We also reviewed SBA’s financial statement auditor’s reports on internal controls for fiscal years 2003 and 2004. The checking of key components, along with our review of SBA’s documentation and E&Y’s evaluation of the model, provided a sufficient level of understanding to conclude on its approach and ability to produce more reliable and reasonable cost estimates for the disaster loan program. To identify any additional steps SBA could take to improve the long-term reliability of its model, we considered additional types of variables that might enhance SBA’s approach. As part of this analysis, we reviewed academic literature on default modeling and discussed alternative variables and modeling techniques with the contractor SBA used to develop the model. Based on these assessments, our assessment of the model, and E&Y’s findings and observations on the cash flow model, we identified opportunities SBA could explore to enhance its procedures to improve the long-term reliability of its cost estimates.

We provided SBA a draft of this report and OMB a draft of applicable sections of this report for review and comment. SBA and OMB provided written comments, which are reprinted in appendix II and III, respectively. They also provided technical comments, which we have incorporated as appropriate. We performed our work in accordance with generally accepted government auditing standards in Washington, D.C. from April 2004 through March 2005.

Comments from the Small Business Administration

Comments from the Office of Management and Budget

GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the above, Marcia Carlsen, Lisa Crye, Austin Kelly, Beverly Ross, Kara Scott, and Brooke Whittaker made key contributions to this report.