When disasters such as floods, tornadoes, or earthquakes strike, federal, state, and local government agencies coordinate to provide assistance to disaster victims. SBA, through its Disaster Loan Program, is part of this concerted effort. In the event of a disaster, SBA, the Federal Emergency Management Agency (FEMA), and other government agencies join together to conduct a preliminary damage assessment to estimate the physical damage of the disaster on the affected region. Among other criteria, if there is extensive physical damage, the governor of the affected state can request that the U.S. President declare that a major disaster or emergency situation exists, in which case federal assistance is made available to disaster victims, and FEMA takes the lead in coordinating response and recovery efforts. The presidential disaster declaration specifies the area that is eligible for federal assistance, referred to as the “immediate” disaster area in this report. In addition, SBA provides certain loans to disaster victims in the counties adjacent to the immediate area; we refer to these counties as the “contiguous” disaster area. In the immediate area of the disaster, homeowners, renters, nonprofit organizations, and nonfarm businesses of all sizes are eligible to apply for SBA loans for the repair and replacement of uninsured physically damaged property. In both the immediate and contiguous areas of the disaster, small businesses with no credit available elsewhere are eligible to apply for loans to cover economic losses. Once a declaration has been made, officials from one of SBA’s four Disaster Area Offices—located in California, Georgia, New York, and Texas—arrive at the disaster site to begin making preparations to serve disaster victims. According to SBA’s procedures, disaster loan officials secure office space— sometimes in FEMA-operated Disaster Recovery Centers for presidential declarations—and begin meeting with victims to explain the disaster loan process, issue loan applications, and, if requested, assist victims in completing applications. Appendix II summarizes the series of steps involved in accepting, reviewing, approving or declining, and disbursing disaster loans. SBA provides loans to households and businesses without credit available elsewhere at a maximum rate of 4 percent and up to a 30-year term. For households or businesses with credit available elsewhere, SBA provides loans at a maximum rate of 8 percent and, for businesses, up to a 3-year term. Business loans are available up to $1.5 million, loans for physical damage to homes are available up to $200,000, and loans for the repair or replacement of personal property are available up to $40,000. Like other federal programs, the performance of SBA’s Disaster Loan Program is reported in accordance with the Government Performance and Results Act (GPRA) of 1993. The purpose of GPRA is to shift the focus of federal management and decisionmaking from a preoccupation with the number of tasks completed or services provided to the real differences the tasks or services make to the nation or individual taxpayer. GPRA requires agencies to set multiyear strategic goals in their strategic plans and corresponding annual goals in their performance plans, measure performance toward the achievement of those goals, and report on their progress in their annual performance reports. The strategic plans, which cover a period of at least 5 years, are the starting point in setting annual goals for programs and in measuring progress toward achieving those goals. Final annual performance plans, first required for fiscal year 1999, are sent to the Congress soon after the transmittal of the President’s budget, and provide a direct linkage between an agency’s longer-term goals and mission and day-to-day activities. Related annual performance reports describe the degree to which performance goals were met. According to Office of Management and Budget (OMB) guidance, strategic goals, and performance goals in annual plans may be identical. According to GPRA, if a performance goal becomes impractical or infeasible to achieve, the agency is to explain in the performance reports why that is the case and what legislative, regulatory, or other actions are needed to accomplish the goal, or whether the goal ought to be modified or discontinued. Table 1 lists GPRA requirements for each of these documents. Both the strategic plan and the performance plan describe the relationship between a program’s goals, outputs, and outcomes. As noted previously, according to OMB guidance, outputs are the level of activity that can be produced or provided over a given period of time or by a specific date. Outcomes are the intended results, effects, or consequences that occur from carrying out program activities. In the case of the Disaster Loan Program, SBA has described the outputs as disaster loans to individuals and businesses, while program outcomes include restored housing and increased survival of businesses. OMB guidance allows agencies to divide outcomes into two categories: end and intermediate outcomes. End outcomes are the results of programs and activities compared to their intended purpose. Intermediate outcomes show progress toward achieving end outcomes. These outcomes are often required for programs when end outcomes are not immediately clear, easily delivered, or quickly achieved. OMB guidance indicates that performance plans should include measures of outcomes when the outcomes can be achieved during the fiscal year covered by the plan. Otherwise, the guidance recognizes that the performance plans will predominantly include measures of outputs rather than outcomes. In addition to OMB guidance, SBA program managers can obtain guidance in the preparation of performance goals and measures from GAO, and more recently, from an SBA primer. In the weeks and months following the terrorist attacks, SBA and the Congress faced the challenge of responding to the lingering effects of the attacks and subsequent federal actions on small businesses throughout the country. SBA responded first in Lower Manhattan, meeting with potential borrowers within 2 days of the attacks. Its response expanded as areas near the site of the attack on the Pentagon and more of the New York City area were designated disaster areas. Ultimately, SBA helped small businesses around the country with disaster lending. After small businesses raised concerns about the Disaster Loan Program’s ability to help businesses recover from the attacks, SBA and the Congress modified the program, raising loan limits and deferring interest payments, expanding eligibility for economic injury loans to small businesses around the country, modifying its size standards for small businesses, expediting its loan approval and disbursement processes, and providing translators for loan applicants. By the end of fiscal year 2002, SBA approved more than 9,700 loans for a total of $966 million to assist in the recovery efforts of September 11 victims nationwide. SBA’s response to the terrorist attacks began September 11, when SBA officials arrived in Lower Manhattan to begin coordinating the agency’s efforts. The President declared the attack on the World Trade Center a major disaster area on September 11. Unlike most of the disasters SBA had been involved in, the economic effects of the terrorist attacks were felt throughout the country. SBA’s initial disaster area in New York City and New Jersey eventually expanded to include additional counties in New York, New Jersey, Connecticut, Massachusetts, and Pennsylvania. On September 21, the President declared the Pentagon attack as a major disaster, establishing counties Maryland and Virginia and parts of the District of Columbia as disaster areas. As the United States began to deploy military personnel in response to the terrorist attacks, small businesses nationwide affected by the loss of employees called up as military reservists were eligible to apply for a disaster loan under the Military Reservist Economic Injury Disaster Loan (EIDL) program. As discussed later in this report, small businesses across the nation that were adversely affected by the lingering effects of the attacks and subsequent government action, such as airport closings and the precipitous drop in tourism, were also eligible to receive disaster loans under SBA’s Expanded EIDL program. In essence, the entire country was deemed a disaster area. As shown in figure 1, more than half the loans went to small businesses outside the area of the attack sites in New York City and at the Pentagon, with businesses in Florida and California receiving the second and third largest share of loans. In general, businesses beyond the immediate sites of the attacks received slightly more than those close by, in part because these businesses did not have the additional resources available to them that were available in New York City. As shown in figure 2, the loans were spread among industries, with no single type of business accounting for most of the funds. The manufacturing sector received the largest amount of funds. Other major industries receiving the most loan funds were professional, scientific, and technical services; transportation and warehousing; wholesale trade; and accommodation and food services. By the end of fiscal year 2002, SBA approved more than 9,700 home and business loans totaling $966 million for victims of the September 11 attacks. The agency expects to disburse $924 million—or 96 percent of the amount approved—due to loan increases, decreases, and cancellations. Individual loan disbursement amounts range from $300 to $1.5 million. Eleven percent of September 11 loan disbursements were for $50,000; the most frequently disbursed amount. Appendix IV presents more details on SBA’s September 11 disaster lending. In the weeks and months following the terrorist attacks, small business owners complained to the Congress about SBA’s Disaster Loan Program. Small business owners’ complaints, which SBA officials regarded as valuable feedback, involved issues such as: (1) the effect of the attacks on small businesses nationwide, (2) SBA’s communication with applicants with low English proficiency, (3) size standards for small businesses, (4) the loan underwriting criteria, and (5) the time required to receive loan approval. These complaints prompted SBA and the Congress to make several modifications to the Disaster Loan Program for September 11 victims, which we discuss in the following sections. Figure 3 provides a timeline of those changes; see appendix III for a summary of regulatory and statutory changes. Small businesses complained that eligibility for SBA loans was limited to firms located within the declared disaster areas, yet the September 11 terrorist attacks had caused economic injury to small businesses nationwide. Small business owners from across the nation, representing small airports as well as aircraft maintenance, travel, and tourism firms, reported losses in revenue as a result of the attacks, which forced them to furlough and/or terminate numerous employees. These small businesses identified SBA as a potential source of assistance to help them recover from the economic injury caused by the attacks. In response to these concerns, in October 2001, SBA issued regulations to make economic injury disaster loans available to small businesses nationwide, an unprecedented change to the Disaster Loan Program, according to SBA officials. SBA’s Expanded EIDL program enabled businesses outside the declared disaster areas to apply for loans to meet ordinary and necessary operating expenses that they were unable to meet, due to the attacks or related action taken by the federal government between September 11 and October 22, 2001. Small businesses in New York City also complained that the application process was particularly confusing and time-consuming for applicants with low English proficiency. To address these concerns, SBA printed informational packets in other languages, such as Spanish and Chinese, and also provided multilingual staff on-site who could speak Mandarin Chinese, Croatian, Arabic, and Spanish and was prepared to send employees with additional language capabilities to New York City. Small businesses, such as travel agencies, also argued that existing size standards—guidelines used to determine whether a firm was a small business on the basis of its annual revenue or number of employees—were overly restrictive. In February 2002, SBA modified the size standards for all September 11 loan applicants, allowing them to take advantage of recent inflation-based adjustments. In addition, in March 2002, SBA increased the threshold specifically for travel agencies adversely affected by the attacks from $1 million to $3 million annual revenues. In July 2002, SBA began to apply this increased size standard to all travel agencies, not just those affected by the terrorist attacks. In commenting on a draft of this report, SBA officials noted that the agency planned to increase the size standard for travel agencies generally, but applied that change sooner for travel agencies affected by the attacks. Small businesses affected by the terrorist attacks also complained that SBA’s underwriting criteria were too restrictive. For example, two small business owners objected to SBA’s requirement for collateral for their loans. They testified that SBA withdrew their applications because they would not use their homes as collateral. They argued that it was too risky to use their homes as collateral, especially since the survival of their businesses was uncertain. A New York Small Business Development Center official also questioned the appropriateness of SBA’s disaster loan underwriting criteria. He said that SBA should account for the location of the businesses affected by the attacks—New York City—where some factors relating to the high cost of doing business fall outside the norms. While SBA approved millions of dollars in loans, 52 percent of the loan applications were withdrawn or declined. SBA officials said that the agency makes every effort to approve each application by applying more lenient credit standards than private lenders. However, the officials said that they adhered to their credit standards to minimize losses and program costs. SBA data indicate that the 52-percent rate for withdrawing and declining September 11-related loan applications was not out of line when compared with other disasters, or with private lenders. By comparison, one bank in New York City reported a 42-percent decline rate for September 11-related loans, while another bank reported an 80-percent decline rate. The primary reasons SBA identified for withdrawing September 11 loan applications was that no Internal Revenue Service (IRS) record, which could provide independent documentation of the applicants’ income, was found, and the applicant failed to furnish additional information requested by SBA. According to SBA officials, the most common reasons for declining September 11 loan applications were the applicant’s inability to repay the loan and unsatisfactory credit. According to SBA, these are also the primary reasons that nearly two-thirds of all SBA disaster loan applications in fiscal year 2001 were withdrawn or declined by SBA. Applicants complained that the elapsed time between submitting an application and loan approval was too long. SBA responded to these complaints by implementing procedures in October 2001 to expedite two stages of the process - loan application processing and disbursement of loan funds. To expedite loan processing, loan officers calculated economic injury loan amounts based on the applicant’s annual sales and gross margin, instead of conducting a more extensive needs analysis. As of the end of fiscal year 2002, on average,SBA processed September 11 business loans in 13 days, compared with 16 days for disaster assistance business loans processed in fiscal year 20. To expedite disbursement of funds to September 11 victims in the World Trade Center and Pentagon disaster areas, SBA decreased the amount of documentation needed to disburse up to $50,000. Last, the Niagara Falls DAO made extensive use of printing selected loan documents in the field, enabling field staff to schedule loan closings within 1 or 2 days of the loan approval. SBA made initial September 11 loan disbursements within about 2 days of receipt of closing documents, compared with 3 days for initial disbursements for other disaster assistance loans, according to agency officials. See appendix II for a summary of the steps in processing SBA disaster loans. Despite SBA’s efforts to be responsive to the needs of small businesses affected by the terrorist attacks, business owners testified that SBA’s existing disaster program did not have the ability to provide adequate loans to small businesses within the declared disaster areas. In January 2002, the Congress enacted supplemental appropriations to SBA for $150 million and made several changes in the disaster loan program specifically for small businesses affected by the September 11 attacks. The changes included raising the maximum loan amount from $1.5 million to $10 million and deferring payments and interest accrual for 2 years. The Emergency Supplemental Act of 2002 also created the Supplemental Terrorist Activity Relief (STAR) Program that provided assistance to small businesses affected by the terrorist attacks through the 7(a) loan guaranty program, which is not part of the Disaster Loan Program. The 7(a) program is intended to serve small business borrowers who cannot otherwise obtain financing under reasonable terms and conditions from the private sector. Under this program, private-sector lenders provide loans to small businesses, which are guaranteed by SBA. Under the STAR program, SBA reduced the on-going fee charged to lenders on new 7(a) loans from 0.50 percent of the outstanding balance of the guaranteed portion of the loan to 0.25 percent. Although the fee reduction for lenders is the key feature of the STAR program, SBA officials anticipate that by making 7(a) loans more cost-effective for lenders, lenders will, in turn, make more small business loans and share the cost savings with their borrowers. As of the end of fiscal year 2002, SBA guaranteed about 4,700 STAR loans for $1.8 billion. (See app. III for a comprehensive list of modifications made to SBA’s Disaster Loan Program for September 11 victims.) SBA officials believed that many of the complaints about the disaster program resulted from the mismatch between victims’ expectations of SBA’s disaster program and the nature of the program. For example, when some victims were told that they could receive “assistance” from SBA, they assumed that the assistance would be in the form of grants instead of loans. SBA officials noted that the media usually does not draw distinctions among FEMA grants, SBA loans, and other forms of assistance available. SBA officials told us that they tried to minimize the public confusion about the nature of the assistance available from SBA by working closely with the media and public officials so that disaster victims would receive accurate information about SBA assistance. As stated earlier, the strategic plan describes the multiyear strategic goals. The performance plans describe the corresponding annual performance goals and the measures or indicators that will be used to assess progress in meeting them. During the past several years, we and SBA’s Inspector General have reviewed SBA’s performance plans and found the plans had significant limitations. Our review of the disaster lending portion of the 2003 performance plan found that the limitations identified in the previous reviews remain. We attribute some of these limitations to the specific nature of the measures SBA uses to describe the performance of the disaster lending program, while other limitations can be attributed to the description of program’s performance in the plan itself. In the past 5 years, SBA has used nine different measures to assess the performance of the Disaster Loan Program. Both we and SBA’s Inspector General have raised numerous concerns about these various measures in the past. The Inspector General found that SBA used inconsistent and subjective measures, and we found that the document used to report program performance to the Congress lacked key information that would have provided a more accurate picture of both the Disaster Lending Program’s performance measures and the results. We observed in our June 2001 report that SBA needed to improve the quality of the measures that it used to assess its performance. On the basis of our review of the 2003 performance plan, we have found that, as a group, the measures SBA currently uses to assess performance— the current measures (table 2, measures 4 to 9) continue to have numerous limitations, despite the guidance provided in SBA’s performance primer. First, the three output measures do not capture the notable progress the program has made in improving its loan processing; improvements that ultimately benefit disaster loan applicants and borrowers, such as better staffing processes and management of staff duties. Second, two of the three outcome measures are actually output measures and the third—a customer survey—has an important limitation. Third, other than the customer survey, SBA does not have measures to assess the intermediate or end outcomes of its Disaster Lending Program. Officials from SBA’s Disaster Area Offices questioned whether the three output measures—field presence within 3 days of a disaster declaration, processing loan applications within 21 days, and disbursing initial loan amounts within 5 days of receiving the closing documents—were appropriate indicators of timely service to disaster victims since they did not, for example, capture recent program improvements. SBA has had a 98- percent success rate in meeting the target for establishing a field presence each fiscal year since 1998. In light of this fact, one official characterized this measure as artificial and noted that it does not drive staff to improve their performance. Officials from the area offices said that improvements in planning, interagency coordination, and technology now can enable them to have staff onsite and preparing to assist disaster victims within 1 day of a disaster declaration. For example, field coordinators in two offices recently developed a database that tracks the level of staffing and other resources used to respond to various types of disasters. The coordinators used this information to help them more efficiently determine the resources required to respond to new disasters. Such preparedness enabled SBA officials to be in Lower Manhattan preparing to serve disaster victims the same day as the September 11 attacks. According to DAO staff, if there are delays in establishing a field presence, it is generally because SBA is waiting for decisions from state officials. SBA data and comments from DAO officials raise questions about the appropriateness of the second output measure—processing loan applications within 21 days of receipt (table 2, measure 5). One official suggested that providing timely, or well-timed, assistance does not always mean providing assistance in the shortest period of time. Rather, providing timely assistance means providing it when the disaster victims need it. While the 21-day measure does capture the elapsed time for multiple loan processing steps, the current target for this measure does not reflect improvements in past performance. The target was set at 70 percent for fiscal year 2000 and 80 percent for fiscal year 2001, and SBA’s performance significantly exceeded this target each year. Moreover, the actual time required for processing averaged 13 days in fiscal year 2001 and 12 days in fiscal year 2002. In fiscal year 2001, as indicated earlier, SBA’s average processing time for business loans was about 16 days. Home loans, which according to DAO officials are less complex, were processed during this period in an average of about 12 days. According to SBA data, the average processing time for both business and home loans improved in fiscal year 2002. The average loan processing time for business loans in fiscal year 2002 was about 13 days. The average time required to process the September 11 business loans was also about 13 days. The average processing time for the simpler home loans in fiscal year 2002 was about 10 days. Thus, SBA exceeded its performance target for both of these measures in fiscal year 2002. DAO officials attributed their faster processing times to several agencywide improvements that have expedited loan processing. For example, in the past SBA relied on hiring new and previously employed temporary staff to help permanent personnel to process loans. This strategy required DAO staff to train significant numbers of new temporary staff on SBA loan processing procedures, with each new disaster. In 2000, SBA implemented the Disaster Personnel Reserve Corps. Each DAO now has a list of reserve corps members who are already trained in SBA procedures and potentially available to assist in responding to disasters. According to DAO staff, utilizing the corps members enables SBA to potentially expedite processing by allowing temporary staff to begin processing loans immediately, because reservists are recruited and trained prior to the occurrence of the disaster. According to one DAO official, using the reserve corps helped her office attain the 21-day processing goal in fiscal year 2001. DAO staff also attributed faster loan processing to increased automation. Although, according to DAO staff, calculations to determine an appropriate loan amount are made electronically for all loans, some steps in loan processing are conducted manually. In 2000, SBA established the Home Expedited Loan Officer Report (HELOR) system so that loan decisions for home and personal property loans under $25,000 can be made automatically, based primarily on credit scores, rather than manually by the loan officer. DAO staff also cited DAO-level strategies that have expedited processing locally. For example, in the past, DAO staff who inspected a victim’s property to estimate the amount of property loss, referred to as loss verifiers, manually completed report forms and submitted the reports to the DAOs using a courier service. In 2002, one DAO pilot tested having their loss verifiers complete their inspection reports in the field using hand-held computers and submit their reports to DAO using electronic mail. One DAO official estimated that this automated approach reduced loan processing time and eliminated courier service expenses. In 2002, SBA began reporting data on the third output measure—ordering initial disbursements within 5 days of receiving closing documents (table 2, measure 9). Yet, DAO staff suggest that the target for this measure also does not reflect past performance and was set at a low threshold. According to DAO staff, before 2002, SBA had an internal goal of ordering disbursements within 3 days of receiving closing documents. When SBA included this measure in the performance plan, the disbursement target was increased to 5 days. SBA headquarters officials commented that the 5- day standard was set to accommodate counting weekend and holidays because the data system SBA uses to track disaster loan processing could not distinguish between workdays and non-workdays. Nonetheless, DAO officials are accustomed to the stricter 3-day standard, they indicated that the 5-day standard can be met with ease. For example, SBA made the initial disbursements on all approved September 11 loans in an average of about 2 days, and in fiscal year 2002, on average, SBA also made initial disbursements within an average of 2 days of receipt of closing documents. Moreover, according to one DAO official, the disbursement target was increased as DAOs were expediting their disbursement process. For example, as part of its response to September 11 borrowers, the Niagara Falls DAO reduced the amount of documentation required for September 11 victims from the World Trade Center and Pentagon disaster areas to receive disbursements of between $25,000 and $50,000, so that the DAO could more quickly disburse the remaining amounts. Since they found this strategy to be successful, the DAO official will recommend to his supervisors that this procedure be used for all future disasters. However, because the 5-day disbursement measure focuses only on the initial disbursement, it cannot capture other improvements that have been made to the multistep disbursement process. In commenting on a draft of this report, SBA indicated that the output measures were established based on what was determined to be a reasonable level of service based on an average year taking into account the amount of resources required. Because of the unpredictability of disasters, officials did not think it would be feasible to adjust production levels simply based on 1 year’s performance. In addition, they noted that large disasters could still generate more volume than SBA could handle quickly, especially if the pre-disaster staffing levels in all area offices were low and a large-scale recruitment and training effort were necessary. Even with some of the program improvements, they believed it would be very difficult and costly to maintain such levels during periods of multiple major disasters. Although SBA acknowledged that there may be a basis for modifying the output measures mentioned (effective field presence, processing loan applications in 21 days, and ordering initial disbursements within 5 days of loan closing), the officials believed that the modifications in the measures should be based on an average level of projected activity taking into consideration some of the permanent improvements they have made to the program. SBA officials indicated that the remaining three measures—number of homes restored to predisaster condition, number of businesses restored to predisaster condition, and customer satisfaction (table 2, measures 7, 8, and 6)—are used to assess the effect, or outcomes from lending to disaster victims. These outcome measures have limitations that are similar to the output measures. First, while the restoration of homes and businesses is a stated outcome in SBA’s strategic and performance plans, SBA does not actually measure the number of homes and businesses restored. As indicated earlier, headquarters officials said that SBA reports on the number of home loans approved as a proxy measure for the number of homes restored to predisaster condition. The agency also uses a proxy measure—the number of business loans approved—for the number of businesses restored to pre-disaster condition. The proxy measures that are used to report disaster loan outcomes have several limitations. First, these measures assess program outputs, loans approved, and not the stated outcomes—restoration of homes and businesses. Second, this proxy measure likely overestimates the number of homes and businesses restored. As SBA staff explained, even when loans are approved, borrowers might cancel the loan or reduce the amount of the loan to avoid using their homes as collateral. For example, about 10 percent of the loans approved for September 11 victims were subsequently cancelled by borrowers. Third, these indicators use annual numbers, which are not useful standards since they are highly dependent on factors outside of SBA’s control, such as the number of disasters that occur during a given fiscal year. A more useful indicator would be the percentage of homes and businesses receiving loans that were restored each year to pre-disaster conditions, which would enable a yearly comparison of performance. However, various SBA officials indicated that it is not easy to obtain evidence on the percentage of homes or businesses that have been restored after a disaster. One DAO official pointed out that though he supported conducting on-site progress inspections to measure whether homes or businesses have been restored, they are currently able to conduct on-site inspections for only a tiny fraction of the properties due to their limited travel budget. He has had to increasingly rely on the integrity of the applicants and SBA reviews of the borrowers’ receipts. Other staff indicated that some alternative strategies, such as reviewing pre- and post-disaster property tax assessments as a proxy measure for the restoration of homes, would also be problematic because of different economic conditions in different communities. To measure another outcome—customer satisfaction (table 2, measure 9)—SBA uses the results of its survey of successful loan applicants. SBA also uses this survey to evaluate the impact of the program. Yet, SBA’s method for conducting the survey has significant limitations. First, the survey measures the satisfaction of only a portion of the customers that the Disaster Loan Program serves. Every DAO director we interviewed indicated that all disaster victims are SBA customers and that a broader population should be surveyed. In 2001, we and SBA’s Inspector General made the same suggestion to SBA. As we indicated then, the current survey method is likely to produce positively skewed responses. SBA headquarters officials indicated that they are resistant to surveying those who were denied loans because they presumed the applicants’ responses would be negative. Yet, as described earlier in this report, it was the complaints from September 11 applicants that informed SBA of problems in the existing loan program and led the agency to revise the disaster program to better serve disaster victims. SBA does not currently plan to expand its fiscal year 2002 survey to a sample of all loan applicants. Second, the target set for this indicator, 80 percent, is set below what the program has reportedly achieved in the past; for example, 97 percent in 1998 and 1999, and 81 percent in 2000. Our review of the 2003 performance plan found that five of the six measures (table 2, measures 4, 5, 7, 8, and 9) that are currently used to assess the performance of SBA’s disaster lending focus on narrow program outputs rather than intermediate or end outcomes. As mentioned earlier, OMB guidance states that the plan should include outcomes when their achievement is scheduled during the fiscal year. In addition, recommendations from the Inspector General and guidance from us and within SBA have encouraged the use of outcome measures for this program. Only the customer satisfaction measure has the potential to assess one of the stated end outcomes from the Disaster Loan Program. The other intended outcomes from disaster lending, which might be measured annually or bi-annually, such as jobs retained or housing restored, are not measured. SBA may be able to measure, for those loans that are fully disbursed by the first or second quarter of the fiscal year, the percent of homes or businesses that have been fully restored at year’s end. In addition, SBA does not measure potential intermediate or end outcomes for the Disaster Loan Program. For example, as described earlier, some September 11 loan applicants criticized SBA’s underwriting criteria as too restrictive. In the past, SBA used two intermediate outcome measures, loan currency, and delinquency rates as listed in table 2, to reflect the quality of disaster loans. Yet, these measures were not included in the 2001 performance plan. Another potential intermediate outcome from the underwriting process, the retention of appropriate insurance, is not measured. As indicated in appendix II, SBA requires loan applicants to obtain insurance related to the nature of the disaster in order to receive a disaster loan. As one DAO official suggested, having insurance, such as flood insurance, potentially reduces the number of disaster loans required in areas that experience recurring disasters. As we reported previously, a greater reliance on insurance can reduce disaster assistance costs and could reduce the effect of a disaster on its victims. SBA headquarters staff said that, while they recognize that the proxy measures for the restoration of homes and businesses are inadequate and are aware that the customer survey only assesses the satisfaction of a portion of their customers, they have a limited ability to develop and use better outcome measures. The staff indicated that the very nature of disaster lending is unpredictable, so it is difficult to set performance targets for intermediate or end outcomes. A headquarters SBA official said that they are reluctant to measure and report intermediate or end outcomes that they cannot control. For example, one DAO official suggested that SBA cannot ensure that businesses that receive a disaster loan will survive. Other factors he suggested, such as differences in the willingness of people from different regions to acquire debt, will affect the borrower’s decisions. Other DAO officials indicated that conducting some end outcome measurement methodologies would be expensive, such as conducting on- site inspections of a sampling of homes and businesses to determine if they have been restored. We identified at least five features of the description of the Disaster Loan Program in the 2002 and 2003 performance plans (see table 3) that make it difficult to assess whether SBA is making progress in attaining its strategic goal. First, as discussed earlier, strategic goals and performance goals in annual plans may be identical, which is the approach SBA uses for the strategic and performance goals for the Disaster Loan Program. Between the 2002 and the 2003 performance plans, the performance goal changed from an outcome-oriented goal--helping families recover from disasters— to an output-oriented goal—streamlining disaster lending, without the required explanation. GPRA requires agencies to explain why they change performance goals, and OMB generally recommends that agencies used goals that are outcome-oriented. Second, the 2002 and 2003 performance plans do not define the linkages between each program output and each intermediate or end outcome. The plans do not explain how the outputs—disaster loans—are related to the performance indicators—field presence, customer satisfaction, and application processing timeframes. Third, the plans do not explain how the performance measures or indicators are related to either program outcomes or outputs. Fourth, the plans do not explain if the targets for the performance measures are set in anticipation of performance improving, regressing, or remaining the same. For example, some targets are at or below the actual performance in previous years. Fifth, performance indicators are added to the plans, or dropped—as shown in table 2— without explanation. These omissions make it difficult to understand how and if SBA expects to improve or sustain its loan processing performance. The performance plans also contain incomplete or inaccurate information on some performance indicators. For example, despite OMB and SBA guidance, validation and verification information on field presence and loan processing measures is omitted, making it difficult to assess the quality of performance data. In addition, the 2003 performance plan indicates that data on the number of homes restored to pre-disaster condition are based on-site inspections of homes. However, SBA officials indicated that they use a proxy measure—the number of original home loans approved—as the actual source of data for homes restored to pre- disaster condition. The September 11 terrorist attacks presented SBA with challenges it had never before faced. First, it had to provide loans to individuals and businesses near the disaster site as well as to small businesses located throughout the country. Rather than providing most of its loans for the repair and replacement of physical structures, SBA found itself dealing with large numbers of economic injury loans to businesses with amended guidelines. Second, given the extent of the economic effects in the wake of the attacks, SBA had to work with the Congress to modify the Disaster Loan Program so that larger loans could be provided to a broader population of disaster victims. Input from small business owners and advocates at congressional hearings was key to the changes that were made—changes that, whether temporary or permanent, will be useful for SBA and other federal agencies to consider in responding to future disasters. In this and previous work, we found that SBA’s Disaster Loan Program performance measures do not fully or adequately reflect the program’s actual performance. Viewing the performance measures in light of SBA’s response to the September 11 attacks underscores this finding. First, two current output measures describe only discrete steps of multistep processes, and some output measures use performance targets that have already been achieved or exceeded. Second, most of SBA’s measures assess program outputs instead of assessing measurable outcomes. We recognize the challenge of identifying end outcome measures, such as restoring a business to predisaster condition given the many factors involved in a business’ success. However, we note that intermediate outcome measures can provide meaningful information about the effect of SBA’s program. But SBA’s plan does not use intermediate outcome measures to link its output measures to the intended outcomes of the program. The one outcome measure SBA uses—a customer survey—is directed only at disaster victims who received loans. SBA misses the opportunity to get feedback from applicants who did not get loans. Yet SBA’s response to September 11 was modified partly as a result of the concerns small businesses expressed. Moreover, the limitations in the program’s performance measures and plans mean that congressional decisionmakers do not have an accurate description of SBA’s progress to help them make informed decisions in directing and funding the Disaster Loan Program. In order to better demonstrate program performance, we recommend that the Administrator of SBA direct the Office of Disaster Assistance to revise the performance measures for disaster lending to (1) include more outcome measures; (2) assess more significant outputs, such as service to applicants or loan underwriting; (3) report achievements that can be compared over several years, such as percentages; and (4) include performance targets that encourage process improvement rather than maintaining past levels of performance; revise and expand its current research to improve its measures and evaluate program impact. To improve its current measures SBA should conduct research, such as surveying DAO staff and reviewing the disaster, lending, and performance literature, to identify and test new outcome measures. To evaluate its program impact, SBA needs to revise its survey approach to survey all disaster loan applicants and to employ other methods, such as periodic analyses of regional statistics, to assess the economic impact of the program on local communities; and revise the disaster section of the performance plan to (1) establish direct linkages between each output and outcome and the associated performance measure; (2) accurately describe proxy measures as either an outcome or output measures; (3) accurately describe the validation and verification of performance measures; and (4) explain additions, deletions, or changes in the current goals or measures used from the previous year. We requested SBA’s comments on a draft of this report, and the Associate Administrator for Disaster Assistance provided written comments that are presented in appendix V. SBA generally agreed with our recommendations and said that they intended to review the existing performance measures and research new ways to evaluate program impact. SBA also provided some technical corrections and comments, which we incorporated as appropriate in this report. We are sending copies of this report to the Ranking Minority Member of the House Committee on Small Business, the Chairman and Ranking Minority Member of the Senate Committee on Small Business and Entrepreneurship, other appropriate congressional committees, and the Administrator of the Small Business Administration. In addition, this report will be available at no charge on GAO’s Web site at http://gao.gov. If you have any questions about this report, please contact M. Kay Harris, Assistant Director, or me at (202) 512-8678. Key contributors to this report were Kristy Brown, Sharon Caudle, Patricia Farrell Donahue, and John Mingus. To review the Small Business Administration’s (SBA) response to the September 11 terrorist attacks, we interviewed officials from the Office of Disaster Assistance (ODA) at SBA headquarters and officials from each of the four SBA Disaster Area Offices. In addition, we interviewed officials from SBA’s Office of the Inspector General. We also reviewed documents related to disaster lending policy and procedures, the agency’s response to the September 11 attacks, and other program documentation. In addition, we reviewed congressional testimony as well as regulatory actions taken by SBA, and legislative action by the Congress, in response to the terrorist attacks. To analyze SBA’s lending to September 11 victims, we obtained data from SBA’s Automated Loan Control System (ALCS), the system used by SBA to track disaster loan applications, approvals, and disbursements. We used these data to calculate descriptive statistics on the numbers of disaster loans, disbursement amounts, and other characteristics of the disaster lending to September 11 victims. We limited our analysis to loan funds approved through September 30, 2002. For our analysis of type of industry, we used the North American Industry Classification System (NAICS) code from the database and grouped the results by the first two letters of the code, which designate the general industry type. We determined the five industry types that received the largest percentage of SBA September 11 loans nationwide, grouping the remaining industries in the “other” category. We conducted similar analysis by industry for each type of September 11-related declaration. We ascertained how information for the ALCS database was collected and maintained to determine its reliability, and we found the information to be reliable for our purposes. We repeatedly consulted with SBA headquarters officials, including those responsible for managing ALCS, during our analyses to ensure our understanding of various data elements was correct. We also obtained summary statistical reports from SBA describing disaster lending during fiscal years 2001 and 2002. To review and analyze SBA’s performance plans and measures for its Disaster Loan Program, we reviewed SBA’s strategic plan for the 2001-2006 period and performance plan for fiscal years 2002 and 2003. A knowledgeable staff member from our Strategic Issues Team also reviewed the plans for compliance with the Office of Management and Budget’s (OMB) guidance on the Government Performance and Results Act (GPRA) of 1993 guidance. We also reviewed SBA’s Inspector General’s recent review of the disaster section of recent performance plans, SBA’s primer on performance measurement, and our recent reviews of SBA. Our overall assessment of SBA’s performance plans was generally based on our knowledge of the Disaster Loan Program and OMB’s guidance on developing strategic and performance plans. We conducted our work between June 2002 and January 2003 in Washington, D.C.; Niagara Falls; Atlanta; and Fort Worth in accordance with generally accepted government auditing standards. State and federal officials conduct a preliminary damage assessment to estimate the extent of the disaster and its impact on individuals and public facilities. SBA participates in the damage assessment when the damages include homes and businesses. The President, USDA, or SBA makes a disaster declaration. SBA establishes field presence – SBA staff arrive at the disaster site and take actions to initiate delivery of disaster assistance. SBA loan officers meet with disaster victims, explain the loan process, and issue applications at the Federal Emergency Management Agency (FEMA) or SBA disaster offices. SBA screens the submitted applications for completeness and to make sure all necessary documentation has been provided. -Home loan application package includes the application, listing of property damage, and authorization for SBA to access applicant’s tax information. -Business loan application package includes the application, a schedule of liabilities, and personal financial statements and tax information authorization for each proprietor, partner, affiliate, or other type of owners. Physical loan applications are forwarded to loss verifiers who conduct on-site appraisals of the damaged property to estimate the cost of restoring the property to pre-disaster condition. Economic injury applications may be sent directly to a Disaster Area Office (DAO) for processing. Once the application arrives at the DAO, SBA staff review the application, examining such issues as duplication of benefits; credit history; criminal record; tax returns; history on other SBA loans; and the history on other federal debt. The applicant’s losses or economic injury are calculated. The loan officer determines whether the applicant has satisfactory credit and the ability to repay the loan; the legal department determines whether there are any legal or regulatory restrictions on receiving a disaster loan. If the applicant meets SBA’s underwriting criteria, then the loan is approved, using the amount of verified losses as the basis for the loan amount. Closing documents are prepared and mailed to the applicant. Applicants are required to obtain insurance. Hazard insurance is required before disbursement over $10,000 for physical loans, and over $5,000 for economic injury loans. Flood insurance is required for properties located in Special Flood Hazard areas before any disbursement can be made. Maximum initial disbursement without collateral: physical loans - $10,000; economic injury loans – $5,000 Initial disbursement with collateral, preferably the applicant’s home: $25,000. Total disbursements with proof of ownership of the damaged property: physical loans and economic injury loans - $25,000. Total disbursements with proof of title insurance: physical loans and economic injury loans - $250,000. For small businesses in declared disaster area, maximum disaster loan amount is $1.5 million, small non-profit institutions and select financial maximum disaster loan amount is $10 million, small nonprofit institutions and select financial and insurance firms were ineligible for economic injury assistance, and insurance firms eligible for economic injury assistance, interest begins to accrue when disbursement is no interest accrues for 2 years following payments of principal and interest are deferred payments of principal and interest are deferred for 4 months. for 2 years following issuance. For 7(a) lenders: 7(a) loan fee = .50 percent of outstanding balance. For 7(a) lenders to small business adversely affected by attacks: 7(a) loan fee = .25 percent of outstanding balance. Economic injury loans available only to small businesses within the declared disaster area, directly affected by the disaster. Economic injury loans available to small businesses nationwide, adversely affected by the disaster or by related action taken by the federal government. Inflation-adjusted size standards, generally effective February 2002, were effective September 11, 2001, for businesses applying for economic injury loans as a result of the terrorist attacks. Threshold for “small” travel agencies is $1 million in annual revenues. For World Trade Center and Pentagon areas, businesses do not have to sustain physical months of gross margin, with maximum loan amount the lesser of (1) 3 times the SBA verified physical loss or (2) $100,000. if business in operation, economic injury loan amount based on up to 3 months gross margin, with maximum loan amount of $200,000; and if business not in operation, economic injury loan amount based on up to 6 months gross margin, with maximum loan amount of $350,000. For EIDL applicants nationwide without any property damage, The loan amount was limited to the lesser of 2 months gross margin or $50,000. Expedited disbursement process Disbursements greater than $25,000 require a title search. For World Trade Center and Pentagon areas, title search is not required for disbursements up to $50,000. SBA’s response to the September 11 disaster commenced immediately after the terrorist attacks occurred, when SBA disaster officials established communication with FEMA and state emergency management officials. By the afternoon of September 11, disaster officials from SBA’s Niagara Falls DAO were in Lower Manhattan coordinating the agency’s recovery efforts with the overall federal response. Once the President declared the World Trade Center attack a major disaster, SBA designated the immediate disaster area of the World Trade Center (“WTC Immediate”) as the five boroughs of New York City, and the contiguous area of the World Trade Center (“WTC Contiguous”) as including two other counties in New York and four counties in New Jersey. SBA officials began meeting with disaster victims on September 13. Following the President’s declaration of the Pentagon attack as a major disaster on September 21, SBA established the immediate area of the Pentagon, which was comprised of Arlington County, Virginia, and the contiguous area of the Pentagon, which included additional counties in Maryland, and Virginia (“Pentagon Contiguous”), and parts of the District of Columbia. FEMA extended the declared disaster areas on September 27 as the widespread impact of the terrorist attacks became more apparent. The immediate area of WTC was extended to include 10 additional counties in New York, including the 2 counties initially included in the WTC Contiguous area. The extension also added additional counties in New York and New Jersey, as well as counties in Connecticut, Massachusetts, and Pennsylvania to the existing WTC Contiguous area. See figure 4 for a map of the disaster areas. As the United States began to deploy military personnel in response to the terrorist attacks, small businesses affected by the loss of employees who serve as reserve military personnel were eligible to apply for a disaster loan under the Military Reservist EIDL Program. We obtained and analyzed SBA data on the loans it approved in response to September 11, 2001, through September 30, 2002. The distribution of September 11 lending varied significantly by amount, geographic location of recipients, and the types of loans. Nearly half of the September 11 loan funds disbursed by the end of fiscal year 2002 was distributed to disaster victims from New York. The balance was disbursed across the country through the expanded EIDL Program. Unlike other recent disasters, almost all of the disbursed loan funds went to businesses rather than homeowners. In just over 1 year, SBA approved more than 9,700 home and business loans totaling $966 million for victims of the September 11 attacks, disbursing about $895 million, or 93 percent, by the end of fiscal year 2002. The peak in monthly disbursement amounts for all September 11 loans was in January 2002 at $120 million. The agency expects to fully disburse $924 million—or 96 percent of the amount approved—due to loan increases, decreases, and cancellations. As of the end of fiscal year 2002, about 10 percent of approved September 11 loans were cancelled by borrowers, compared with 16 percent of approved disaster loans in fiscal year 2001. The greatest percentage of loan cancellations occurred in the immediate area of WTC, where 13 percent of the loans in this area were cancelled. The contiguous area of the Pentagon experienced the greatest percentage of loan increases, where 11 percent of September 11 loans were increased from their original approved amount. Given the difference between the approved amounts and the disbursed amounts—due to loan increases, decreases, and cancellations—we have chosen to describe the distribution of September 11 loans in terms of the actual disbursed loan amounts. September 11 loan disbursement amounts range from $300 to $1.5 million, with a median amount of $50,000. Fifty percent of disbursements were between $18,700 and $119,700. Eleven percent of September 11 loan disbursements were for $50,000, the most frequently disbursed amount. In commenting on our draft SBA, indicated that the agency applied the expedited EIDL process for “stand-alone” EIDLs, that is, applicants without any property damage. The loan amount was limited to the lesser of 2 months gross margin or $50,000, which SBA described as the reason why the most commonly disbursed amount was $50,000. The distribution of September 11 loans also varied by state, type of loan, declaration area, and by business industry. Typically, about 80 percent of approved SBA disaster loans are home loans to repair physical damage to homes and personal property. However, about 97 percent of September 11 loans were disbursed to businesses. Even in New York City, only 6 percent of loans were disbursed to households. SBA officials attribute this difference from the historic lending pattern to the fact that the physical damage caused by the terrorist attacks was concentrated in the World Trade Center business district and at the Pentagon. Seventy percent of the businesses receiving September 11 loans had 10 or fewer employees, while 50 percent had 5 or fewer employees. Businesses with more than 100 employees received less than 2 percent of disbursed loan funds. Overall, only about 9 percent of September 11 loan applicants in the declared disaster areas sustained physical losses compared with about 80 percent of disaster loan applicants in fiscal year 2001. Consequently, 92 percent of September 11 loans went to small businesses that suffered economic injury, but no physical damage, and about 5 percent of the loans were disbursed to businesses with physical damage from the attacks. Although SBA provided loans to affected small businesses nationwide, about 45 percent of all disbursed September 11 loan funds were distributed to applicants in New York State. Of that 45 percent, approximately 36 percent was disbursed to disaster victims in New York City. As shown in figure 1, Florida received the second greatest percentage of disbursed September 11 loans (11 percent), followed by California (6 percent), New Jersey (4 percent), Texas (3 percent), and Virginia (3 percent). More than half of all September 11 loan funds were disbursed to small businesses outside of the immediate and surrounding areas of the World Trade Center and the Pentagon. SBA data indicate that, in general, businesses located closest to the WTC disaster site received smaller loans than businesses near the Pentagon and nationwide. For example, the median disbursement in the immediate area of WTC, specifically New York City, was about $40,000, while the median disbursements under the expanded EIDL Program and in the area of the Pentagon were $50,000 and $60,000, respectively. SBA disaster officials reasoned that firms near WTC may have received smaller SBA loan disbursements because there were other resources available to them, whereas SBA was the sole source of assistance for affected small businesses outside of New York City. In addition, SBA officials suggested that since many September 11 loan recipients in New York City were service-oriented firms, they had fewer operating expenses than the more capital-intensive loan recipients nationwide. SBA loan disbursement data appear to indicate that a wide variety of businesses received September 11 loans. As shown in figure 2, no one sector of the economy received a substantial portion of these loans. We summarized SBA’s loan data according to the type of business that received the loan. The manufacturing sector received the greatest percentage of September 11 loans, though this represents only about one-sixth of these loans. We combined business types with less than 7 percent of the loans into an “other” category, which includes such sectors as retail trade and waste management. As shown in figure 6, the distribution of the loan disbursements by industry for the expanded EIDL was similar to the distribution for all September 11 loans, with the manufacturing sector receiving the second largest portion of these loan disbursements. In contrast, to the distribution of loan disbursements at the national level, the greatest percentage of disaster loan funds in New York City, and the immediate and contiguous areas of the Pentagon was disbursed to the professional, scientific, and technical service industry. The General Accounting Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to GAO Mailing Lists” under “Order GAO Products” heading.

Summary: The September 11 terrorist attacks and subsequent federal action had a substantial impact on businesses in both the declared disaster areas and around the nation. In the aftermath of the attacks, the Congress, among other actions, appropriated emergency supplemental funds to the Small Business Administration (SBA) to aid September 11 victims. Given the uniqueness of this disaster and changes in the program, GAO analyzed SBA's lending to September 11 victims, as well as the loan program's performance goals and measures. As part of its response to the September 11 terrorist attacks, SBA modified several aspects of its Disaster Loan Program and its processes. For example, SBA increased the maximum loan amounts available and decreased the amount of documentation required for certain loans. By the end of fiscal year 2002, approximately $1 billion in loans had been approved for victims of the attacks. On average, SBA processed business loans to September 11 victims in an average 13 days compared with 16 days for business loans to other disaster victims in fiscal year 2001. Like other federal programs, SBA has developed a multiyear strategic goal for the Disaster Loan Program--helping families and businesses recover from disasters--and has developed annual goals and measures to assess its yearly progress toward attaining their strategic goals. GAO reviewed the measures and found that they have numerous limitations. For instance, these measures do not capture the notable progress the program has made in improving its loan processing--progress that ultimately affects disaster loan applicants and borrowers. The inadequacies of SBA's measures are especially evident when considered in light of the agency's performance in responding to the September 11 terrorist attacks. GAO attributes some of these limitations to the nature of the measures SBA uses to describe the performance of the Disaster Loan Program, while others can be attributed to the description of the program's performance. Without better performance measures and plans, the Congress does not have an accurate description of SBA's annual progress toward helping Americans recover from disasters.