Small Business Administration: Status of Achieving Key Outcomes and Addressing Major Management Challenges
Released: 2001-07-23T00:00:00

Letter

Background

GPRA is intended to shift the focus of government decisionmaking, management, and accountability from activities and processes to the results and outcomes achieved by federal programs. New and valuable information on the plans, goals, and strategies of federal agencies has been provided since federal agencies began implementing GPRA. Under GPRA, annual performance plans are to clearly inform the Congress and the public of (1) the annual performance goals for agencies’ major programs and activities, (2) the measures that will be used to gauge performance, (3) the strategies and resources required to achieve the performance goals, and (4) the procedures that will be used to verify and validate performance information. These annual plans, issued soon after the transmittal of the President’s budget, provide a direct linkage between an agency’s longer-term goals and mission and day-to-day activities. Annual performance reports are to subsequently report on the degree to which performance goals were met. The issuance of the agencies’ performance reports, due by March 31, represents a new and potentially more substantive phase in the implementation of GPRA—the opportunity to assess federal agencies’ actual performance for the prior fiscal year and to consider what steps are needed to improve performance and reduce costs in the future.

SBA is responsible for aiding, counseling, assisting, and protecting the interests of the nation’s small businesses and for helping businesses and families recover from natural disasters. SBA is also a financial institution with significant commitments and exposure. As of September 30, 2000, SBA’s total portfolio was about $52 billion, including $45 billion in direct and guaranteed small business loans and other guarantees and $7 billion in disaster loans. Since its inception, SBA has, among other things, made 1.1 million small business loans and has approved 1.4 million disaster loans to individual homeowners, renters, and businesses of all sizes. SBA also administers the 8(a) business development program, which is designed to assist small disadvantaged businesses become successful through counseling, training and assistance in obtaining federal contracts. SBA also provides entrepreneurial assistance through partnerships with private entities that offer small businesses counseling and technical assistance.

Assessment of SBA’s Progress and Strategies in Achieving Selected Key Outcomes

This section discusses our analysis of SBA’s performance in achieving its selected key outcomes and the strategies the agency has in place, particularly strategic human capital management and information technology, for achieving these outcomes. In discussing these outcomes, we have also provided information drawn from our prior work on the extent to which the agency provided assurance that the performance information it is reporting is credible.

Small Business Self- Reliance and Success in the Competitive Marketplace

SBA’s performance in achieving the goal of helping small businesses succeed is mixed. SBA reported that it met about half of its quantifiable measures relating to this goal. When goals were not met, SBA did not identify strategies for achieving the unmet goals in the future. At the time SBA issued its fiscal year 2000 report, data were not yet available for the percentage increase of federal prime contracts to small businesses, small disadvantaged businesses, women-owned businesses, and HUBZone small businesses. SBA provided us with data that showed that it met only the small disadvantaged business goal.

We had difficulty assessing SBA’s progress for this outcome because SBA continues to use output measures without showing how strategies and measures relate to helping businesses succeed. In support of this goal, SBA lists four strategies, but does not provide an explanation as to how these strategies support the overall goal accomplishment of helping businesses succeed. In addition, SBA included several measures of the number of loans, but often did not correlate the impact of increasing the number of loans on small business success. For example, one goal is increasing the number of loans to veteran-owned businesses to 7,395. The narrative does not explain why this number is important. In the area of access to business development, SBA measures the number of clients being counseled or trained, but does not include outcome information on the impacts of the counseling or training. SBA’s IG suggested that SBA redefine current output measures because inconsistencies in the methods used to count clients mean that one client may be counted more than once. SBA recognizes that it still relies on output measures, and that for future reports, it will make an effort to explain how the accomplishment of these output measures support the established outcome.

In addition, two of the performance indicators — “expand research, analyses, and publication of information,” and “improve small business impact analyses of regulatory alternatives” – were stated as qualitative indicators and the actual performance stated as “achieved” at 100 percent. However, SBA did not provide criteria or a performance indicator that would explain how it assessed accomplishment of these goals.

SBA’s performance report includes the following strategies for the accomplishment of this outcome: (1) improving access to capital and credit, (2) increasing access to procurement opportunities, (3) act as a voice for America’s small businesses, and (4) providing access to entrepreneurial development assistance. These strategies were shown as goals in SBA’s fiscal year 1999 report. Because of the lack of explanation in the plan and report regarding how these strategies relate to helping businesses succeed, we were unable to assess whether they are clear and reasonable.

SBA’s performance plan does not generally categorize its human capital or information technology strategies by outcome, so we could not determine specifically how this outcome of helping businesses succeed is impacted by these strategies. However, in its plan, SBA states that automation and asset sales will allow staff to shift their attention from the processing of transactions to using information to analyze programs, activities, and performance. SBA states that it plans to continue to develop and deliver training in marketing and outreach, commercial credit analysis, lender oversight, and lender relations. Additional strategies for human capital and information technology are discussed in the plan and reports as part of SBA’s overall internal goal of “improving SBA management.” In its performance plan, SBA provides a description of its use of interagency coordination as a strategy for this outcome. For example, SBA states that it meets regularly with the Commerce-directed Trade Promotion Coordinating Committee to discuss challenges, propose program initiatives, work on developing new products, and avoid duplication of effort.

Assistance for Businesses and Families Recovering From Disasters

SBA reported that it succeeded in meeting its fiscal year 2000 goal of providing timely service to disaster victims, yet we have concerns about the quality of SBA’s measures. For example, while SBA’s fiscal year 2000 performance report shows that it met its 3-day field presence measure, SBA’s IG determined that this measure has not consistently been applied by disaster area offices. For example, two disaster area offices defined field presence as the date they arrived at the disaster scene, while one area office defined the term as the date it was available to assist disaster victims. SBA did not address this discrepancy in its performance report. Also, for fiscal years 1999 and 2000, SBA’s performance report shows that it met its underwriting compliance rate goal. However, SBA’s IG reported that it did not consider the underwriting compliance rate to be an objective indicator. SBA did not provide a discussion on this issue in its fiscal year 2000 performance report.

Furthermore, as shown in table 1, SBA adjusted its target goal annually for its measure of processing disaster loans within 21 days, depending on the extent to which the goal was accomplished in the previous year. In its performance report, SBA explained that its performance deteriorated because of the need to respond to widespread multiple disasters in fiscal years 1998 and 1999, including Hurricane Floyd, that affected 10 states along the East Coast and caused major widespread flooding in Texas. However, the report does not explain SBA’s justification for changing its target goal rates in order to align them with their actual annual performance.

SBA’s fiscal year 2000 report acknowledges that SBA considers the disaster assistance goal difficult to achieve, due to the unpredictability of disaster activity. Our 2001 Performance and Accountability Series Report on SBA pointed out that one step that would assist SBA in stabilizing the indicator for this goal would be to modernize its loan processing in order to consistently meet its timeliness goals. Presently, few of the processes followed by SBA loan officers are automated in an integrated manner, and this lack of automation contributes to processing time. SBA officials said that they are taking various actions to revise and/or clarify the measures for the disaster assistance goal. For example, the 2002 performance plan states that SBA has developed a draft definition of “effective field presence” to be applied by its Area Directors. Also, SBA plans to incorporate a “customer satisfaction indicator” as a measure for this goal, which will be designed to assess issues related to the quality and timeliness of services provided.

In its 5-year strategic plan, SBA refers to two completed evaluations to assist in formulating its strategies for establishing this indicator. One survey evaluated customer satisfaction with the services provided to recipients of disaster loans approved after Hurricane George. Another survey was also done to measure the customer satisfaction level during the disaster loan making process. According to the strategic plan, the surveys indicated a high customer satisfaction rate. However, we identified the following limitations with the results from SBA’s disaster loan making survey: (1) the survey population may not have been representative of all recipients, (2) those who were denied loans were not surveyed, (3) the survey response rate was low, and (4) the selection of response categories may have skewed the responses toward higher ratings.

Neither SBA’s plan nor its report discusses strategies for accomplishing this goal. However, SBA’s Strategic Plan for fiscal years 2001 through 2006 mentions strategies that include (1) developing a flexible infrastructure of resources that can be applied to a disaster area, (2) using the Internet to facilitate the disaster home loan application process, and (3) outsourcing disaster home loan servicing and carrying out asset sales. Since these strategies were not discussed in the plan and report, we could not determine how they relate directly to goal accomplishment.

SBA’s performance plan does not generally categorize its human capital or information technology strategies by outcome, so we could not determine specifically how this outcome of providing assistance to families and businesses recovering from disasters is impacted by these strategies. Additional strategies for human capital and information technology are a part of SBA’s overall internal goal of “improving SBA management.” In its performance plan, SBA provides a description of its use of interagency coordination activities as a strategy for this outcome. For example, SBA states that systematic coordination among federal, state, and local agencies is necessary before and during a disaster to promote efficient, consistent action. SBA states that this coordination is described in the federal response plan and is overseen by the Federal Emergency Management Agency.

Eligible Small Business Participation and Success Due to SBA Programs

SBA’s reported success in achieving the portion of its outcome that more eligible small disadvantaged businesses participate in its programs was mixed. SBA reported that it met its output measure that at least 60 percent of small disadvantaged firms (including 8(a) firms) receive federal contracts and its measure that at least 3.4 percent of 8(a) firms receive mentoring. However, SBA reported that it did not meet its measure of certifying a total of 12,000 small disadvantaged business firms as being eligible to receive price credits when bidding on prime contracts or to perform as subcontractors in certain industries. SBA said that it is reevaluating its goal for the number of small disadvantaged business firms it will certify because the number of firms seeking certification was much smaller than projected. SBA does not explain the strategies it is using to reevaluate this goal. SBA’s original projection of the number of firms it would certify was based on the number of firms that had previously self- certified as small disadvantaged businesses. Our work has shown that a variety of factors, including uncertainty about the program, the administrative and financial burden of applying, and questions regarding the benefits of the program may have contributed to the number of small disadvantaged business certifications being lower than anticipated by SBA.

It is not possible to determine SBA’s progress in accomplishing the portion of its outcome that more eligible small disadvantaged businesses will become more successful because SBA’s current success measure is not aligned with the mission of the 8(a) business development program, SBA’s key program in this area. SBA’s measure for the 8(a) business development program does not capture program success in terms of the number of competitive firms that exit the program without being unreasonably reliant on 8(a) and that can compete in the mainstream economy, as required by the Small Business Act, as amended. SBA’s performance report states that it will measure achievement by the percentage of firms that are economically viable 3 years after graduation and states that SBA began capturing the data for this measure in fiscal year 2000. SBA’s basis for reporting that it has just begun to collect this data is unclear because SBA reported on actual performance in previous years.

SBA includes this outcome in its plan as a part of its outcome of helping businesses succeed. The strategies include: (1) developing methods to improve access to contracting opportunities for 8(a) firms; (2) working with other agencies to reform and improve the program; and (3) developing legislative, regulatory, and procedural documentation for the reform recommendations. In the human capital area, SBA noted, among other things, that it plans to provide sufficient financial and analytical training to Business Opportunity Specialists to help them more accurately evaluate a company’s business profile and competitive potential. SBA’s performance plan does not generally categorize its information technology strategies by outcome, so we could not determine specifically how it is impacted by this strategy. Additional strategies for information technology are a part of SBA’s overall internal goal of “improving SBA management.” In its performance plan, SBA provides a description of its use of interagency coordination activities as strategies for this outcome. For example, SBA states that it participates in monthly meetings with other federal agencies to discuss strategies to increase small business participation in federal contracts.

Comparison of SBA’s Fiscal Year 2000 Performance Report and Fiscal Year 2002 Performance Plan With the Prior Year Report and Plan for Selected Key Outcomes

For the selected key outcomes, this section describes major improvements or remaining weaknesses in SBA’s (1) fiscal year 2000 performance report in comparison with its fiscal year 1999 report and (2) fiscal year 2002 performance plan in comparison with its fiscal year 2001 plan. This section also discusses the degree to which SBA’s fiscal year 2000 report addresses concerns and recommendations by the Congress, us, SBA’s IG, and others.

Comparison of Performance Reports for Fiscal Years 1999 and 2000

SBA made some improvements in its performance reporting, but in certain areas, additional effort is warranted. The fiscal year 2000 performance report includes more clearly labeled headings and provides more guidance so that the reader can quickly identify specific information. For example, the 2000 performance report includes a section that summarizes SBA’s programs and a section that exclusively discusses SBA’s goals, resources, and outcomes. Another strength of the 2000 report is that it concisely presents SBA’s status in responding to management challenges identified by SBA’s IG, documents ongoing and closed GAO and IG reviews, as well as the number of recommendations associated with each.

However, several weaknesses we previously noted remain in SBA’s report. For example, the 1999 performance report did not discuss data limitations that could affect the quality of data used by SBA to assess performance. Also, although the 1999 report generally discussed the reasons certain goals were not met, it did not include time frames or schedules for achieving the unmet goals. In comparison, the 2000 performance report did discuss data limitations, but did not include time frames or schedules for achieving the unmet goals. The 2000 performance report does include a narrative explanation of why the goal accomplishment fell short, but as with the 1999 report, it does not provide strategies for meeting unmet goals. In addition, SBA does not sufficiently link its strategies to indicators and measures and does not consistently provide summarized explanations about the data that are presented.

Another limitation of the fiscal year 2000 report is that it did not provide a brief summarization in its section that addresses the number of indicators that were met. SBA did present this information in its fiscal year 1999 report. Although this information is included elsewhere in the report, we believe that such a narrative leading into the ‘Performance Indicators’ section for the 2000 report would have been helpful in determining SBA’s approach in presenting these indicators. For example, in fiscal year 1999, SBA had a total of 59 indicators, and in 2000, it only had 16. Since the data are presented without SBA’s explanation for the substantial reduction of indicators, we did not have any insight into SBA’s rationale for doing so. We believe that the lack of a discussion on this action inhibits our capability to track a clear link between the identified strategies and the corresponding indicators and measures in order for decisionmakers to determine if progress has been made in achieving outcomes.

Comparison of Performance Plans for Fiscal Years 2001 and 2002

We noted that SBA’s presentation of information in the plan was an improvement from the fiscal year 2001 plan. The layout of data was better designed, and SBA included its organizational chart, as well as more graphics to illustrate its points. Also, the fiscal year 2002 plan addresses SBA’s mission, strategic goals and objectives, core values, and budgetary requirements. Another improvement from the fiscal year 2001 plan is that SBA’s fiscal year 2002 plan generally discusses SBA’s crosscutting activities with other agencies and discusses human capital resources needed to achieve SBA’s planned performance. However, the plan lacks a clear link of how strategies relate to outcomes and how they link to indicators and measures. Specifically, it is difficult to ascertain how SBA’s measures will indicate successful performance beyond meeting output targets.

SBA’s Efforts to Address Its Major Management Challenges Identified by GAO

We have identified two governmentwide, high-risk areas: strategic human capital management and information security. Regarding strategic human capital management, we found that SBA’s performance plan did have goals, but not measures, related to strategic human capital management, but SBA’s performance report explained its progress in resolving strategic human capital management challenges. For example, in July 2000, we said that SBA had begun to take the steps necessary to better manage its human capital activities, but needed to do more. SBA reported that, among other things, it has (1) issued a comprehensive workforce transformation plan, (2) issued a contract to conduct a workload and staffing analysis of SBA headquarters, and (3) provided leadership training to executives and senior managers. With respect to information security, we found that SBA’s performance plan did have goals, but not measures, related to information security, and SBA’s performance report explained its progress in resolving its information security challenges. For example, SBA stated that it has, among other things, (1) committed more than $1.2 million in personnel and contract support to enhance computer security, (2) increased the number of authorized personnel for information technology security, and (3) issued an updated computer security policy document.

As shown in table 2, we identified four major management challenges facing SBA. We found that SBA’s performance report discussed progress in resolving many of the challenges we identified, but it did not discuss SBA’s progress in resolving the challenge of streamlining and automating disaster loan assistance to improve timeliness. Of the four major management challenges we identified, SBA’s performance plan had (1) a goal and measures related to one of the challenges; (2) a goal, but no measures, directly related to one of the challenges; (3) a goal and measures indirectly applicable to one of the challenges; and (4) had no goals and measures related to the last challenge.

Conclusions

GPRA is intended to shift the focus of government decisionmaking, management, and accountability from activities and processes to the results and outcomes achieved by federal programs. In order for such a shift to occur, the information presented in GPRA plans and reports needs to be presented in a logical format that allows the reader to easily discern how the agency plans to accomplish its goals and objectives and how the measures will indicate successful performance beyond meeting output targets. We had significant difficulty assessing SBA’s progress in achieving its outcomes because of weaknesses in the report and plan. Although improved over last year in terms of presentation issues, SBA’s fiscal year 2000 performance report and fiscal year 2002 performance plan do not follow GPRA guidance in several areas. SBA did not provide criteria or a performance indicator to explain the accomplishment of its qualitative measures. We believe, as we stated in our fiscal year 1999 report, that SBA is still relying heavily on outputs without sufficiently linking them to achievement of the outcome. SBA’s performance report also lacks information about time frames or schedules and strategies for achieving unmet goals. In our view, SBA’s fiscal year 2000 performance report and 2002 performance plan do not present information in a logical manner linking strategies to outcomes, indicators, and measures.

Recommendations

To make SBA’s plan and report more useful for decisionmakers and more consistent with GPRA, OMB Circular A-11, and related guidance, we recommend that the Administrator of SBA ensure that the fiscal year 2001 performance report and fiscal year 2003 performance plan (1) clearly link strategies to outcomes, indicators, and measures; (2) present criteria or a performance indicator to explain the accomplishment of the goal when using qualitative measures; and (3) provide information about strategies, time frames, and schedules for achieving unmet targets.

Scope and Methodology

Our evaluation was generally based on the requirements of GPRA, the Reports Consolidation Act of 2000, guidance to agencies from OMB for developing performance plans and reports (OMB Circular A-11, Part 2), previous reports and evaluations by us and others, our knowledge of SBA’s operations and programs, our identification of best practices concerning performance planning and reporting, and our observations on SBA’s other GPRA-related efforts. We also discussed our review with agency officials in the Office of the Administrator and with SBA’s IG. The agency outcomes that were used as the basis for our review were identified by the Ranking Minority Member of the Senate Governmental Affairs Committee as important mission areas for the agency and generally reflect the outcomes for all of SBA’s programs or activities.

We identified the major management challenges confronting SBA, including the governmentwide, high-risk areas of strategic human capital management and information security, in our January 2001 performance and accountability series and high-risk update and SBA’s IG identified them in December 2000. We did not independently verify the information contained in the performance report and plan, although we did draw from other GAO work in assessing the validity, reliability, and timeliness of SBA’s performance data. We conducted our review from April through June 2001 in accordance with generally accepted government auditing standards.

Agency Comments and Our Evaluation

SBA provided written comments on a draft of this report. In its response, SBA said that it intended to conduct a major review of its current plan once it has an Administrator confirmed and senior political leadership in place. In revising the plan, SBA said that it would take into account our comments and would fully comply with GPRA, as well as promote President Bush's agenda. SBA disagreed with our conclusion that its fiscal year 2000 performance report and 2002 performance plan do not present information in a logical manner linking strategies to outcomes, indicators, and measures; however, SBA did not comment specifically on the report's recommendations. SBA said it believes the 2002 budget and performance plan offers a clear logical construct that helps the reader to understand how SBA activities can contribute to the success of a firm, as defined by job creation, revenue generation, and viability in the marketplace. SBA also said that it used logical diagrams extensively to convey how activities produce outputs, which in turn contribute to outcomes. We continue to believe that it is difficult for a reader to follow SBA's report and plan. While SBA employed diagrams and tables that should have helped the reader, inconsistencies in SBA's use of terms such as strategies and outcomes make following SBA's logic a laborious process. SBA also provided technical clarifications, which were incorporated as appropriate. SBA’s comments are in appendix II.

As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this report. At that time, we will send copies to appropriate congressional committees, the Acting SBA Administrator, and the Director, Office of Management and Budget. Copies will also be made available to others on request.

If you or your staff have any questions, please call me at (202) 512-8678. Key contributors to this report were Susan Campbell, Cheri Truett, and Tina Morgan.

Appendix I: Observations on SBA’s Efforts To Address Its Major Management Challenges

Table 3 identifies the major management challenges confronting the Small Business Administration (SBA), including the governmentwide high-risk areas of strategic human capital management and information security. The first column lists the management challenges that we and/or SBA’s Inspector General (IG) have identified. The second column discusses what progress, as discussed in its fiscal year 2000 performance report, SBA has made in resolving its challenges. The third column discusses the extent to which SBA’s fiscal year 2002 performance plan includes performance goals and measures to address the challenges that we and the IG identified. The SBA IG told us that in the fiscal year 2000 Performance and Accountability Report, SBA did not update its description of the actions it has taken in response to the challenges to reflect actions taken since the IG’s December 2000 management challenges report.

We found that SBA’s performance report discussed the agency’s plans and progress in responding to most of its challenges, but it did not discuss the agency’s progress in resolving the challenge of streamlining and automating disaster loan processing to improve timeliness. The plan discusses SBA’s efforts to improve timeliness, but not as a result of automated loan processing. Of SBA’s 13 major management challenges, its performance plan had (1) goals, but no measures that were related to 11 of the challenges, and (2) no goals or measures related to two of the challenges. However, SBA discussed strategies for 12 of the 13 challenges.

Appendix II: Comments From the U.S. Small Business Administration