State Small Business Credit Initiative: Opportunities Exist to Improve Program Oversight
Released: 2011-12-07T00:00:00

Letter

Background

The Small Business Jobs Act of 2010 (the act) aims to address the ongoing effects of the 2007-2009 financial crisis on small businesses and stimulate job growth by establishing the SSBCI program, among other things. SSBCI is designed to strengthen state programs that support private financing to small businesses and small manufacturers that, according to Treasury, are not getting the loans or investments they need to expand and to create jobs. The act did not require a specific number of jobs to be created or retained as a result of SSBCI funds.

The act appropriated $1.5 billion to be used by Treasury to provide direct support to states for use in programs designed to increase access to credit for small businesses. Using a formula contained in the act, Treasury calculated the amount of SSBCI funding for which each of the 50 states, as well as the District of Columbia, the Commonwealth of Puerto Rico, the Commonwealth of the Northern Mariana Islands, Guam, American Samoa, and the United States Virgin Islands were eligible to apply. This formula takes into account a state’s job losses in proportion to the aggregate job losses of all states. (See app. III for more information on available funding by location). In addition to states, the act granted permission to municipalities to apply directly for funding under SSBCI in the event that a state either failed to file a Notice of Intent to Apply for its allocation of program funds by November 26, 2010, or, after filing a Notice of Intent, failed to submit an application to Treasury by June 27, 2011. Treasury officials stated that municipalities granted permission to submit an application for program funds were generally subject to the same approval criteria and program requirements as states.

Municipalities were eligible to apply for up to the total amount of their state’s SSBCI allocation, but the final approved amounts were to be apportioned based on their pro rata share by population of all applicants. Figure 1 provides a timeline of major SSBCI milestones.

The act allowed Treasury to provide SSBCI funding for two state program categories: capital access programs (CAP) and other credit support programs (OCSP). A CAP is a loan portfolio insurance program wherein the borrower and lender, such as a small business owner and a bank, contribute to a reserve fund held by the lender. Under the act, approved CAPs are eligible to receive federal contributions to the reserve funds held by each participating financial institution in an amount equal to the total amount of the insurance premiums paid by the borrower and the lender on a loan-by-loan basis. Amounts in the lender’s reserve fund are then available to cover any losses incurred in its portfolio of CAP loans. For an SSBCI loan to be eligible for enrollment in a state’s approved CAP, the borrower must have 500 or fewer employees and the loan amount cannot exceed $5 million.

In addition, the following types of OCSPs are eligible to receive SSBCI funds under the act:

Collateral support programs: These programs supply cash collateral accounts to lenders to enhance the collateral coverage of borrowers. The accounts will cover all or a portion of the collateral shortfall identified by a lending institution. These programs can be designed to target certain regions or industries, such as equipment lending, in which a lender may be willing to fund at 80 percent loan-to-value, but a borrower may not be able to bridge the difference in cash at closing.

Loan participation programs: These programs enable small businesses to obtain medium- to long-term financing, usually in the form of term loans, to help them expand their businesses. States may structure a loan participation program in two ways: (1) purchase transactions, also known as purchase participation, in which the state purchases a portion of a loan originated by a lender and (2) companion loans, also known as co-lending participation or parallel loans, in which a lender originates one loan and the state originates a second (usually subordinate) loan to the same borrower. This program enables the state to act as a lender, in partnership with a financial institution, to provide small business loans at attractive terms.

Direct loan programs: Although Treasury does not consider these programs to be a separate SSBCI program type, it acknowledges that some states may identify programs that they plan to support with SSBCI funds as direct loan programs. The programs that some states label as direct loan programs are viewed by Treasury as co-lending programs categorized as loan participation programs, which have lending structures that are allowable under the statute.

Loan guarantee programs: These programs enable small businesses to obtain term loans or lines of credit to help them grow and expand their businesses by providing a lender with the necessary security, in the form of a partial guarantee, for them to approve a loan or line of credit. In most cases, a state sets aside funds in a dedicated reserve or account to guarantee a specified percentage of each approved loan.

Venture capital programs: These programs provide investment capital to create and grow start-ups and early-stage businesses, often in one of two forms: (1) a state-run venture capital fund (which may include other private investors) that invests directly in businesses or (2) a fund of funds, which is a fund that invests in other venture capital funds that in turn invest in individual businesses. Many factors, particularly resources and available talent, inform a state’s decision on which form to choose. For example, a state may choose to invest in a large venture fund that agrees to reinvest in that state an amount equal to that invested by the state, as opposed to trying to attract that same talent to a smaller fund capitalized with state money.

Qualified loan or swap funding facilities: States may enter into qualifying loan or swap funding transactions under which SSBCI funds are pledged as collateral for private loans or credit lines. The private financing proceeds must, however, be used exclusively for the reserve or other accounts that back the credit support obligations of a borrowing CAP or OCSP. Presumably, fees paid by borrowers and lenders will provide a return to the providers of private capital.

Other OCSPs: States were also able to submit an application to Treasury outlining their plans to support OCSPs that, though not able to be categorized in any of the above OCSP types, feature combinations of aspects of these eligible types.

OCSPs approved to receive SSBCI funds are required to target borrowers with an average size of 500 or fewer employees and to target support towards loans with average principal amounts of $5 million or less. In addition, these programs cannot lend to borrowers with more than 750 employees or make any loans in excess of $20 million.

In applying for funding, applicants had to demonstrate that their CAPs and OCSPs could satisfy SSBCI criteria. For example, applicant states had to demonstrate that all legal actions had been taken at the state level to accept SSBCI funds and implement the state programs. States were also required to demonstrate that the state possessed the operational capacity, skills, and financial and management capacity to meet the objectives set forth in the act. In addition, each applicant was required to demonstrate a “reasonable expectation” that its participating programs, taken together, would generate an amount of private financing and investment at least 10 times its SSBCI funding (that is, a leverage ratio of 10:1) by the program’s end in December 2016. Furthermore, each application had to include a report detailing how the state would use its SSBCI allocation to provide access to capital for small businesses in low- and moderate-income, minority, and other underserved communities, including women- and minority-owned small businesses.

The act requires that each state receive its SSBCI funds in three disbursements of approximately one-third of its approved allocation. Prior to receipt of the second and third disbursements, a state must certify that it has expended, transferred, or obligated 80 percent or more of the previous disbursement to or for the account of one or more approved state programs. Treasury may terminate any portion of a state’s allocation that Treasury has not yet disbursed within 2 years of the date on which its SSBCI Allocation Agreement was signed. Treasury may also terminate, reduce, or withhold a state’s allocation at any time during the term of the Allocation Agreement upon an event of default under the agreement. Following the execution of the Allocation Agreement, states are required All to submit quarterly and annual reports on their use of SSBCI funds. SSBCI Allocation Agreements, the primary tool signed by Treasury and each participating state, which outline how recipients are to comply with program requirements, will expire on March 31, 2017.

The program’s reporting requirements are detailed in section 4.8 of the SSBCI allocation agreement. The obligations of participating states and territories to perform and report on progress will expire as outlined in the terms of the agreement.

Most States Are Participating in SSBCI and Plan to Use the Funds to Support a Variety of Programs

Nearly all of the states eligible for SSBCI funds submitted applications to Treasury. Fifty-four of the 56 states and territories that were eligible to apply for program funds submitted an application prior to the June 27, 2011, deadline, although one state later withdrew its application. In total, states requested more than $1.4 billion in SSBCI funds—95 percent of the program’s appropriation—and only one applied for less than its maximum allocation. Following the application deadline for states, Treasury received five additional applications from municipalities in three states—Alaska, North Dakota, and Wyoming—by the September 27, 2011, deadline requesting a total of $39.5 million in program funds. Figure 2 illustrates the distribution of SSBCI funds applied for by states and territories.

Participating states indicated that they are planning to support various new, existing, and dormant (that is, previously suspended) lending programs with their respective SSBCI allocations. According to our survey results, states are planning to support 153 different lending programs, 69 of which are new programs that were created to be supported by SSBCI funds (see fig. 3). Forty-one states indicated they are planning to support more than one program with their allocation. For example, Alabama plans to support a CAP, four loan participation programs, and a loan guarantee program, and New Jersey plans to support a loan participation program, four loan guarantee programs, five direct loan programs, and a venture capital program.

According to our survey results, states are planning to support CAPs and all types of eligible OCSPs except loan and swap funding facilities (see fig. 4). Venture capital programs are to receive the largest amount of SSBCI funds of any program type. According to Treasury officials, states submitted their respective applications with plans for developing programs in response to unique gaps in local markets or the specific expertise of their staff. Consequently, there is variation in program design across states. For example, Treasury officials stated that Michigan plans to use its funds to support a collateral support program because of difficulties that manufacturing companies in the state were experiencing in obtaining credit. Specifically, Treasury officials noted that as these manufacturers’ real estate and equipment declined in value, they were facing difficulties in obtaining credit due to collateral shortfalls (see app. IV for more information on planned uses of funds by location).

States indicated that they expect SSBCI funds to result in a total of $18.7 billion in new private financing and investment throughout the life of the program. In responding to our survey, officials from 39 of the states that applied for SSBCI funds indicated that they expect to achieve a private leverage ratio between 10:1 and 15:1, and 14 projected a ratio of 15:1 or greater. However, each participating state’s generation of an amount of private financing and investment at least 10 times its SSBCI allocation by December 2016 is not a requirement, and some states indicated that they believe reaching a 10:1 private leverage ratio could prove challenging. For example, officials from one state expressed some concern that the state’s final leverage ratio may ultimately fall short of the estimate included in its approved application because the state was creating a new program and, therefore, did not have prior experience operating a similar program. Treasury officials noted that a state’s mix of programs, as well as the design of each individual program, drives the leverage estimates. For example, Treasury officials stated that private leverage ratios for CAPs tend to be the highest among program types and are evident immediately because the program design is such that the SSBCI subsidy per loan is quite small and is not dependent on subsequent private financing. However, the officials noted that OCSPs tend to have lower leverage ratios initially but may see those grow in later years as program funds are recycled for additional lending over time.

Treasury’s Evolving Processes Have Created Some Delays

Treasury Revised Its Application Guidance and Review Procedures to Clarify Requirements and Thoroughly Reviewed Selected Applications

With the enactment of the Small Business Jobs Act of 2010 on September 27, 2010, Treasury was tasked with quickly starting up an SSBCI program office and developing processes and guidance to implement this new program. After accepting Notices of Intent to Apply from states and territories by the end of November 2010, Treasury issued an initial set of policy guidelines and application materials via its website on December 21, 2010. According to Treasury officials, Treasury received a few applications shortly thereafter and was able to review and approve them and to obtain signed Allocation Agreements with and distribute first installments of funds to two states in January 2011. In response to feedback from states, discussions with other federal agencies, such as the Small Business Administration, and current trends in the small business banking arena, Treasury determined that it needed to revise its guidelines and application paperwork to better articulate what documentation was required for both the application and review processes. As a result, Treasury issued revised guidance materials and Allocation Agreements for applicants in April 2011 as well as a reviewers’ manual for its review staff in May 2011. According to our survey of SSBCI applicants, five states submitted the final version of their application to Treasury before these documents were finalized. Treasury officials told us that although they took steps to help ensure consistent treatment of applicants, Treasury did not revisit previously approved applications once review procedures were finalized. Treasury officials said they were confident that no additional review was required, as those early applications were from states with well-established programs. However, as a result of the revisions to the Allocation Agreement made in April 2011, Treasury asked the two states that had signed the previous versions to sign an amended Allocation Agreement that incorporated the new terms.

Some states reported that they delayed submitting their applications until Treasury’s final application guidance was issued. According to our survey results, 37 states did not submit their final applications for SSBCI funds until June 2011, the month that applications were due. Despite the delay in providing application guidance, applicants generally viewed Treasury officials as helpful throughout the application process—providing answers to most questions immediately and determining answers as soon as possible when not readily available. Treasury officials stated that they also hosted multiple webinars and conference calls to field questions about the application process that were highly attended by states and territories.

In our review of the eight applications reviewed and approved before June 30, 2011, we found that Treasury considered each aspect of the application. Although only one of the applications we reviewed was processed under the revised application and review guidelines, we found that each application was subject to five stages of review: an initial review, a subsequent review by a quality assurance reviewer, review by the application review committee, a legal review, and final approval by the designated Treasury official. Our reviews of the applications and the experiences of the states suggest that applications were scrutinized in terms of their completeness as well as the eligibility of the programs for which states intended to use SSBCI funds. For example, Treasury reviewers noted that in one state’s application, the state proposed several modifications to its existing CAP, thereby bringing it under compliance with SSBCI requirements. Similarly, SSBCI applicants reported that Treasury scrutinized their applications. According to our survey results, 50 of the 54 applicants reported they were required to resubmit at least parts of their applications for further review after their original submissions. For example, one state noted that Treasury wanted significant changes in its application, mainly in the areas of internal controls, mix of programs, and contractor oversight. Another state noted that Treasury determined that the state failed to specify that it was to match the borrower and lender premium between 2 percent and 3.5 percent; Treasury officials asked the state to revise its application to reflect this information and submit an amended application.

Delays in Finalizing Disbursement Procedures Slowed the Disbursement of Funds

As required under the act, Treasury is distributing SSBCI funds to recipients in three installments. As of October 31, 2011, Treasury had provided first installments to 46 states and territories, totaling about $424 million. However, Treasury did not begin processing state requests for their second installment of funds until November 2011. According to Treasury officials, Treasury had previously not acted on these requests because they wanted to ensure that proper procedures were established to ensure all certifications made as part of the request were adequately substantiated. Specifically, they had to resolve how to determine whether 80 percent of a state’s initial disbursement of funds has been expended, transferred, or obligated as required under the act. Treasury finalized its disbursement procedures for second and third installments of SSBCI funds at the beginning of November 2011. According to Treasury officials, as of that date, no state had yet expended 80 percent of its initial disbursement to support loans or investment to small businesses.

While Treasury was working to finalize these procedures, states were potentially delayed in receiving their remaining SSBCI funding. For example, officials from one state that we contacted told us they were ready for their second installment after their first installment was transferred to the accounts of their designated SSBCI lending programs, but they were told by Treasury officials that they would have to wait until the disbursement procedures were finalized. Consequently, the officials told us their state faced additional interest expenses as a result of the delay.

Treasury Is Implementing a Plan to Monitor Recipient Compliance with Program Requirements

Treasury is implementing a multi-step plan to monitor recipient compliance with SSBCI program requirements. These steps include (1) collecting and reviewing quarterly and annual reports, as well as quarterly use of funds certifications, from recipients, (2) evaluating the accuracy of recipient-level reporting on an annual basis by sampling transaction-level data, (3) monitoring recipient requests for second and third installments of SSBCI funds, and (4) contacting recipients on a quarterly basis to inquire as to their adherence with plans outlined in their respective SSBCI applications, as well as monitoring requirements. Treasury has developed a secure, online system for states to report on those data fields included in the Allocation Agreements signed by states, including (1) total amount of principal loaned and of that amount, the portion that is from nonprivate sources; (2) estimated number of jobs created or retained as a result of the loan; and (3) amount of additional private financing occurring after the loan closing. States are to provide these data to Treasury on an annual basis beginning in March 2012. Treasury officials told us they plan to sample states’ transaction-level data to help ensure the accuracy of state reporting. Specifically, the SSBCI compliance manager is to take samples of transaction-level data from all recipients in order to determine whether states are entering these data accurately, including verifying that transactions listed match the underlying loan or investment documents. Treasury officials noted that the system is to automatically flag any loans for which the data entered do not comply with program requirements.

Treasury officials told us they have also assigned three relationship managers to serve as the primary Treasury contacts for the SSBCI program. These managers, who have each been assigned 15 to 20 recipients, are to hold quarterly phone conversations with recipients. During these calls, the managers are to ask a series of generic questions, as well as recipient-specific questions regarding plans the states described in their applications, such as hiring staff and monitoring the use of program funds.

The Treasury Inspector General recently made recommendations to further enhance Treasury’s oversight of SSBCI recipients. In August 2011, the Inspector General issued a report describing the results of its review of SSBCI policy guidance and other key program documents, including allocation agreements.recommendations to improve Treasury’s compliance and oversight framework, including that Treasury’s guidance should clearly define the oversight obligations of recipients and specify minimum standards for determining whether recipients have fulfilled their oversight responsibilities. Treasury concurred with eight of the recommendations and has begun to take action to address them. Treasury disagreed with the Inspector General’s recommendation to make additional provisions for The report made nine states to certify their allocation agreements, stating that states certify that they are implementing their programs in compliance with SSBCI procedures as part of their quarterly reporting to Treasury.

Treasury Is in the Process of Developing Performance Measures for SSBCI

Treasury officials told us that they have not yet established performance measures for the SSBCI program. Although Treasury plans to rely primarily on the department’s overall performance measures in evaluating the SSBCI program, officials noted they are considering several draft performance measures to assess the efficiency of the program. Treasury officials described to us some of the potential measures they are considering, but we are not including them in this report because they have not yet been finalized. Treasury officials told us that they have not finalized the program’s performance measures because they have been focused on starting up the program quickly to meet statutorily required deadlines. Furthermore, officials noted that because SSBCI is a multilayered program that is implemented at the state level and dependent upon private sector entities, Treasury’s ability to influence program outcomes will be limited. Therefore, Treasury officials have been trying to develop measures that focus on the aspects of the program under Treasury’s control. According to Treasury officials they do not have a time frame for fully developing and finalizing SSBCI-specific performance measures.

The potential performance measures described by Treasury do not currently include measures related to the number of jobs created or retained as a result of the SSBCI program. As required in their allocation agreements with Treasury, states are to report information on estimated jobs resulting from SSBCI programs on a per loan or investment basis. According to Treasury officials, gathering this information from the states serves two purposes: (1) it allows Treasury to track the progress of the states against the anticipated benefits articulated for their programs in their SSBCI applications and (2) it provides Treasury with a potential data point that may be useful when measuring overall program performance over time. However, Treasury’s ability to use this information moving forward could be limited, as the jobs data will be based on estimates and not actual jobs. In particular, as part of the SSBCI loan and investment application process, borrowers and investors are required to provide in their application paperwork estimates of the number of jobs to be created and retained as a result of participating in SSBCI programs. States then provide these estimates in their annual reports to Treasury. However, the states are not required to validate these jobs estimates, and they are not required to follow up with borrowers and investors to determine whether the actual number of jobs they were able to create or retain matched their original estimates. According to one lending official we spoke with, validating these estimates would be difficult and lenders could be discouraged from participating in the SSBCI program if they were required to track actual jobs created and retained. Concerned about the burden that reporting on actual jobs created and retained would place on the small businesses receiving SSBCI funds, Treasury officials told us that they elected to capture instead estimated jobs data at the time of the closing of the loan or investment. Treasury officials noted they are currently consulting with officials from the Small Business Administration to learn what methods that agency uses in measuring jobs using estimated data.

The importance of performance measures for gauging the progress of programs and projects is well recognized. Measuring performance allows organizations to track the progress they are making toward their goals and gives managers crucial information on which to base their organizational and management decisions. Leading organizations recognize that performance measures can create powerful incentives to influence organizational and individual behavior. In addition, the Government Performance and Results Act of 1993 (GPRA) incorporates performance measurement as one of its most important features. Under GPRA, executive branch agencies are required to develop annual performance plans that use performance measurement to reinforce the connection between the long-term strategic goals outlined in their strategic plans and the day-to-day activities of their managers and staff. The Office of Management and Budget (OMB) has also directed agencies to define and select meaningful outcome-based performance measures that indicate the intended result of carrying out a program or activity. Additionally, we have previously reported that aligning performance metrics with goals can help to measure progress toward those goals, emphasizing the quality of the services an agency provides or the resulting benefits to users.

We have also previously identified criteria to evaluate an agency’s performance measures. While GPRA focuses on the agency level, performance measures are important management tools for all levels of an agency—such as the bureau, program, project, or activity level—and these criteria are applicable at those levels as well. Among other criteria, we have identified nine key attributes of successful performance measures. These attributes include the following: (1) Linkage. Measure is aligned with division- and agency-wide goals and mission and clearly communicated throughout the organization. (2) Clarity. Measure is clearly stated, and the name and definition are consistent with the methodology used to calculate it. (3) Measurable target. Measure has a numerical goal. (4) Objectivity. Measure is reasonably free from significant bias or manipulation. (5) Reliability. Measure produces the same result under similar conditions. (6) Core program activities. Measures cover the activities that an entity is expected to perform to support the intent of the program. (7) Limited overlap. Measure should provide new information beyond that provided by other measures. (8) Balance. Balance exists when a suite of measures ensures that an organization’s various priorities are covered. (9) Governmentwide priorities. Each measure should cover a priority such as quality, timeliness, and cost of service.

Given the preliminary nature of Treasury’s potential performance measures, assessing whether the measures will reflect the attributes of successful performance measures would be premature. Nevertheless, considering these attributes as it works to finalize SSBCI-specific performance measures could help Treasury to develop robust measures. Until such measures are developed and implemented, Treasury will not be able to determine whether the program is achieving its goals.

Conclusions

In response to SSBCI’s short time frame, Treasury was able to design, implement, and execute an application process for the program in a matter of months. Appropriately, Treasury’s early efforts were focused on establishing the application process and the process for disbursing initial installments of funds to recipients as quickly as possible. Treasury is still in the process of developing performance measures for the SSBCI program. Measuring performance allows organizations to track progress toward their goals and gives managers crucial information on which to base decisions. At the program level, agencies can create a set of performance measures that addresses important dimensions of program performance and balances competing priorities. Performance measures that successfully address important and varied aspects of program performance are key elements of an orientation toward results. Effective performance measures can provide a balanced perspective on the intended performance of a program’s multiple priorities. While Treasury is considering potential draft performance measures, it has not fully developed or finalized a set of measures for the SSBCI program. Until such measures are developed and implemented, Treasury will not be in a position to determine whether the SSBCI program is effective in achieving its goals.

Recommendation for Executive Action

We are making one recommendation to Treasury to improve its implementation and oversight of the SSBCI program as follows:

To help ensure that the performance measures for the SSBCI program are as robust and meaningful as possible, we recommend that the Secretary of the Treasury direct the SSBCI Program Manager to consider key attributes of successful performance measures as the program’s measures are developed and finalized.

Agency Comments and Our Evaluation

We provided a draft of this report to Treasury for review and comment. Treasury provided written comments that we have reprinted in appendix V. Treasury also provided technical comments, which we have incorporated, as appropriate.

In their written comments, Treasury agreed with our recommendation. Treasury noted that it will consider the key attributes of successful performance measures as it works to finalize measures for the SSBCI program.

We are sending copies of this report to the appropriate congressional committees, the Secretary of the Treasury, and other interested parties. The report is also available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at clowersa@gao.gov or (202) 512-8678. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

To determine which states applied for and received State Small Business Credit Initiative (SSBCI) funds and the planned uses of the funds, we developed a Web-based questionnaire to collect information from the 54 states and territories that filed a Notice of Intent to Apply for SSBCI funds with the Department of the Treasury (Treasury) by the November 26, 2010 deadline. The questionnaire included questions on the timing of applications for SSBCI funds, the receipt of funds to date, the intended uses of funds, and the potential impacts of program funds. See appendix II for a copy of the questionnaire.

To minimize errors arising from differences in how questions might be interpreted and to reduce variability in responses that should be qualitatively the same, we conducted pretests with officials in three states, both in person and over the telephone. To help ensure that we obtained a variety of perspectives on our questionnaire, we selected officials from states planning to support various types of programs with SSBCI funds. Based on feedback from these pretests, we revised the questionnaire in order to improve response quality. For instance, in response to one state official’s comment that it would be difficult for respondents to answer with confidence how many capital access programs (CAP) and other credit support programs (OCSP) have recently been in operation across all municipalities in a state, we removed the historical and specific program budget questions and clarified our focus on the planned uses of SSBCI funds. We conducted two additional pretests with other state officials to ensure that the updated questionnaire was understandable.

After completing the pretests, we administered the survey. On August 4, 2011, we began sending e-mail announcements of the questionnaire to the state and territory officials that had been identified as points of contact in a list provided to us by Treasury, notifying them that our online questionnaire would be activated in approximately 1 week. On August 15, 2011, we sent a second e-mail message to officials in which we informed them that the questionnaire was available online and provided them with unique passwords and usernames. On August 26, 2011, we began making telephone calls to officials and sent them follow-up e-mail messages, as necessary, to ensure their participation as well as to clarify and gain a contextual understanding of their responses. By September 14, 2011, we had received completed questionnaires from 54 states and territories, for a 100 percent response rate.

We used standard descriptive statistics to analyze responses to the questionnaire. Because this was not a sample survey, there are no sampling errors. To minimize other types of errors, commonly referred to as nonsampling errors, and to enhance data quality, we employed recognized survey design practices in the development of the questionnaire and in the collection, processing, and analysis of the survey data. For instance, as previously mentioned, we pretested the questionnaire with state officials to minimize errors arising from differences in how questions might be interpreted and to reduce variability in responses that should be qualitatively the same. In addition, during survey development, we reviewed the survey to ensure the ordering of survey sections was appropriate and that the questions within each section were clearly stated and easy to comprehend. We also received feedback from survey experts who we asked to review the survey instrument. To reduce nonresponse, another source of nonsampling error, we sent out e-mail reminder messages to encourage officials to complete the survey. In reviewing the survey data, we performed automated checks to identify inappropriate answers. We further reviewed the data for missing or ambiguous responses and followed up with respondents when necessary to clarify their responses. On the basis of our application of recognized survey design practices and follow-up procedures, we determined that the data were of sufficient quality for our purposes.

In addition to the survey, we conducted interviews with Treasury officials, as well as selected state officials and financial institutions within those states either via teleconference or site visits to collect documentation that informed our understanding of states’ planned uses of SSBCI funds. We limited our selection of states to interview to those states whose SSBCI applications had been reviewed, approved, and for which the applicant had signed an allocation agreement by June 30, 2011: California, Hawaii, Indiana, Kansas, Maryland, Missouri, North Carolina, and Vermont.

To evaluate Treasury’s implementation of the SSBCI program, we compared and contrasted Treasury’s SSBCI procedures and planned control activities with GAO’s internal control standards, including Internal Control in the Federal Government. We interviewed Treasury officials about the types of training it provided its staff to help ensure compliance with its procedures. We also utilized data obtained through our questionnaire to identify the dates on which states submitted their SSBCI applications and whether Treasury required resubmission. Additionally, we reviewed a nonprobability sample of SSBCI applications consisting of all eight states that had signed an SSBCI allocation agreement by June 30, 2011, to determine whether all aspects of these states’ applications were considered. We assessed whether Treasury followed its procedures and appropriately documented its decisions by analyzing the documentation of the application reviews. Because we used a nongeneralizable sample to select the applications to review, our findings cannot be used to make inferences about SSBCI applications of states that signed allocation agreements after June 30, 2011. However, we determined that the sample would be useful in providing illustrative examples on procedures and documentation practices applied by Treasury. Furthermore, we conducted interviews with Treasury officials about the type of testing the agency plans to perform of its controls to ensure compliance with SSBCI procedures, lessons learned about the review process, how they addressed problems, and their plans to follow up with states to ensure that SSBCI funds are used for the intended purposes outlined in approved applications for program funds.

To review Treasury’s efforts to measure whether the SSBCI program achieves its goals of increasing small business investment and creating jobs, we discussed with Treasury their proposed performance metrics for the SSBCI program. We also interviewed Treasury officials, as well as officials from the eight states that had signed a SSBCI allocation agreement with Treasury by June 30, 2011, to collect documentation that was used to inform our understanding of SSBCI program performance and Treasury’s metrics.

We conducted this performance audit from February 2011 to December 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Copy of GAO Questionnaire

Appendix III: SSBCI Funds Applied for, Approved, and Disbursed, by Location, as of October 31, 2011

Table 1 below contains the amounts of SSBCI funds that have been applied for, approved, and disbursed as of October 31, 2011. This information was provided by state and territory officials who responded to a GAO survey between August 15 and September 14, 2011 and by the U.S. Treasury on October 31, 2011.

Appendix IV: Planned Uses of SSBCI Funds, by Location

Table 2 below contains information on states and territories’ plans for the distribution of SSBCI funds among eligible program types, provided by officials between August 15 and September 14, 2011.

Appendix V: Comments from the Department of the Treasury

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Paul Schmidt, Assistant Director; Pamela Davidson; Jill Lacey; Marc Molino; Patricia Moye; Deena Richart; Christine San; Jennifer Schwartz; and Chad Williams made key contributions to this report.