State Small Business Credit Initiative: Opportunities Exist to Enhance Performance Measurement and Evaluation
Released: 2013-12-18T00:00:00

Letter

Background

SSBCI provides direct support to participants for use in programs designed to increase access to credit for small businesses. Using a formula contained in the Small Business Jobs Act of 2010, Treasury calculated the amount of SSBCI funding for which each of the 50 states, as well as the District of Columbia, the Commonwealth of Puerto Rico, the Commonwealth of the Northern Mariana Islands, Guam, American Samoa, and the United States Virgin Islands, were eligible to apply. This formula takes into account a state’s job losses in proportion to the aggregate job losses of all states. In addition to states, the act granted permission to municipalities to apply directly for funding under SSBCI in the event that their state did not apply for funding. Municipalities in Alaska, North Dakota, and Wyoming used this option. Participants are expected to use their SSBCI funds to leverage private financing and investment that is at least 10 times the amount of their SSBCI allocation (a leverage ratio of 10-to-1) by December 31, 2016. Forty-seven states; the District of Columbia; the Commonwealth of Puerto Rico; the Commonwealth of the Northern Mariana Islands; Guam; American Samoa; the United States Virgin Islands; Anchorage, Alaska; Carrington, North Dakota; Mandan, North Dakota; and Laramie, Wyoming, currently participate in the program.

The act allowed Treasury to provide SSBCI funding for two program categories: capital access programs (CAP) and other credit support programs (OCSP). For both CAPs and OCSPs, the act requires that lenders have a meaningful amount of their own capital at risk. Loan terms, such as interest and collateral, are typically negotiated between the lender and the borrower, although in some cases loan terms are subject to SSBCI participant approval and, in many cases, the SSBCI participant and lender will discuss and negotiate loan terms and guarantee options prior to reaching agreement to approve the loan and issue a guarantee.

A CAP is a loan portfolio insurance program wherein the borrower and lender, such as a small business owner and a bank, contribute to a reserve fund held by the lender. Under a CAP, when a participating lender originates a loan, the lender and borrower combine to contribute an amount equal to a percentage of the loan to a loan reserve fund, which is held by the lender. Under SSBCI, the contribution must be from 2 percent to 7 percent of the amount borrowed. Typically, the contribution ranges from 3 percent to 4 percent. The SSBCI participant then matches the combined contribution and sends that amount to the lender, which deposits the funds into the lender-held reserve fund. Under SSBCI, approved CAPs are eligible to receive federal contributions to the reserve funds held by each participating financial institution in an amount equal to the total amount of the contributions paid by the borrower and the lender on a loan-by-loan basis.

In addition, the following OCSPs are examples of types of programs eligible to receive funding under the act:

Collateral support programs. A collateral support program is designed to enable financing that might otherwise be unavailable due to a collateral shortfall. It provides pledged cash collateral to lenders to enhance the collateral coverage of individual loans. The SSBCI participant and lender negotiate the amount of cash collateral to be pledged by the participant.

Loan participation programs. SSBCI participants may structure a loan participation program in two ways: (1) through purchase transactions, also known as purchase participations, in which the SSBCI participant purchases a portion of a loan originated by a lender, or (2) by participating in a loan as a co-lender, where a lender originates a senior loan and the SSBCI participant originates a second loan to the same borrower that is usually subordinate to the lender’s senior loan should a default occur. SSBCI loan participation programs encourage lending to small businesses because the lender is able to reduce its potential loss by sharing its exposure to loan losses with the SSBCI participant.

Loan guarantee programs. These programs enable small businesses to obtain a term loan or line of credit by providing the lender with the necessary security in the form of a partial guarantee. In most cases, the SSBCI participant sets aside funds in a dedicated reserve or account to collateralize the guarantee of a specified percentage of each approved loan. The guarantee percentage is determined by the participants and lenders but, under SSBCI, may not exceed 80 percent of loan losses.

Venture capital programs. These programs provide investment capital to create and grow start-ups and early-stage businesses, often in one of two forms: (1) a SSBCI participant-run venture capital fund (which may include other private investors) that invests directly in businesses, or (2) a fund of funds, which is a fund that invests in other venture capital funds that in turn invest in individual businesses.

Direct loan programs. Although Treasury does not consider these programs to be a separate SSBCI program type, it acknowledges that some participants may identify programs that they plan to support with SSBCI funds as direct loan programs. The programs that some SSBCI participants label as direct loan programs are viewed by Treasury as co-lending programs categorized as loan participation programs, which have lending structures that are allowable under the statute.

OCSPs approved to receive SSBCI funds are required to target small businesses with an average size of 500 or fewer employees and to target support towards loans with an average principal amount of $5 million or less. In addition, these programs cannot lend to borrowers with more than 750 employees or make any loans in excess of $20 million.

After their applications were approved, SSBCI participants entered into Allocation Agreements with Treasury before they received their funds. SSBCI Allocation Agreements are signed by Treasury and participants, and they outline how participants are to comply with program requirements. The act requires that each participant receive its SSBCI funds in three disbursements or tranches of approximately one-third of its approved allocation. As part of its request to receive the second and third disbursements, a participant must certify that it has expended, transferred, or obligated 80 percent or more of the previous disbursement. When participants request subsequent disbursements, Treasury reviews the request and certification for completeness. Treasury may also review a sample of the participant’s transactions for compliance with SSBCI requirements. All SSBCI Allocation Agreements will expire on March 31, 2017. Treasury may terminate any portion of a participant’s allocation that Treasury has not yet transferred to the participant within 2 years of the date on which its SSBCI Allocation Agreement was signed. Treasury may also reduce or terminate a participant’s allocation at any time during the term of the Allocation Agreement upon an event of default under the agreement. Under the act, participants are required to submit quarterly and annual reports on their use of SSBCI funds.

Changes to participants’ Allocation Agreements generally must be approved by Treasury through a program modification. For example, participants must submit a program modification to Treasury for approval in order to use SSBCI funds for a new program that was not originally approved by Treasury, to materially change the scope or purpose of an approved program, or to reapportion and transfer allocated SSBCI funds among approved programs when the cumulative amounts transferred exceed 10 percent of the participant’s total SSBCI allocation. Treasury has procedures for participants to apply for modifications and for its processing of modification approval requests. In general, the procedures require participants to submit, among other things, justification of the need for the modification and the impact of the change on program performance, including the 10-to-1 private leverage expectation. As of June 30, 2013, Treasury had approved 32 modifications of SSBCI participants’ programs.

The act requires Treasury’s OIG to conduct audits of participants’ use of their SSBCI allocations. As of November 8, 2013, Treasury’s OIG has conducted 12 audits of SSBCI participants’ use of SSBCI funds.

SSBCI Participants Have Used Funds at Varying Rates and a Number of Challenges Have Been Addressed

Between June 30, 2012, and June 30, 2013, SSBCI participants nearly quadrupled their overall use of SSBCI funds, but individual participants continue to use their funds at varying rates and for different types of programs. SSBCI participants faced challenges using their SSBCI funds in the early stages of the program. For example, because Treasury initially implemented the program without issuing final policies and procedures, some SSBCI participants experienced delays in receiving subsequent disbursements of funds. While initial challenges have mostly been addressed, participants continue to face ongoing challenges with using SSBCI funds, including reluctance from large banks to participate in the program.

SSBCI Fund Usage Has Increased but Continues to Vary Widely across Participants

As of June 30, 2013, SSBCI participants had used a total of about $549 million in SSBCI funds, which represents about 38 percent of the $1.5 billion allocated for the program. As shown in figure 1, usage of SSBCI funds by participants has nearly quadrupled since June 30, 2012, when participants had used about $150 million.

In 2012, we found that participants had used their SSBCI funds at different rates. As of June 30, 2013, SSBCI participants were continuing to use their SSBCI allocations at varying rates, as shown in figure 2. Fifty- five of the 57 participants had used some portion of their allocation as of June 30, 2013. Idaho had used the highest percentage of its allocated funds, approximately 94 percent of the $13 million it was allocated. Two territories—American Samoa and Northern Mariana Islands—had not used any of their allocation, which, according to Treasury officials, is because both territories have limited small business credit and investment markets. As of June 30, 2013, 18 participants had used more than 50 percent of their allocations, 14 participants had used more than 25 to 50 percent of their allocation, 19 participants had used more than 10 to 25 percent of their allocation, and 6 participants had used 10 percent or less of their allocation. As previously discussed, participants can request subsequent SSBCI disbursements when they have used at least 80 percent of their current tranche of SSBCI funds. As of June 30, 2013, Treasury had disbursed almost $811 million in SSBCI funding to participants. Eight participants had received their third and final disbursement of funds, 19 participants had received their second disbursement, and 30 participants were still working to use their first disbursement of SSBCI funding.

SSBCI participants have used their allocations to fund a variety of different program types. SSBCI allows participants to use their allocations to support program types that best meet their individual circumstances and demands. This flexibility has resulted in participants implementing numerous types of programs to use their SSBCI allocations (see fig. 3). Similar to what we reported in 2012, loan participation and venture capital programs had used the most funds, as of June 30, 2013.between June 30, 2012, and June 30, 2013, venture capital programs surpassed loan participation programs to become the program type with the highest percentage of funds used. As of June 30, 2013, venture capital programs accounted for approximately 37 percent of SSBCI funds used. Loan participation programs accounted for the next highest percentage of funds used (36 percent), followed by loan guarantee programs and collateral support programs (each about 13 percent). The remaining program type—CAP—represented only approximately 1 percent of the funds used. Figure 3 also shows that venture capital programs and loan participation programs were the most common program types implemented, representing more than half of all SSBCI programs. Three of the six participants with whom we spoke during this review said that demand for their venture capital programs was higher than their lending programs. Further, figure 3 demonstrates that while CAPs accounted for 17 percent of the programs implemented, their fund usage represented only 1 percent of the funds used by SSBCI participants. Multiple participants we interviewed during our 2012 and 2013 reviews noted that the demand for their CAPs was low, which resulted in them reallocating funds to SSBCI programs with higher demand.

Treasury and Stakeholders Undertake Various Outreach Activities to Promote Use of SSBCI

Treasury, SSBCI participants, federal banking regulators, and trade associations all conduct numerous outreach activities to promote the use of SSBCI. These activities include presenting information on SSBCI at conferences and lender roundtables, conducting face-to-face meetings with lenders and investors, and distributing information through brochures and websites. Challenges with promoting SSBCI have been identified by Treasury and SSBCI stakeholders, including the variation among SSBCI participants’ programs and the amount of time and resources involved in conducting outreach. To address these challenges, Treasury is working with SSBCI participants to identify best practices for marketing SSBCI.

Treasury Collaborates with Stakeholders to Promote Use of SSBCI

Treasury conducts numerous outreach activities to promote the use of SSBCI, many of which are conducted in conjunction with program stakeholders, including federal banking regulators, trade associations, and SSBCI participants. Outreach conducted in collaboration with external stakeholders is consistent with federal internal control standards, which state that adequate means of communicating with, and obtaining information from, external stakeholders that may have a significant impact on the agency achieving its goals should exist. To help promote use of the participants’ SSBCI lending programs, Treasury officials stated that they work with federal banking regulators on a regular basis. For example, Treasury officials have participated in lender roundtables sponsored by federal banking regulators and written articles for their newsletters that describe SSBCI and, in some cases, contain case study examples of some SSBCI participants’ programs. Treasury officials also said that they have developed a national awareness campaign in which they work with relevant trade associations—representing both the lending and venture capital communities—to promote use of SSBCI. Treasury officials explained that this outreach consists of presenting information on SSBCI at trade associations’ conferences and writing articles for their member newsletters. Treasury officials also participate in SSBCI participants’ outreach activities. For example, representatives from Montana’s SSBCI program told us that the Treasury SSBCI program director participated in meetings with CDFIs that they held when the program was first started. Finally, Treasury maintains a webpage dedicated to SSBCI. The site contains program policy and procedure documents, guidance materials on specific program types, and links to each participant’s SSBCI program website, among other items.

In 2012, Treasury piloted a program in which it hired consultants to assist three SSBCI participants—Mississippi, Ohio, and South Carolina—with program outreach. The goal of the pilot was to increase financial institutions’ awareness and use of the states’ SSBCI lending programs. In each state, consultants developed action plans detailing the assistance they would provide to the state to help promote the use of SSBCI by lenders. The consultants also conducted in-person meetings or telephone calls directly to lending institutions to discuss SSBCI, designed program marketing materials, and reviewed the states’ websites, among other things. In addition, in South Carolina, the consultant assisted in the development of a webinar presentation, which was offered to all South Carolina banks and was later posted to the state’s website. Treasury officials said that while the consultants’ work could provide a useful outside perspective and help to identify improvements needed in participants’ programs, the process of finding and contracting with an appropriate consultant is time consuming and its effectiveness depends on whether participants are able to take advantage of the consultants’ recommendations. Treasury officials explained that they would like to continue to provide consultant assistance to participants and have requested additional funds for their 2014 budget to do so. They explained that, with congressional approval for additional funding, they would expand the amount of time the consultants work with the states. For example, the outreach consultant pilot program provided for 100 hours of consultant assistance to each state, and officials said they would like to expand this type of assistance to selected participants for 6 to12 months.

Treasury officials explained that it is difficult to quantify the success of their outreach efforts and that they mostly rely on the feedback they receive from SSBCI participants to gauge the success of their outreach efforts. Treasury officials also explained that they conduct satisfaction surveys after each conference to ask what attendees thought of the conference. Our review of Treasury’s analysis of the conference survey results showed that the majority of those that responded to the surveys found the conferences very good or excellent. Treasury officials said that they also track some indicators in addition to conference survey results. For example, they track the number of attendees at the SSBCI conferences, the number of participants that join webinars or conference calls, and, when feasible, the number of hits received by articles posted on the SSBCI webpage.

Treasury plans to continue to provide assistance to SSBCI participants on program outreach in order to increase the effectiveness and rate of deployment of SSBCI funds. Treasury’s 2014 SSBCI Congressional Justification for its budget notes that Treasury will expand individualized technical assistance to participants by training and supporting SSBCI participants on outreach to financial institutions. Treasury officials explained that they plan to do this in several ways, including conducting state-specific outreach to state banking associations and state banking regulators. They also plan to conduct roundtables consisting of local bankers, some of whom have used SSBCI and can share their experiences with bankers who have not used the program. As described previously, they would also like to provide longer-term consultant assistance to states. Finally, Treasury officials would like to develop a “tool box” of SSBCI marketing materials, such as templates for websites, presentations, work plans, and forums. The 2014 Congressional Justification also noted that Treasury is planning to add additional relationship managers to support state outreach to the lending industry. SSBCI relationship managers are responsible for working with an assigned group of participants to help them successfully allocate funds to lenders and subsequently borrowers. Treasury officials explained that SSBCI currently has three relationship managers, each of whom handles a portfolio of between 15 and 20 participants, which officials view as a heavy workload. Treasury officials noted that all outreach efforts discussed in Treasury’s 2014 Congressional Justification, including its ability to expand assistance to participants on program outreach and to hire additional relationship managers, are contingent upon congressional approval of Treasury’s proposed budget for the agency.

SSBCI Stakeholders Also Conduct Outreach Activities

In addition to Treasury’s outreach efforts, SSBCI participants, federal banking regulators, and a number of trade associations also independently conduct outreach to promote the use of the program. Treasury’s program guidance states that one of the roles of SSBCI participants is to conduct program outreach. The guidance varies depending on the type of program. For example, guidance on venture capital programs is general and states only that SSBCI participants are responsible for marketing the program in their region or community.

Guidance on loan guarantee programs is more explicit and instructs that participants are responsible for conducting outreach to inform lenders and trade associations of the program, including activities such as distributing press releases and speaking at industry or small business conferences.

All of the SSBCI participants with whom we spoke conduct outreach on an ongoing basis to promote use of their programs. Many of the outreach activities they conduct are similar to those used by Treasury.

Collaboration with other organizations. Eight of the nine SSBCI participants with whom we spoke told us that they collaborate with other organizations to help them promote SSBCI. Many work with state banking associations, local economic development groups, small business development centers, and local or regional Chambers of Commerce to reach potentially interested parties. For example, officials from Virginia’s SSBCI program explained that they collaborate with the Virginia Bankers Association, Virginia Association of Community Banks, small business development centers, regional Chambers of Commerce, and Virginia Economic Development Partnership by educating these groups about SSBCI and soliciting their assistance in promoting the program to their members.

Presentations on SSBCI. SSBCI participants also told us that they hold lender roundtables; present information on SSBCI to groups of potentially interested parties, including CDFIs, economic developers, and manufacturing associations; and hold conference calls and webinars for potential program users.

Face-to-face meetings. Five of the nine participants with whom we spoke discussed having face-to-face meetings with lenders or economic developers to promote use of the program. In fact, some participants stated that they found this type of outreach, although time consuming, to be the most effective in encouraging use of the program.

Distribution of information. SSBCI participants also distribute marketing materials, such as program brochures and electronic newsfeeds; issue press releases; publish print advertisements and testimonials on the experiences of program users; send emails with program information; and use social media to promote SSBCI.

Website. Each SSBCI participant also maintains a website where SSBCI program information can be found. Our review of all SSBCI participants’ websites found that the majority included detailed descriptions of their SSBCI programs, links to related materials, information on how to apply for the program, and contacts for additional information. In addition, some websites included other useful promotional information, such as a list of participating financial institutions and testimonials on the program’s benefits.

In addition to Treasury and SSBCI participants, federal banking regulators also conduct outreach activities to promote the use of SSBCI. Outreach conducted by the federal banking regulators has been done on both a national and regional basis. However, regulators’ regional offices focus more specifically on states in their regions and tend to be in a better position to focus on the differences between SSBCI participants’ programs. Specific outreach activities they have conducted include participating in or sponsoring lender roundtables, conferences, and forums in which SSBCI is discussed. These events are often held in conjunction with the promotion of other small business credit programs and include representatives from each of the federal banking regulators, as well as state economic development agencies. For example, an official from OCC’s Southern District office stated that her office held a series of outreach events throughout the district. Each event included a regulatory panel of officials from OCC, FDIC, and the Federal Reserve Bank of Atlanta in which they discussed SSBCI with lenders and CDFIs. In addition, some federal banking regulators have used their newsletters to provide information about SSBCI. For example, regulators have written articles about how states are using SSBCI to increase the amount of credit available for small businesses, and they have also published articles that Treasury has written on SSBCI. Further, as discussed previously, both FDIC and OCC published a list of Frequently Asked Questions to address their regulated entities’ concerns regarding the regulatory implications of participating in SSBCI.

Finally, trade associations for lenders, CDFIs, and venture capitalists have also helped to promote the use of SSBCI to their members. Similar to the types of outreach activities conducted by Treasury, SSBCI participants, and federal banking regulators, trade associations have provided information on SSBCI at their conferences and, as noted earlier, included articles in member newsletters and magazines. In addition, some trade associations, including the American Bankers Association, Opportunity Finance Network, and Council of Development Finance Agencies, hosted conference calls and webinars for their members to promote the use of SSBCI. Representatives from the Opportunity Finance Network—a national network of CDFIs—explained that because some CDFIs were not familiar with SSBCI and some agencies that implement SSBCI at the state level were not familiar with the function of CDFIs, they used conference calls and webinars to connect their members with the appropriate SSBCI stakeholders.

Treasury and Stakeholders Identified Challenges to Promoting SSBCI, as Well as Promising Practices for Conducting Outreach

Treasury and other stakeholders have identified various challenges with promoting SSBCI, including variation among SSBCI participants’ programs, limited time and resources, and limited experience with marketing small business programs.

Variation among SSBCI participants’ programs. Treasury, Federal Reserve, and OCC officials, as well as representatives from the National Association of Federal Credit Unions (NAFCU), all said that the variation among SSBCI participants’ programs creates marketing challenges because different program types have different users. For example, if a SSBCI participant operates only a venture capital program, lenders, such as credit unions in that state or territory, would not be eligible to participate in SSBCI. Treasury officials added that the variation makes it hard for them to build awareness of SSBCI at a national level, and representatives from NAFCU stated that the variation in participants’ programs made it difficult to educate their member credit unions about how they could promote SSBCI to their customers.

Time and resources. Five of the nine SSBCI participants with whom we spoke stated that reaching out to and educating interested parties on SSBCI has been a challenge because of the time commitment and a lack of resources. For example, officials from the municipality of Carrington, North Dakota—whose SSBCI program consists of a consortium of 36 municipalities—explained that outreach is challenging because of the amount of time involved in developing relationships, setting up meetings, and educating lenders and CDFIs about their program. As noted earlier, some participants who implemented new programs did not have established networks of lenders and said that establishing such networks was time consuming. Further, the Carrington, North Dakota, officials noted that small community banks and rural economic development groups can have high turnover rates, which requires them to conduct continual outreach to re-educate staff on the program. Officials from both Carrington and Montana also added that the large geographic footprint covered by their SSBCI programs makes it difficult to reach all lenders in the communities. Montana officials added that this is why they collaborated with other organizations in the state, such as CDFIs, to help promote SSBCI.

SSBCI participants’ experience. SSBCI participants’ experience with marketing small business programs may also vary, resulting in challenges in promoting use of the program. At its October 2012 national SSBCI conference, Treasury identified participants’ lack of experience in program marketing as a challenge for participants’ deploying funds in SSBCI lending programs.

Other challenges that participants mentioned with conducting outreach for SSBCI include that it can be difficult to distinguish SSBCI from other federal and state programs that assist small businesses and, as noted earlier, that some lenders have concerns about being subject to additional regulatory scrutiny for using SSBCI programs to underwrite loans.

Treasury has stated that promoting best practices in marketing is one of SSBCI’s priorities. To achieve this goal and to help address some of the challenges mentioned above, Treasury developed working groups consisting of officials from participating states, territories, and municipalities for the different SSBCI program types. These groups have been working to identify best practices for how to operate each program type, including best practices on marketing the program. As of September 2013, working groups for loan guarantee programs, loan participation programs, and collateral support programs had issued best practice guidance for these program types. Best practices identified in the guidance for all three program types include using consistent marketing; developing and maintaining websites that are current, easy to find, and contain all pertinent information; identifying and reaching out to the small group of key small business lenders; publishing testimonials from lenders that have successfully used SSBCI; and asking the state banking regulators for the opportunity to brief the state examiners so that the examiners can inform lenders about SSBCI. The guidance also includes additional best practices unique to each program type. Treasury officials said that they are working with the group for venture capital programs as well to develop similar best practice guidance for SSBCI participants who operate that type of program. The federal banking regulators also offered some examples of best practices they have identified to promote their programs. One of these was using technology and mobile resources to conduct outreach, such as using webinars to reach a larger audience. OCC officials stated that they have found that collaborating with state banking associations to help design the outreach efforts is effective, as is having lenders that participate in the program share their experiences with other lenders. Similarly, FDIC officials noted that it is important to leverage relationships with the appropriate networks of organizations and to conduct follow-up on activities in which relationships are established.

Prior to the development of the loan participation, collateral support, and loan guarantee programs best practice guidance, Treasury had not provided SSBCI participants with any specific written guidance on program outreach. Instead, Treasury officials told us they mostly relied on peer-to-peer learning among SSBCI participants. Treasury officials explained that its SSBCI conferences provide a good forum for participants to exchange ideas on how to market their programs. In addition, each of Treasury’s three SSBCI conferences has included a session on different aspects of conducting program outreach. The March 2012 SSBCI conference included a session on how to market SSBCI to underserved communities, the October 2012 conference session focused on how to market SSBCI to lenders, and the June 2013 conference session was on how to market SSBCI with partner organizations.

Treasury’s Efforts to Measure and Evaluate Performance Could Be Enhanced

Treasury uses performance measures to manage SSBCI in a number of ways, such as to allocate program resources and to identify participants that might need additional assistance. However, Treasury has not developed targets for all of its performance measures. Federal internal control standards and leading performance management practices identified in prior GAO work state that, where appropriate, management should set performance targets and measure results against these targets, and this practice can help management to identify performance gaps. In addition, Treasury has started to determine how it will evaluate the performance of SSBCI at the end of the program in 2017. Our guidance on designing program evaluations suggests that gathering input from external stakeholders during the design of an evaluation can help better ensure that the evaluation is useful for decision makers and other interested parties.

Treasury Uses Performance Measures to Manage SSBCI but Has Not Set Targets for All Measures

In 2012 we reported that Treasury had established performance measures for both the timeliness of program administration and the monitoring of program performance, which are listed in table 1.

Treasury officials explained that they have used these performance measures in a variety of ways to manage SSBCI. First, Treasury officials stated that they use the measure on the amount of SSBCI funds participants have used to adjust staff allocations, which has helped them to improve efficiency in program operations. For example, by tracking quarterly report data on participants’ use of SSBCI funds, officials realized that a large number of participants would be submitting requests for subsequent disbursements of funds. To manage this increase in disbursements, Treasury increased the number of compliance team staff from two to four in order to better handle the processing of the requests.

Second, Treasury officials also stated that they use information on participants’ fund usage to identify participants that have been slow to request subsequent disbursements of funds, and that they contact these participants to see if they need additional technical assistance from Treasury. Treasury officials explained that the relationship managers monitor participants’ fund usage each quarter and develop action plans for participants with the lowest fund usage to help them address challenges they are facing in using SSBCI funds. Third, Treasury officials told us that they use performance measures on the timeliness of program administration to help manage staff performance. They explained that they compare the amount of time it takes Treasury staff to respond to program modifications and disbursement requests to the goals they have set. They maintain a spreadsheet to track the status of pending modifications and disbursements and distribute it during weekly staff meetings for staff to see if they are on track to meet the performance goals. Officials added that they have started to use these measures in staff performance reviews to hold staff accountable.

As shown in table 1, Treasury has created targets for its measures related to the timeliness of program administration but not for its measures used to monitor the performance of SSBCI. In our 2012 review of SSBCI, as well as in this review, Treasury officials said that they have not developed targets for their measures that monitor program performance and that SSBCI’s performance cannot be summarized using a single number or performance indicator because SSBCI now consists of 150 different programs. Further, they explained that specific numeric indicators, such as the number of loans resulting from participants’ programs, may or may not indicate the performance of SSBCI. They added that outcomes can also be highly dependent on factors outside of the program’s control, such as the demand for credit in a given locality and the quality of the small business borrowers’ requests for such funds. In addition, because many participants did not receive their first SSBCI allocation until late 2011, they did not submit annual reports—which contain data on SSBCI funds used to support small business loans or investments—for that year. As a result, Treasury had little data to assess program performance and set baselines for the targets for program performance measures. In early 2013, Treasury received participants’ 2012 annual reports, which represent the first complete set of annual reports Treasury has received since the inception of the program.

Federal internal control standards state that management should review performance reports and measure results against targets. Leading practices in federal agency performance management previously identified by GAO also state that, where appropriate, performance measures should have quantifiable, numerical targets, and that agencies can use baselines to set realistic but challenging targets. Numerical targets facilitate future assessments of whether overall goals and objectives were achieved because comparisons can be easily made between projected performance and actual results. In addition, numerical targets can also create motivation for improving program performance. Further, Treasury has established targets for performance measures for other programs, including its CDFI Fund programs—programs similar to SSBCI that are administered by locally based organizations and help to promote access to capital and local economic growth. Our previous work has recognized the challenges and provided strategies for agencies in measuring performance and setting performance targets when factors outside of an agency’s control impact the results of a program. For example, one strategy is to set goals at the lowest, most disaggregated level for distinct target populations for which the agency has different expectations. In the case of SSBCI, this could mean setting different performance targets for different program types. Despite these challenges, developing targets for selected SSBCI measures that monitor program performance could help Treasury identify performance gaps and highlight program areas that need additional attention.

Treasury Recently Released SSBCI Performance Information

Treasury officials stated that they share the performance information they collect on SSBCI with Treasury management, other agencies, and SSBCI participants. For example, officials told us that they share SSBCI performance information regularly with SSBCI program management, including the director and deputy director, and with Treasury departmental management to provide information on the progress of the program. In addition, Treasury shares SSBCI performance information with other agencies working on the Entrepreneurship and Small Business Cross- Agency Priority goal. This Cross-Agency Priority goal seeks to increase federal services, including promoting access to capital, to entrepreneurs and small businesses. Within the Cross-Agency Priority goal, a working group consisting of Treasury, U.S. Export-Import Bank (Ex-Im), Small Business Administration (SBA), and U.S. Department of Agriculture (USDA) was developed specifically for the goal’s objective related to increasing access to capital for entrepreneurs and small businesses. SSBCI is one of the programs that will help to contribute to this objective. The fiscal year 2013 third-quarter status update for this Cross-Agency Priority goal included information on the amount of new private sector lending or investment that SSBCI supported and the percentage of participants who had used SSBCI funds to support loans or investments.

Treasury has also shared some performance information with SSBCI participants and recently released summary performance information from participants’ 2012 annual reports to the public. Officials from Treasury and some participants with whom we spoke told us that Treasury has also been sharing some SSBCI performance information with participants at its SSBCI national conferences. For example, at its June 2013 conference, Treasury shared preliminary performance information from the 2012 annual reports on the number of loans and investments made with SSBCI funds and the amount of SSBCI funds loaned or invested. In 2012 we recommended that in order to provide information on the progress of SSBCI to Congress and the SSBCI participants, Treasury should make information publicly available on its performance indicators for SSBCI.September 2013, Treasury released a summary report of performance In information collected in participants’ 2012 annual reports.contains information on each of Treasury’s program performance measures, as of December 31, 2012 (see table 2). The report also summarizes participants’ data in numerous ways, such as by program type, distribution of loans or investments by size and industry, loans or investments made in low- and moderate-income communities, and lender and investor participation by size of lender. Treasury officials added that they expect to continue to release quarterly report data and issue annual reports to the public going forward.

As previously discussed, because of the variety of SSBCI programs implemented by participants, as well as the influence of factors beyond Treasury’s control, Treasury officials said that SSBCI’s performance cannot be evaluated using a single number or performance indicator. Further, performance measures may not always give a complete understanding of a program’s effectiveness. To address these issues, Treasury has started to determine how it will evaluate the performance of SSBCI at the end of the program in 2017. As part of Treasury’s SSBCI oversight and performance strategy, all Allocation Agreements require participants to conduct a final assessment on their programs’ performance at the end of the program in 2017. Treasury officials explained that they are in the early stages of thinking about how they will use this information to help them evaluate SSBCI’s performance and to determine if the program model for SSBCI was successful. Officials explained that they are planning on convening a working group of program participants to help Treasury design the format for participants’ final assessments. They added that they are planning on inviting experts in program measurement to the meeting to help determine the design of the evaluation. Officials added that they have reviewed evaluations of other Treasury programs, as well as evaluations conducted by other agencies and SSBCI participants, to help them in their planning process. Treasury officials explained that they expect to convene the working group of participants by the end of 2013, but that they expect that the design of the evaluation will not be finalized until late in 2014.

The Small Business Jobs Act of 2010 was signed by the President on September 27, 2010. As described in this report, as well as our 2011 and 2012 reports, Treasury has spent the last few years starting up the SSBCI program office; developing policies, procedures, and guidance; assisting participants with the implementation of their programs; and developing measures to monitor program administration and performance. Also as previously discussed, 2013 is the first year in which Treasury had a complete set of participants’ annual reports. As a result, Treasury now has more complete information to analyze the program. As Treasury begins to design its evaluation, it should consider requirements in the act, including the Secretary of the Treasury consulting with SBA and the appropriate federal banking agencies on the administration of the program. In addition, because the SSBCI program is included as one of the programs that will help to meet a Cross-Agency Priority goal, obtaining input from the agencies involved in the working group on increasing access to capital for entrepreneurs and small businesses (Ex- Im, SBA, and USDA) could help to ensure that the evaluation is useful in measuring progress toward that goal. Further, our guidance on designing program evaluations encourages agencies to obtain input from external stakeholders, such as Congress, that are expected to be interested in the evaluation results. By seeking input from Congress on the types of information in which it is interested, Treasury can help ensure that it designs an evaluation that is useful for decision makers.

Conclusions

In SSBCI’s third year, participants’ fund usage has started to accelerate, with participants nearly quadrupling the amount of funds they used between June 30, 2012, and June 30, 2013. Treasury implemented measures in 2012 to help monitor the administration and performance of SSBCI. However, while it developed targets for the measures related to administration, it did not develop targets for the measures related to program performance. Treasury officials stated that it did not develop these targets because, among other things, factors outside of the program’s control can have a significant impact on SSBCI’s outcomes. Federal internal control standards and leading performance management practices state that agencies should set performance targets, where appropriate. Targets for selected performance measures could help Treasury better manage SSBCI by identifying performance gaps and highlighting program areas that may need additional attention. While we recognize that it could be challenging to set performance targets when the program is influenced by external factors, SSBCI is not the only federal program facing this challenge, and our previous work has identified strategies for setting targets in similar situations. Moreover, Treasury has set targets for performance measures of other programs that, like SSBCI, include multiple programs that are administered locally and help to promote access to capital. In addition, now that Treasury has received participants’ 2012 annual reports, it has baseline data to analyze and use to help set targets for selected measures on program performance.

Treasury has also begun to consider how it will evaluate the effectiveness of the program at the program’s conclusion in 2017. Our guidance on designing performance evaluations encourages agencies to obtain input from external stakeholders, including Congress, who are expected to be interested in the evaluation results. In addition, the Small Business Jobs Act of 2010 requires the Secretary of the Treasury to consult with SBA and the appropriate federal banking agencies on the administration of SSBCI. Because Treasury’s SSBCI evaluation could be useful in informing future policymakers about programs that are intended to increase capital to small businesses in distressed credit markets, gathering input from external stakeholders on what information would be most valuable to them is essential.

Recommendations for Executive Action

We recommend that the Secretary of the Treasury take the following actions: To enhance the use of performance measures to manage SSBCI and assess progress, Treasury should establish targets for selected performance measures related to monitoring program performance.

To better ensure that Treasury’s SSBCI program evaluation provides information useful for future decision making and effective management and oversight, Treasury should seek input from program stakeholders— including other agencies involved in promoting small businesses and Congress—as it designs its SSBCI program evaluation.

Agency Comments and Our Evaluation

We provided a draft of this report to Treasury, FDIC, Federal Reserve, and OCC for their review and comment. Treasury provided written comments, which are reprinted in appendix III. FDIC, Federal Reserve, and OCC did not provide written comments on the draft report. In its written comments, Treasury agreed with our recommendations and stated that it has begun to take steps to implement both of them. Specifically, Treasury said it has begun establishing targets for program performance measures and gathering input from program stakeholders in designing SSBCI’s program evaluation. Treasury noted that the report reflected the progress SSBCI participants have made in using their SSBCI funds and that program stakeholders have undertaken numerous outreach activities to promote SSBCI. Treasury also stated that it is confident that as SSBCI progresses it will continue to promote access to credit for small businesses, allowing them to expand and create new jobs. Treasury also provided a technical comment on the draft report, which we’ve incorporated in the final report.

We are sending copies of this report to appropriate congressional committees and Treasury, FDIC, Federal Reserve, and OCC. In addition, the report is also available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-8678 or garciadiazd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to examine the extent to which (1) State Small Business Credit Initiative (SSBCI) participants have used their SSBCI allocations and faced challenges in using the funds, (2) the Department of the Treasury (Treasury) and program stakeholders have used outreach activities to promote use of the program and assessed the success of these efforts, and (3) Treasury has used performance measures and program evaluation to manage SSBCI.

To assess the extent to which SSBCI participants have used their SSBCI allocations, we analyzed Treasury program data on participants’ allocation amounts, disbursement requests, and program types, and whether participants developed new programs to participate in SSBCI or expanded existing programs with SSBCI funds. In addition, we analyzed participants’ June 30, 2013, SSBCI quarterly report data—the most recent quarter available—on the total amount of funds used and fund usage by program type. To show the trend in participants’ total fund usage from June 30, 2012, to June 30, 2013, we also analyzed data from participants’ June 30, 2012, and March 31, 2013, quarterly reports. To assess the reliability of SSBCI data on participants’ fund usage, allocation amount, and program type, and whether participants developed new programs or expanded existing ones, we performed electronic testing for obvious errors in accuracy and completeness and compared Treasury’s data on program type to participants’ quarterly data. Where there were questions or discrepancies we identified related to the data, we clarified them through discussions with Treasury officials. We also interviewed knowledgeable Treasury officials on how they assess the reliability of participants’ quarterly data and the information they maintain on SSBCI. We determined that the data were sufficiently reliable for the purposes of describing these factors.

To examine the extent to which SSBCI participants faced challenges in using SSBCI funds, we reviewed reports from Treasury’s Office of the Inspector General (OIG) on participants’ usage of SSBCI funds; our 2011 and 2012 reports on SSBCI; and Treasury’s policies and procedures, including its policies on disbursing subsequent tranches of SSBCI funds and terminating the availability of participants’ SSBCI allocations.addition, we interviewed officials from Treasury and nine SSBCI In participants: Arizona; Iowa; Maryland; Missouri; Montana; New York; City of Carrington, North Dakota; Vermont; and Virginia. We selected these nine participants from the total of 57 participants—which included 47 states; Washington, D.C.; four municipalities; and five U.S. territories— using participants’ March 31, 2013, quarterly report data. We based our selection on the following criteria: (1) the percentage of SSBCI funds participants had used as of March 31, 2013; (2) the amount of SSBCI funds allocated to participants; (3) the number of programs participants implemented and whether the programs were new or existing; and (4) geographic dispersion. We selected this nonrandom sample of participants because we assumed that the types of challenges related to using SSBCI funds would vary for participants characterized by a mix of fund usage histories, allocation amounts, and numbers of new and existing programs across all regions. To reduce administrative burden, we excluded from our selection (1) the seven participants that the OIG told us it was in the process of auditing or was planning to audit at the time we selected the participants, and (2) the nine participants we interviewed in our 2012 SSBCI review. When we selected participants to interview, the March 31, 2013, quarterly data were the most recent available. Because Treasury had not completed its review of the data at that time, we used preliminary quarterly report data for our selection. We also reviewed testimonial evidence we obtained during our 2011 and 2012 SSBCI reports related to challenges SSBCI participants faced in using SSBCI funds. Finally, we interviewed representatives from a number of trade associations representing the lending and venture capital communities: the Conference of State Bank Supervisors, Community Development Venture Capital Alliance, Council of Development Finance Agencies, Independent Community Bankers Association, National Association of Federal Credit Unions, Opportunity Finance Network, and State Science & Technology Institute. We selected these trade associations based on information from Treasury officials on the organizations with whom they had worked.

To examine the extent to which Treasury and program stakeholders, including SSBCI participants, federal banking regulators, and trade associations, have used outreach activities to promote use of the program and assessed the success of these efforts, we interviewed Treasury officials and representatives of stakeholder groups and reviewed materials pertinent to SSBCI outreach. Specifically, we interviewed Treasury officials, representatives from the nine SSBCI participants and seven trade associations identified above, and officials from the federal banking regulators—Board of Governors of the Federal Reserve System (Federal Reserve), Federal Deposit Insurance Corporation (FDIC), and Office of the Comptroller of the Currency (OCC)—to learn about outreach activities they have conducted for SSBCI and to obtain relevant materials and documentation. We reviewed Treasury guidance on SSBCI outreach, including marketing best practices for SSBCI loan participation, loan guarantee, and collateral support programs, as well as Treasury budget documents. We also reviewed materials related to Treasury’s pilot program to hire outreach consultants for the SSBCI programs in Mississippi, Ohio, and South Carolina. These materials included the solicitation for subject-matter experts for lender outreach support, technical assistance plans, and final reports developed by the consultants for the three states. We reviewed outreach materials SSBCI participants, federal banking regulators, and trade associations had developed related to SSBCI. We also reviewed all 57 SSBCI participants’ websites to understand how SSBCI participants are providing information about SSBCI to interested parties.they contained detailed program descriptions; information on the amount of SSBCI funds allocated to the participants; links to additional, related materials; information or materials on how to apply for the program; and contact information for additional information, among other things. We interviewed officials from Treasury, the federal banking regulators, and the nine SSBCI participants identified above on the methods they used to assess the success of their outreach efforts. We also reviewed Treasury’s conference summary memorandums, which include conference attendees’ responses to the satisfaction surveys that Treasury administers after each conference, as well as other general observations on the conferences.

We examined the websites to determine if To assess the extent to which Treasury has used performance measures and program evaluation to manage SSBCI, we reviewed performance measures and targets Treasury has set for SSBCI. We also reviewed a tool Treasury uses to track its progress on some performance measures and interviewed officials on actions they have taken to manage the program by using performance measures. We identified relevant criteria on setting targets for performance measures in the federal internal control standards and leading practices for federal agencies’ performance management. In addition, we reviewed performance information Treasury has shared with SSBCI participants at its conferences and a summary report of performance information that Treasury developed in September 2013, and we interviewed officials on their plans to publicly release SSBCI performance information. We also reviewed the second and third quarter fiscal year 2013 status update documents developed for the Cross-Agency Priority goal on Entrepreneurship and Small Business to identify SSBCI performance data Treasury has shared with other federal agencies. Finally, we reviewed the Small Business Jobs Act of 2010 to identify requirements for Treasury to evaluate SSBCI. We also interviewed Treasury officials on their plans to evaluate SSBCI, including how they intend to use participants’ 2017 assessments of their programs, as required by their Allocation Agreements. We used criteria from our guidance on designing program evaluations to evaluate Treasury’s plans for its evaluation of SSBCI.

We conducted this performance audit from May 2013 to December 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: SSBCI Participants’ Allocation Amount and Fund Usage, as of June 30, 2013

We reviewed Treasury’s program data and SSBCI participants’ quarterly report data as of June 30, 2013, and found that SSBCI participants had used their SSBCI allocations at varying rates.

Appendix III: Comments from the Department of the Treasury

Appendix IV: GAO Contacts and Staff Acknowledgements

GAO Contact

Daniel Garcia-Diaz, (202)-512-8678, garciadiazd@gao.gov.

Staff Acknowledgments

In addition to the individual named above, Kay Kuhlman (Assistant Director), Bethany Benitez, Anna Chung, Elizabeth Curda, Pamela Davidson, Christine Houle, Beverly Ross, Jennifer Schwartz, and Jena Sinkfield made key contributions to this report.