State Small Business Credit Initiative: Opportunities Exist to Improve Program Oversight
Published: 2011-12-07T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Congress enacted the Small Business Jobs Act of 2010 in September 2010 in response to concerns that small businesses have been unable to access capital that would allow them to create jobs. Among other things, the act aims to stimulate job growth by establishing the $1.5 billion State Small Business Credit Initiative (SSBCI) within the Department of the Treasury (Treasury) to strengthen state and territory (state) programs that support lending to small businesses and small manufacturers. Participating states are expected to leverage the SSBCI funds to generate an amount of private financing and investment at least 10 times the amount of their SSBCI funds (that is, a leverage ratio of 10:1). The act also requires GAO to audit SSBCI annually. Accordingly, this report examines (1) which states applied for SSBCI funds and the planned uses of those funds; (2) Treasury's implementation of SSBCI; and (3) Treasury's efforts to measure whether SSBCI achieves its goals. GAO surveyed state SSBCI applicants (for a 100 percent response rate), analyzed data from Treasury case files, and interviewed officials from Treasury and eight participating states.
section_title: What GAO Found
paragraphs: Fifty-four of the 56 eligible states and territories submitted applications requesting a total of about $1.4 billion in SSBCI funds. According to GAO's survey of SSBCI applicants, states plan to support 153 lending programs nationwide with SSBCI funds, 69 of which are new programs being created because of the SSBCI program. These lending programs include a variety of capital access programs and other credit support programs, with venture capital programs receiving the largest amount of funds among eligible program types. SSBCI applicants anticipate that their SSBCI funds will allow them to leverage up to $18.7 billion in new private financing and investment. Some applicants, however, expressed concern that achieving a 10:1 leverage ratio of private financing and investment to program funds could ultimately prove challenging, especially for states creating new programs. Treasury's procedures for SSBCI have evolved throughout its implementation of the program. Treasury began approving applications for SSBCI funds in January 2011 in accordance with guidance it issued in December 2010. However, Treasury did not finalize its application guidance and review procedures until April and May 2011, respectively. Some states indicated they delayed submitting their applications until Treasury's guidance was finalized, with 37 states not submitting an application until June 2011--the deadline for applications. In addition, Treasury did not finalize its procedures for disbursing subsequent installments of funds to states until November 2011, citing potential different legal interpretations of the act's disbursement requirements as the cause for the delay. Treasury is implementing a plan to monitor states' compliance with program requirements, which will include sampling transaction-level data to evaluate the accuracy of the states' annual reports. The Treasury Inspector General made recommendations in August 2011 to improve the tools Treasury will use to monitor state compliance. Treasury has not yet established performance measures for the SSBCI program. Treasury officials noted they are considering several draft performance measures to assess the efficiency of the program. However, Treasury has not finalized its plans for measuring the SSBCI program's performance. GAO and others have recognized the importance of using performance measures to gauge the progress of programs. GAO has also identified key attributes of successful performance measures. Given the preliminary nature of Treasury's potential performance measures, assessing whether the measures reflect the attributes of successful performance measures is premature. Nonetheless, considering these attributes as it works to finalize the SSBCI-specific performance measures could help Treasury to develop robust measures. Until such measures are developed and implemented, Treasury will not be able to determine whether the program is achieving its goals.
section_title: What GAO Recommends
paragraphs: GAO recommends that Treasury direct the SSBCI Program Manager to consider key attributes of successful performance measures when developing and finalizing SSBCI-specific performance measures. Treasury concurred with the report's recommendation.

section_title: Letter
paragraphs: Small businesses play a vital role in the U.S. economy, accounting for  about half of private sector output and employing more than half of private  sector workers. Because of their importance, Congress has grown  increasingly concerned that small businesses might not be able to access  enough capital to enable them to spur job creation in the nation’s ongoing  economic recovery. In 2008 and early 2009, major disruptions of business  credit markets made accessing credit difficult for small businesses. For  example, a Wells Fargo survey showed that the number of small  businesses having difficulty accessing credit had more than tripled  between 2007 and 2010, with ultimately almost 40 percent of small  businesses indicating that credit was difficult to obtain. Although most  financial institutions reported in April 2010 that they were no longer  tightening their small business lending standards, the Federal Reserve  noted at the time that those lending standards “remained quite stringent.”  The Secretary of the Department of the Treasury (Treasury) testified in  June 2011 that small businesses were concentrated in sectors that had  been especially hard-hit by the recession, including construction-related  industries. As a result, during the depths of the crisis, the rate of job  losses was almost twice as high in small businesses as it was in larger  firms.
The Small Business Jobs Act of 2010 created the State Small Business  Credit Initiative (SSBCI) within Treasury to strengthen state programs that  support lending to small businesses and small manufacturers. Funded  with $1.5 billion, SSBCI was designed to help spur up to $15 billion in  lending to small businesses. Under the program, participating states,  territories, and the District of Columbia (states) are to use the federal  funds for programs that leverage private lending to help finance small  businesses and manufacturers that are creditworthy but have been  unable to secure the loans needed to expand and to create jobs. SSBCI  is designed to allow states to build on existing or new models for state  small business programs.
The act also requires that we annually audit SSBCI. Accordingly, this  report examines (1) which states are participating in SSBCI and their  planned uses of those funds, (2) Treasury’s implementation of the SSBCI  program, and (3) Treasury’s efforts to measure whether the SSBCI  program achieves its goals.
To determine which states applied for and received SSBCI funds and the  planned uses of the funds, we developed a Web-based questionnaire to  collect information from the 54 states and territories that filed a Notice of  Intent to Apply for SSBCI funds for which we achieved a 100 percent  response rate. On the basis of our application of recognized survey  design practices and follow-up procedures, we determined that the data  collected via our survey were of sufficient quality for our purposes. We  also conducted interviews with Treasury officials, as well as selected  state officials and financial institutions within those states, either via  teleconference or site visits to inform our understanding of states’ planned  uses of SSBCI funds. We limited our selection of states to interview to  those states whose SSBCI applications had been reviewed and approved  and for which the applicant had signed an allocation agreement by June  30, 2011: California, Hawaii, Indiana, Kansas, Maryland, Missouri, North  Carolina, and Vermont.
To evaluate Treasury’s implementation of the SSBCI program, we  compared and contrasted Treasury’s SSBCI procedures and planned  control activities with GAO’s internal control standards, including Internal  Control in the Federal Government. We also interviewed Treasury  officials about the types of training it provided its staff to help ensure  compliance with its procedures. Additionally, we utilized data obtained  through our questionnaire to identify the dates on which states submitted  their SSBCI applications and whether Treasury required resubmission  and reviewed a nonprobability sample of SSBCI applications consisting of  all eight states that had signed an SSBCI allocation agreement by June  30, 2011, to determine whether all aspects of these states’ applications  were considered. We assessed whether Treasury followed its procedures  and appropriately documented its decisions by analyzing the  documentation of the application reviews. Because we used a  nongeneralizable sample to select the applications to review, our findings  cannot be used to make inferences about SSBCI applications of states  that signed allocation agreements after June 30, 2011. However, we  determined that the sample would be useful in providing illustrative  examples on procedures and documentation practices applied by  Treasury. Furthermore, we conducted interviews with Treasury officials  about the type of testing the agency plans to perform of its controls to  ensure compliance with SSBCI procedures, lessons learned about the  review process, how they addressed problems, and their plans to follow  up with states to ensure that SSBCI funds are used for the intended  purposes outlined in approved applications for program funds.
To review Treasury’s efforts to measure whether the SSBCI program  achieves its goals of increasing small business investment and creating  jobs, we discussed with Treasury their proposed performance metrics for  the SSBCI program. We also interviewed Treasury officials as well as  officials from the eight states that had signed a SSBCI allocation  agreement with Treasury by June 30, 2011, to collect documentation that  was used to inform our understanding of SSBCI program performance  and Treasury’s metrics. A more detailed description of the scope of our  review and the methods we used is contained in appendix I, and a copy  of our questionnaire can be found in appendix II.
We conducted this performance audit from February 2011 to December  2011 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The Small Business Jobs Act of 2010 (the act) aims to address the  ongoing effects of the 2007-2009 financial crisis on small businesses and  stimulate job growth by establishing the SSBCI program, among other  things. SSBCI is designed to strengthen state programs that support  private financing to small businesses and small manufacturers that,  according to Treasury, are not getting the loans or investments they need  to expand and to create jobs. The act did not require a specific number of  jobs to be created or retained as a result of SSBCI funds.
The act appropriated $1.5 billion to be used by Treasury to provide direct  support to states for use in programs designed to increase access to  credit for small businesses. Using a formula contained in the act,  Treasury calculated the amount of SSBCI funding for which each of the  50 states, as well as the District of Columbia, the Commonwealth of  Puerto Rico, the Commonwealth of the Northern Mariana Islands, Guam,  American Samoa, and the United States Virgin Islands were eligible to  apply. This formula takes into account a state’s job losses in proportion  to the aggregate job losses of all states. (See app. III for more information  on available funding by location). In addition to states, the act granted  permission to municipalities to apply directly for funding under SSBCI in  the event that a state either failed to file a Notice of Intent to Apply for its  allocation of program funds by November 26, 2010, or, after filing a  Notice of Intent, failed to submit an application to Treasury by June 27,  2011. Treasury officials stated that municipalities granted permission to  submit an application for program funds were generally subject to the  same approval criteria and program requirements as states.
Municipalities were eligible to apply for up to the total amount of their  state’s SSBCI allocation, but the final approved amounts were to be  apportioned based on their pro rata share by population of all  applicants. Figure 1 provides a timeline of major SSBCI milestones.
The act allowed Treasury to provide SSBCI funding for two state program  categories: capital access programs (CAP) and other credit support  programs (OCSP). A CAP is a loan portfolio insurance program wherein  the borrower and lender, such as a small business owner and a bank,  contribute to a reserve fund held by the lender. Under the act, approved  CAPs are eligible to receive federal contributions to the reserve funds  held by each participating financial institution in an amount equal to the  total amount of the insurance premiums paid by the borrower and the  lender on a loan-by-loan basis. Amounts in the lender’s reserve fund are  then available to cover any losses incurred in its portfolio of CAP loans.  For an SSBCI loan to be eligible for enrollment in a state’s approved  CAP, the borrower must have 500 or fewer employees and the loan  amount cannot exceed $5 million.
In addition, the following types of OCSPs are eligible to receive SSBCI  funds under the act:
Collateral support programs: These programs supply cash collateral  accounts to lenders to enhance the collateral coverage of borrowers.  The accounts will cover all or a portion of the collateral shortfall  identified by a lending institution. These programs can be designed to  target certain regions or industries, such as equipment lending, in which  a lender may be willing to fund at 80 percent loan-to-value, but a  borrower may not be able to bridge the difference in cash at closing.
Loan participation programs: These programs enable small  businesses to obtain medium- to long-term financing, usually in the  form of term loans, to help them expand their businesses. States may  structure a loan participation program in two ways: (1) purchase  transactions, also known as purchase participation, in which the state  purchases a portion of a loan originated by a lender and (2)  companion loans, also known as co-lending participation or parallel  loans, in which a lender originates one loan and the state originates a  second (usually subordinate) loan to the same borrower. This  program enables the state to act as a lender, in partnership with a  financial institution, to provide small business loans at attractive  terms.
Direct loan programs: Although Treasury does not consider these  programs to be a separate SSBCI program type, it acknowledges that  some states may identify programs that they plan to support with  SSBCI funds as direct loan programs. The programs that some states  label as direct loan programs are viewed by Treasury as co-lending  programs categorized as loan participation programs, which have  lending structures that are allowable under the statute.
Loan guarantee programs: These programs enable small businesses  to obtain term loans or lines of credit to help them grow and expand  their businesses by providing a lender with the necessary security, in  the form of a partial guarantee, for them to approve a loan or line of  credit. In most cases, a state sets aside funds in a dedicated reserve  or account to guarantee a specified percentage of each approved  loan.
Venture capital programs: These programs provide investment capital  to create and grow start-ups and early-stage businesses, often in one  of two forms: (1) a state-run venture capital fund (which may include  other private investors) that invests directly in businesses or (2) a fund  of funds, which is a fund that invests in other venture capital funds  that in turn invest in individual businesses. Many factors, particularly  resources and available talent, inform a state’s decision on which form  to choose. For example, a state may choose to invest in a large  venture fund that agrees to reinvest in that state an amount equal to  that invested by the state, as opposed to trying to attract that same  talent to a smaller fund capitalized with state money.
Qualified loan or swap funding facilities: States may enter into  qualifying loan or swap funding transactions under which SSBCI funds  are pledged as collateral for private loans or credit lines. The private  financing proceeds must, however, be used exclusively for the  reserve or other accounts that back the credit support obligations of a  borrowing CAP or OCSP. Presumably, fees paid by borrowers and  lenders will provide a return to the providers of private capital.
Other OCSPs: States were also able to submit an application to  Treasury outlining their plans to support OCSPs that, though not able  to be categorized in any of the above OCSP types, feature  combinations of aspects of these eligible types.
OCSPs approved to receive SSBCI funds are required to target  borrowers with an average size of 500 or fewer employees and to target  support towards loans with average principal amounts of $5 million or  less. In addition, these programs cannot lend to borrowers with more than  750 employees or make any loans in excess of $20 million.
In applying for funding, applicants had to demonstrate that their CAPs  and OCSPs could satisfy SSBCI criteria. For example, applicant states  had to demonstrate that all legal actions had been taken at the state level  to accept SSBCI funds and implement the state programs. States were  also required to demonstrate that the state possessed the operational  capacity, skills, and financial and management capacity to meet the  objectives set forth in the act. In addition, each applicant was required to  demonstrate a “reasonable expectation” that its participating programs,  taken together, would generate an amount of private financing and  investment at least 10 times its SSBCI funding (that is, a leverage ratio of  10:1) by the program’s end in December 2016. Furthermore, each  application had to include a report detailing how the state would use its  SSBCI allocation to provide access to capital for small businesses in low-  and moderate-income, minority, and other underserved communities,  including women- and minority-owned small businesses.
The act requires that each state receive its SSBCI funds in three  disbursements of approximately one-third of its approved allocation. Prior  to receipt of the second and third disbursements, a state must certify that  it has expended, transferred, or obligated 80 percent or more of the  previous disbursement to or for the account of one or more approved  state programs. Treasury may terminate any portion of a state’s allocation  that Treasury has not yet disbursed within 2 years of the date on which its  SSBCI Allocation Agreement was signed. Treasury may also terminate,  reduce, or withhold a state’s allocation at any time during the term of the  Allocation Agreement upon an event of default under the agreement.  Following the execution of the Allocation Agreement, states are required   All  to submit quarterly and annual reports on their use of SSBCI funds. SSBCI Allocation Agreements, the primary tool signed by Treasury and  each participating state, which outline how recipients are to comply with  program requirements, will expire on March 31, 2017.
The program’s reporting requirements are detailed in section 4.8 of the SSBCI allocation  agreement. The obligations of participating states and territories to perform and report on  progress will expire as outlined in the terms of the agreement.
section_title: Most States Are Participating in SSBCI and Plan to Use the Funds to Support a Variety of Programs
paragraphs: Nearly all of the states eligible for SSBCI funds submitted applications to  Treasury. Fifty-four of the 56 states and territories that were eligible to  apply for program funds submitted an application prior to the June 27,  2011, deadline, although one state later withdrew its application. In total,  states requested more than $1.4 billion in SSBCI funds—95 percent of  the program’s appropriation—and only one applied for less than its  maximum allocation. Following the application deadline for states,  Treasury received five additional applications from municipalities in three  states—Alaska, North Dakota, and Wyoming—by the September 27,  2011, deadline requesting a total of $39.5 million in program funds. Figure  2 illustrates the distribution of SSBCI funds applied for by states and  territories.
Participating states indicated that they are planning to support various  new, existing, and dormant (that is, previously suspended) lending  programs with their respective SSBCI allocations. According to our survey  results, states are planning to support 153 different lending programs, 69  of which are new programs that were created to be supported by SSBCI  funds (see fig. 3). Forty-one states indicated they are planning to support  more than one program with their allocation. For example, Alabama plans  to support a CAP, four loan participation programs, and a loan guarantee  program, and New Jersey plans to support a loan participation program,  four loan guarantee programs, five direct loan programs, and a venture  capital program.
According to our survey results, states are planning to support CAPs and  all types of eligible OCSPs except loan and swap funding facilities (see  fig. 4). Venture capital programs are to receive the largest amount of  SSBCI funds of any program type. According to Treasury officials, states  submitted their respective applications with plans for developing  programs in response to unique gaps in local markets or the specific  expertise of their staff. Consequently, there is variation in program design  across states. For example, Treasury officials stated that Michigan plans  to use its funds to support a collateral support program because of  difficulties that manufacturing companies in the state were experiencing in  obtaining credit. Specifically, Treasury officials noted that as these  manufacturers’ real estate and equipment declined in value, they were  facing difficulties in obtaining credit due to collateral shortfalls (see app.  IV for more information on planned uses of funds by location).
States indicated that they expect SSBCI funds to result in a total of $18.7  billion in new private financing and investment throughout the life of the  program. In responding to our survey, officials from 39 of the states that  applied for SSBCI funds indicated that they expect to achieve a private  leverage ratio between 10:1 and 15:1, and 14 projected a ratio of 15:1 or  greater. However, each participating state’s generation of an amount of  private financing and investment at least 10 times its SSBCI allocation by  December 2016 is not a requirement, and some states indicated that they  believe reaching a 10:1 private leverage ratio could prove challenging.  For example, officials from one state expressed some concern that the  state’s final leverage ratio may ultimately fall short of the estimate  included in its approved application because the state was creating a new  program and, therefore, did not have prior experience operating a similar  program. Treasury officials noted that a state’s mix of programs, as well  as the design of each individual program, drives the leverage estimates.  For example, Treasury officials stated that private leverage ratios for  CAPs tend to be the highest among program types and are evident  immediately because the program design is such that the SSBCI subsidy  per loan is quite small and is not dependent on subsequent private  financing. However, the officials noted that OCSPs tend to have lower  leverage ratios initially but may see those grow in later years as program  funds are recycled for additional lending over time.
section_title: Treasury’s Evolving Processes Have Created Some Delays
subsections: section_title: Treasury Revised Its Application Guidance and Review Procedures to Clarify Requirements and Thoroughly Reviewed Selected Applications
paragraphs: With the enactment of the Small Business Jobs Act of 2010 on  September 27, 2010, Treasury was tasked with quickly starting up an  SSBCI program office and developing processes and guidance to  implement this new program. After accepting Notices of Intent to Apply  from states and territories by the end of November 2010, Treasury issued  an initial set of policy guidelines and application materials via its website  on December 21, 2010. According to Treasury officials, Treasury received  a few applications shortly thereafter and was able to review and approve  them and to obtain signed Allocation Agreements with and distribute first  installments of funds to two states in January 2011. In response to  feedback from states, discussions with other federal agencies, such as  the Small Business Administration, and current trends in the small  business banking arena, Treasury determined that it needed to revise its  guidelines and application paperwork to better articulate what  documentation was required for both the application and review  processes. As a result, Treasury issued revised guidance materials and  Allocation Agreements for applicants in April 2011 as well as a reviewers’  manual for its review staff in May 2011. According to our survey of SSBCI  applicants, five states submitted the final version of their application to  Treasury before these documents were finalized. Treasury officials told us  that although they took steps to help ensure consistent treatment of  applicants, Treasury did not revisit previously approved applications once  review procedures were finalized. Treasury officials said they were  confident that no additional review was required, as those early  applications were from states with well-established programs. However,  as a result of the revisions to the Allocation Agreement made in April  2011, Treasury asked the two states that had signed the previous  versions to sign an amended Allocation Agreement that incorporated the  new terms.
Some states reported that they delayed submitting their applications until  Treasury’s final application guidance was issued. According to our survey  results, 37 states did not submit their final applications for SSBCI funds  until June 2011, the month that applications were due. Despite the delay  in providing application guidance, applicants generally viewed Treasury  officials as helpful throughout the application process—providing answers  to most questions immediately and determining answers as soon as  possible when not readily available. Treasury officials stated that they  also hosted multiple webinars and conference calls to field questions  about the application process that were highly attended by states and  territories.
In our review of the eight applications reviewed and approved before  June 30, 2011, we found that Treasury considered each aspect of the  application. Although only one of the applications we reviewed was  processed under the revised application and review guidelines, we found  that each application was subject to five stages of review: an initial  review, a subsequent review by a quality assurance reviewer, review by  the application review committee, a legal review, and final approval by the  designated Treasury official. Our reviews of the applications and the  experiences of the states suggest that applications were scrutinized in  terms of their completeness as well as the eligibility of the programs for  which states intended to use SSBCI funds. For example, Treasury  reviewers noted that in one state’s application, the state proposed several  modifications to its existing CAP, thereby bringing it under compliance  with SSBCI requirements. Similarly, SSBCI applicants reported that  Treasury scrutinized their applications. According to our survey results,  50 of the 54 applicants reported they were required to resubmit at least  parts of their applications for further review after their original  submissions. For example, one state noted that Treasury wanted  significant changes in its application, mainly in the areas of internal  controls, mix of programs, and contractor oversight. Another state noted  that Treasury determined that the state failed to specify that it was to  match the borrower and lender premium between 2 percent and 3.5  percent; Treasury officials asked the state to revise its application to  reflect this information and submit an amended application.
section_title: Delays in Finalizing Disbursement Procedures Slowed the Disbursement of Funds
paragraphs: As required under the act, Treasury is distributing SSBCI funds to  recipients in three installments. As of October 31, 2011, Treasury had  provided first installments to 46 states and territories, totaling about $424  million. However, Treasury did not begin processing state requests for  their second installment of funds until November 2011. According to  Treasury officials, Treasury had previously not acted on these requests  because they wanted to ensure that proper procedures were established  to ensure all certifications made as part of the request were adequately  substantiated. Specifically, they had to resolve how to determine whether  80 percent of a state’s initial disbursement of funds has been expended,  transferred, or obligated as required under the act. Treasury finalized its  disbursement procedures for second and third installments of SSBCI  funds at the beginning of November 2011. According to Treasury officials,  as of that date, no state had yet expended 80 percent of its initial  disbursement to support loans or investment to small businesses.
While Treasury was working to finalize these procedures, states were  potentially delayed in receiving their remaining SSBCI funding. For  example, officials from one state that we contacted told us they were  ready for their second installment after their first installment was  transferred to the accounts of their designated SSBCI lending programs,  but they were told by Treasury officials that they would have to wait until  the disbursement procedures were finalized. Consequently, the officials  told us their state faced additional interest expenses as a result of the  delay.
section_title: Treasury Is Implementing a Plan to Monitor Recipient Compliance with Program Requirements
paragraphs: Treasury is implementing a multi-step plan to monitor recipient  compliance with SSBCI program requirements. These steps include (1)  collecting and reviewing quarterly and annual reports, as well as quarterly  use of funds certifications, from recipients, (2) evaluating the accuracy of  recipient-level reporting on an annual basis by sampling transaction-level  data, (3) monitoring recipient requests for second and third installments of  SSBCI funds, and (4) contacting recipients on a quarterly basis to inquire  as to their adherence with plans outlined in their respective SSBCI  applications, as well as monitoring requirements. Treasury has developed  a secure, online system for states to report on those data fields included  in the Allocation Agreements signed by states, including (1) total amount  of principal loaned and of that amount, the portion that is from nonprivate  sources; (2) estimated number of jobs created or retained as a result of  the loan; and (3) amount of additional private financing occurring after the  loan closing. States are to provide these data to Treasury on an annual  basis beginning in March 2012. Treasury officials told us they plan to  sample states’ transaction-level data to help ensure the accuracy of state  reporting. Specifically, the SSBCI compliance manager is to take samples  of transaction-level data from all recipients in order to determine whether  states are entering these data accurately, including verifying that  transactions listed match the underlying loan or investment documents.  Treasury officials noted that the system is to automatically flag any loans  for which the data entered do not comply with program requirements.
Treasury officials told us they have also assigned three relationship  managers to serve as the primary Treasury contacts for the SSBCI  program. These managers, who have each been assigned 15 to 20  recipients, are to hold quarterly phone conversations with recipients.  During these calls, the managers are to ask a series of generic questions,  as well as recipient-specific questions regarding plans the states  described in their applications, such as hiring staff and monitoring the use  of program funds.
The Treasury Inspector General recently made recommendations to  further enhance Treasury’s oversight of SSBCI recipients. In August  2011, the Inspector General issued a report describing the results of its  review of SSBCI policy guidance and other key program documents,  including allocation agreements.recommendations to improve Treasury’s compliance and oversight  framework, including that Treasury’s guidance should clearly define the  oversight obligations of recipients and specify minimum standards for  determining whether recipients have fulfilled their oversight  responsibilities. Treasury concurred with eight of the recommendations  and has begun to take action to address them. Treasury disagreed with  the Inspector General’s recommendation to make additional provisions for   The report made nine  states to certify their allocation agreements, stating that states certify that  they are implementing their programs in compliance with SSBCI  procedures as part of their quarterly reporting to Treasury.
section_title: Treasury Is in the Process of Developing Performance Measures for SSBCI
paragraphs: Treasury officials told us that they have not yet established performance  measures for the SSBCI program. Although Treasury plans to rely  primarily on the department’s overall performance measures in evaluating  the SSBCI program, officials noted they are considering several draft  performance measures to assess the efficiency of the program. Treasury  officials described to us some of the potential measures they are  considering, but we are not including them in this report because they  have not yet been finalized. Treasury officials told us that they have not  finalized the program’s performance measures because they have been  focused on starting up the program quickly to meet statutorily required  deadlines. Furthermore, officials noted that because SSBCI is a  multilayered program that is implemented at the state level and  dependent upon private sector entities, Treasury’s ability to influence  program outcomes will be limited. Therefore, Treasury officials have been  trying to develop measures that focus on the aspects of the program  under Treasury’s control. According to Treasury officials they do not have  a time frame for fully developing and finalizing SSBCI-specific  performance measures.
The potential performance measures described by Treasury do not  currently include measures related to the number of jobs created or  retained as a result of the SSBCI program. As required in their allocation  agreements with Treasury, states are to report information on estimated  jobs resulting from SSBCI programs on a per loan or investment basis.  According to Treasury officials, gathering this information from the states  serves two purposes: (1) it allows Treasury to track the progress of the  states against the anticipated benefits articulated for their programs in their  SSBCI applications and (2) it provides Treasury with a potential data point  that may be useful when measuring overall program performance over  time. However, Treasury’s ability to use this information moving forward  could be limited, as the jobs data will be based on estimates and not actual  jobs. In particular, as part of the SSBCI loan and investment application  process, borrowers and investors are required to provide in their application  paperwork estimates of the number of jobs to be created and retained as a  result of participating in SSBCI programs. States then provide these  estimates in their annual reports to Treasury. However, the states are not  required to validate these jobs estimates, and they are not required to  follow up with borrowers and investors to determine whether the actual  number of jobs they were able to create or retain matched their original  estimates. According to one lending official we spoke with, validating these  estimates would be difficult and lenders could be discouraged from  participating in the SSBCI program if they were required to track actual jobs  created and retained. Concerned about the burden that reporting on actual  jobs created and retained would place on the small businesses receiving  SSBCI funds, Treasury officials told us that they elected to capture instead  estimated jobs data at the time of the closing of the loan or investment.  Treasury officials noted they are currently consulting with officials from the  Small Business Administration to learn what methods that agency uses in  measuring jobs using estimated data.
The importance of performance measures for gauging the progress of  programs and projects is well recognized. Measuring performance allows  organizations to track the progress they are making toward their goals  and gives managers crucial information on which to base their  organizational and management decisions. Leading organizations  recognize that performance measures can create powerful incentives to  influence organizational and individual behavior. In addition, the  Government Performance and Results Act of 1993 (GPRA) incorporates  performance measurement as one of its most important features. Under  GPRA, executive branch agencies are required to develop annual  performance plans that use performance measurement to reinforce the  connection between the long-term strategic goals outlined in their  strategic plans and the day-to-day activities of their managers and staff.  The Office of Management and Budget (OMB) has also directed agencies  to define and select meaningful outcome-based performance measures  that indicate the intended result of carrying out a program or activity.  Additionally, we have previously reported that aligning performance  metrics with goals can help to measure progress toward those goals,  emphasizing the quality of the services an agency provides or the  resulting benefits to users.
We have also previously identified criteria to evaluate an agency’s  performance measures. While GPRA focuses on the agency level,  performance measures are important management tools for all levels of  an agency—such as the bureau, program, project, or activity level—and  these criteria are applicable at those levels as well. Among other criteria,  we have identified nine key attributes of successful performance  measures. These attributes include the following:  (1)  Linkage. Measure is aligned with division- and agency-wide  goals and mission and clearly communicated throughout the  organization.  (2)  Clarity. Measure is clearly stated, and the name and definition  are consistent with the methodology used to calculate it.  (3)  Measurable target. Measure has a numerical goal.  (4)  Objectivity. Measure is reasonably free from significant bias or  manipulation.  (5)  Reliability. Measure produces the same result under similar  conditions.  (6)  Core program activities. Measures cover the activities that an  entity is expected to perform to support the intent of the program.  (7)  Limited overlap. Measure should provide new information beyond  that provided by other measures.  (8)  Balance. Balance exists when a suite of measures ensures that  an organization’s various priorities are covered.  (9)  Governmentwide priorities. Each measure should cover a priority  such as quality, timeliness, and cost of service.
Given the preliminary nature of Treasury’s potential performance  measures, assessing whether the measures will reflect the attributes of  successful performance measures would be premature. Nevertheless,  considering these attributes as it works to finalize SSBCI-specific  performance measures could help Treasury to develop robust measures.  Until such measures are developed and implemented, Treasury will not  be able to determine whether the program is achieving its goals.
section_title: Conclusions
paragraphs: In response to SSBCI’s short time frame, Treasury was able to design,  implement, and execute an application process for the program in a  matter of months. Appropriately, Treasury’s early efforts were focused on  establishing the application process and the process for disbursing initial  installments of funds to recipients as quickly as possible. Treasury is still  in the process of developing performance measures for the SSBCI  program. Measuring performance allows organizations to track progress  toward their goals and gives managers crucial information on which to  base decisions. At the program level, agencies can create a set of  performance measures that addresses important dimensions of program  performance and balances competing priorities. Performance measures  that successfully address important and varied aspects of program  performance are key elements of an orientation toward results. Effective  performance measures can provide a balanced perspective on the  intended performance of a program’s multiple priorities. While Treasury is  considering potential draft performance measures, it has not fully  developed or finalized a set of measures for the SSBCI program. Until  such measures are developed and implemented, Treasury will not be in a  position to determine whether the SSBCI program is effective in achieving  its goals.
section_title: Recommendation for Executive Action
paragraphs: We are making one recommendation to Treasury to improve its  implementation and oversight of the SSBCI program as follows:
To help ensure that the performance measures for the SSBCI  program are as robust and meaningful as possible, we recommend  that the Secretary of the Treasury direct the SSBCI Program Manager  to consider key attributes of successful performance measures as the  program’s measures are developed and finalized.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to Treasury for review and comment.  Treasury provided written comments that we have reprinted in appendix  V. Treasury also provided technical comments, which we have  incorporated, as appropriate.
In their written comments, Treasury agreed with our recommendation.  Treasury noted that it will consider the key attributes of successful  performance measures as it works to finalize measures for the SSBCI  program.
We are sending copies of this report to the appropriate congressional  committees, the Secretary of the Treasury, and other interested parties.  The report is also available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me at clowersa@gao.gov or (202) 512-8678. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. GAO staff who made major contributions to  this report are listed in appendix VI.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: To determine which states applied for and received State Small Business  Credit Initiative (SSBCI) funds and the planned uses of the funds, we  developed a Web-based questionnaire to collect information from the 54  states and territories that filed a Notice of Intent to Apply for SSBCI funds  with the Department of the Treasury (Treasury) by the November 26,  2010 deadline. The questionnaire included questions on the timing of  applications for SSBCI funds, the receipt of funds to date, the intended  uses of funds, and the potential impacts of program funds. See appendix  II for a copy of the questionnaire.
To minimize errors arising from differences in how questions might be  interpreted and to reduce variability in responses that should be  qualitatively the same, we conducted pretests with officials in three states,  both in person and over the telephone. To help ensure that we obtained a  variety of perspectives on our questionnaire, we selected officials from  states planning to support various types of programs with SSBCI funds.  Based on feedback from these pretests, we revised the questionnaire in  order to improve response quality. For instance, in response to one state  official’s comment that it would be difficult for respondents to answer with  confidence how many capital access programs (CAP) and other credit  support programs (OCSP) have recently been in operation across all  municipalities in a state, we removed the historical and specific program  budget questions and clarified our focus on the planned uses of SSBCI  funds. We conducted two additional pretests with other state officials to  ensure that the updated questionnaire was understandable.
After completing the pretests, we administered the survey. On August 4,  2011, we began sending e-mail announcements of the questionnaire to  the state and territory officials that had been identified as points of contact  in a list provided to us by Treasury, notifying them that our online  questionnaire would be activated in approximately 1 week. On August 15,  2011, we sent a second e-mail message to officials in which we informed  them that the questionnaire was available online and provided them with  unique passwords and usernames. On August 26, 2011, we began  making telephone calls to officials and sent them follow-up e-mail  messages, as necessary, to ensure their participation as well as to clarify  and gain a contextual understanding of their responses. By September  14, 2011, we had received completed questionnaires from 54 states and  territories, for a 100 percent response rate.
We used standard descriptive statistics to analyze responses to the  questionnaire. Because this was not a sample survey, there are no  sampling errors. To minimize other types of errors, commonly referred to as  nonsampling errors, and to enhance data quality, we employed recognized  survey design practices in the development of the questionnaire and in the  collection, processing, and analysis of the survey data. For instance, as  previously mentioned, we pretested the questionnaire with state officials to  minimize errors arising from differences in how questions might be  interpreted and to reduce variability in responses that should be  qualitatively the same. In addition, during survey development, we  reviewed the survey to ensure the ordering of survey sections was  appropriate and that the questions within each section were clearly stated  and easy to comprehend. We also received feedback from survey experts  who we asked to review the survey instrument. To reduce nonresponse,  another source of nonsampling error, we sent out e-mail reminder  messages to encourage officials to complete the survey. In reviewing the  survey data, we performed automated checks to identify inappropriate  answers. We further reviewed the data for missing or ambiguous  responses and followed up with respondents when necessary to clarify  their responses. On the basis of our application of recognized survey  design practices and follow-up procedures, we determined that the data  were of sufficient quality for our purposes.
In addition to the survey, we conducted interviews with Treasury officials,  as well as selected state officials and financial institutions within those  states either via teleconference or site visits to collect documentation that  informed our understanding of states’ planned uses of SSBCI funds. We  limited our selection of states to interview to those states whose SSBCI  applications had been reviewed, approved, and for which the applicant  had signed an allocation agreement by June 30, 2011: California, Hawaii,  Indiana, Kansas, Maryland, Missouri, North Carolina, and Vermont.
To evaluate Treasury’s implementation of the SSBCI program, we  compared and contrasted Treasury’s SSBCI procedures and planned  control activities with GAO’s internal control standards, including Internal  Control in the Federal Government. We interviewed Treasury officials  about the types of training it provided its staff to help ensure compliance  with its procedures. We also utilized data obtained through our  questionnaire to identify the dates on which states submitted their SSBCI  applications and whether Treasury required resubmission. Additionally,  we reviewed a nonprobability sample of SSBCI applications consisting of  all eight states that had signed an SSBCI allocation agreement by June  30, 2011, to determine whether all aspects of these states’ applications  were considered. We assessed whether Treasury followed its procedures  and appropriately documented its decisions by analyzing the  documentation of the application reviews. Because we used a  nongeneralizable sample to select the applications to review, our findings  cannot be used to make inferences about SSBCI applications of states  that signed allocation agreements after June 30, 2011. However, we  determined that the sample would be useful in providing illustrative  examples on procedures and documentation practices applied by  Treasury. Furthermore, we conducted interviews with Treasury officials  about the type of testing the agency plans to perform of its controls to  ensure compliance with SSBCI procedures, lessons learned about the  review process, how they addressed problems, and their plans to follow  up with states to ensure that SSBCI funds are used for the intended  purposes outlined in approved applications for program funds.
To review Treasury’s efforts to measure whether the SSBCI program  achieves its goals of increasing small business investment and creating  jobs, we discussed with Treasury their proposed performance metrics for  the SSBCI program. We also interviewed Treasury officials, as well as  officials from the eight states that had signed a SSBCI allocation  agreement with Treasury by June 30, 2011, to collect documentation that  was used to inform our understanding of SSBCI program performance  and Treasury’s metrics.
We conducted this performance audit from February 2011 to December  2011 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Copy of GAO Questionnaire
section_title: Appendix III: SSBCI Funds Applied for, Approved, and Disbursed, by Location, as of October 31, 2011
paragraphs: Table 1 below contains the amounts of SSBCI funds that have been  applied for, approved, and disbursed as of October 31, 2011. This  information was provided by state and territory officials who responded to  a GAO survey between August 15 and September 14, 2011 and by the  U.S. Treasury on October 31, 2011.
section_title: Appendix IV: Planned Uses of SSBCI Funds, by Location
paragraphs: Table 2 below contains information on states and territories’ plans for the  distribution of SSBCI funds among eligible program types, provided by  officials between August 15 and September 14, 2011.
section_title: Appendix V: Comments from the Department of the Treasury
section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Paul Schmidt, Assistant  Director; Pamela Davidson; Jill Lacey; Marc Molino; Patricia Moye; Deena  Richart; Christine San; Jennifer Schwartz; and Chad Williams made key  contributions to this report.