You are given two reports by the government agency.  

Report 1:
Under SBA’s 7(a) loan program, SBA guarantees loans made by commercial lenders to small businesses for working capital and other general business purposes. These lenders are mostly banks, but some are non-bank lenders, including small business lending companies— lenders whose lending activities are not subject to regulation by any federal or state regulatory agency, but were previously licensed by SBA and authorized to provide 7(a) loans to qualified small businesses. The guarantee assures the lender that if a borrower defaults on a loan, the lender will receive an agreed-upon portion (generally between 50 percent and 85 percent) of the outstanding balance. For a majority of 7(a) loans, SBA relies on lenders with delegated authority to approve and service 7(a) loans and to ensure that borrowers meet the program’s eligibility requirements. To be eligible for the 7(a) program, a business must be an operating for-profit small firm (according to SBA’s size standards) located in the United States and must meet the credit elsewhere requirement. Because the 7(a) program is required to serve borrowers who cannot obtain conventional credit at reasonable terms, lenders making 7(a) loans must take steps to ensure that borrowers meet the program’s credit elsewhere requirement. Because SBA relies on lenders with delegated authority to make these determinations, SBA’s oversight of these lenders is particularly important. However, we found in a 2009 report that SBA’s lack of guidance to lenders on how to document compliance with the credit elsewhere requirement was impeding the agency’s ability to oversee compliance with the credit elsewhere requirement. To improve SBA’s oversight of lenders’ compliance with the credit elsewhere requirement, we recommended in 2009 that SBA issue more detailed guidance to lenders on how to document their compliance with the credit elsewhere requirement. As a result, SBA revised its standard operating procedure to state that each loan file must contain documentation that specifically identifies the factors in the present financing that meet the credit elsewhere test, which we believe met the spirit of our recommendation. SBA’s current credit elsewhere criteria for determining 7(a) loan eligibility include the following factors: 1. the business needs a longer maturity than the lender’s policy permits; 2. the requested loan exceeds the lender’s policy limit regarding the amount that it can lend to one customer; 3. the collateral does not meet the lender’s policy requirements; 4. the lender’s policy normally does not allow loans to new businesses or businesses in the applicant’s industry; or 5. any other factors relating to the credit which, in the lender’s opinion, cannot be overcome except for the guarantee. When the 7(a) program was first implemented, borrowers were generally required to show proof of credit denials from banks that documented, among other things, the reasons for not granting the desired credit. Similar requirements remained in effect until 1985, when SBA amended the rule to permit a lender’s certification made in its application for an SBA guarantee to be sufficient documentation. This certification requirement remained when the rule was rewritten in 1996. SBA stated that it believed requiring proof of loan denials was demoralizing to small businesses and unenforceable by SBA. SBA and lender roles vary among 7(a) program categories—including regular 7(a), the Preferred Lenders Program, and SBA Express. Under the regular (nondelegated) 7(a) program, SBA makes the loan approval decision, including the credit determination. Under the Preferred Lenders Program and SBA Express, SBA delegates to the lender the authority to make loan approval decisions, including credit determinations, without prior review by SBA. For each 7(a) program category, lenders are required to ensure that borrowers meet the credit elsewhere requirement for all 7(a) loans. The maximum loan amount under the SBA Express program is $350,000, as opposed to $5 million for other 7(a) loans. The program allows lenders to utilize, to the maximum extent possible, their own credit analyses and loan underwriting procedures. In return for the expanded authority and autonomy provided by the program, SBA Express lenders agree to accept a maximum SBA guarantee of 50 percent. Other 7(a) loans generally have a maximum guarantee of 75 percent or 85 percent, depending on the loan amount. In fiscal year 2016, 1,991 lenders approved 7(a) loans, of which 1,321 approved at least one loan with some form of delegated authority. SBA’s Office of Credit Risk Management is responsible for overseeing 7(a) lenders, including those with delegated authority. SBA created this office in fiscal year 1999 to help ensure consistent and appropriate supervision of SBA’s lending partners. The office is responsible for managing all activities regarding lender reviews; preparing written reports; evaluating new programs; and recommending changes to existing programs to assess risk potential. Generally, the office oversees SBA lenders to identify unacceptable risk profiles using its risk rating system and enforce loan program requirements. According to SBA’s standard operating procedures, one of the agency’s purposes of its monitoring and oversight activities is to promote responsible lending that supports SBA’s mission to increase access to capital for small businesses. In the federal budget, the 7(a) program is generally required to set fees that it charges to lenders and borrowers at a level to cover the estimated cost of the program associated with borrower defaults (in present value terms). To offset some of the costs of the program, such as default costs, SBA assesses lenders two fees on each 7(a) loan. First, depending on the term of the loan, the guarantee fee must be paid by the lender within either 90 days of loan approval or 10 business days of the SBA loan number being assigned. This fee is based on the amount of the loan and the level of the guarantee, and lenders can pass the fee on to the borrower. Second, the servicing fee must be paid annually by the lender and is based on the outstanding balance of the guaranteed portion of the loan. The 7(a) program accounts for a small portion of total small business lending. According to a May 2017 report by the Consumer Financial Protection Bureau, the total debt financing available to small businesses was estimated to be $1.4 trillion. Of that amount, the Consumer Financial Protection Bureau estimated that about 7 percent was SBA loans, including 7(a) loans. SBA and some other researchers have suggested that there may be disparities in credit access among small businesses, based on characteristics of the borrower and firm. SBA lists as a strategic objective to “ensure inclusive entrepreneurship by expanding access and opportunity to small businesses and entrepreneurs in communities where market gaps remain.” In 2007, we reported that some studies had noted disparities among some races and genders in the conventional lending market, but the studies did not offer conclusive evidence on the reasons for those differences. Much of the research we reviewed in 2007 relied on the Board of Governors of the Federal Reserve System’s Survey of Small Business Finance, which was last implemented in 2003. Although this survey is no longer available, recently the 12 Federal Reserve Banks conducted the Small Business Credit Survey. In a series of reports based on the more recent survey, researchers found disparities in credit availability based on gender, the age of the firm, and minority status. From fiscal years 2007 through 2016, a majority of loan dollars guaranteed under the 7(a) program went to small businesses that were new, partially or wholly owned by women, or located in a distressed area. As previously mentioned, recent studies we reviewed by the Federal Reserve Banks and other researchers suggest that certain small business borrowers—including businesses that are new or owned by women—have difficulty obtaining conventional small business loans, which may put them at a disadvantage. As shown in figure 1, almost two-thirds of loan dollars guaranteed under the 7(a) program for this period went to small businesses that were in these two categories or located in a distressed area. The remaining 37 percent of 7(a) loan dollars went to businesses that were established, solely male-owned, and not located in economically distressed areas. See appendixes II and III for additional data on 7(a) loans, such as the total volume, percentage of lending provided by year and by state, and other borrower characteristics, including SBA’s loan- and lender-level Small Business Risk Portfolio Solutions score (predictive score) information. In the following figures, we present more detailed data on 7(a) loans to small businesses based on their status as a new business; gender of ownership; location relative to economically distressed areas; and minority ownership for fiscal years 2007 through 2016. New businesses. As shown in figure 2, the percentage of 7(a) loans that went to new businesses decreased from 36 percent in fiscal year 2007 to 23 percent in fiscal year 2011 before increasing to 35 percent by 2016. Gender. From fiscal years 2007 through 2016, the share of the total value of approved 7(a) loans by gender of owner remained fairly consistent (see fig. 3). An average of 70 percent of the total loan value went to male- owned businesses, and the remaining 30 percent went to businesses that were majority (more than 50 percent) or partially (50 percent or less) owned by women. Economically distressed areas. SBA did not provide data on whether 7(a) loans go to businesses located in economically distressed neighborhoods. However, we used data from the American Community Survey for 2011 through 2015, the most recent version available at the time of our analysis, along with zip code information provided by SBA to determine the average poverty rate by zip code (see fig. 4). From fiscal years 2007 through 2016, the proportion of the total value of 7(a) loans approved that went to borrowers in economically distressed areas remained between 23 percent and 26 percent. We defined distressed areas as zip codes where at least 20 percent of the households had incomes below the national poverty line. Minority/Nonminority status of borrower. From fiscal years 2007 through 2016, the proportion of the total value of 7(a) loans approved that went to minority borrowers decreased overall—from 43 percent to 30 percent—with the lowest share at 24 percent in fiscal year 2010 (see fig. 5). The share of approved loan dollars that went to nonminority borrowers varied, increasing to 69 percent in fiscal year 2010 before decreasing to 56 percent in fiscal year 2016. Notably, the share of the total value of loans approved that went to borrowers whose race/ethnicity was categorized as undetermined increased from 5 percent in fiscal year 2007 to 13 percent in fiscal year 2016. This increase does not fully account for the declined share for minority borrowers. However, according to SBA officials, borrowers voluntarily provide self-reported information on race and ethnicity and therefore the associated trend data should be viewed with caution. SBA relies on on-site reviews as its primary mechanism for evaluating lenders’ compliance with the credit elsewhere requirement. The reviews are performed by third-party contractors with SBA staff participation and additional oversight from SBA. According to SBA’s standard operating procedures, these reviews are generally conducted every 12 to 24 months for all 7(a) lenders with outstanding balances on the SBA- guaranteed portions of their loan portfolios of $10 million or more, although SBA may conduct on-site reviews of any SBA lender at any time as it considers necessary. In fiscal year 2016, SBA conducted 40 on-site reviews of 7(a) lenders, representing approximately 35 percent of SBA’s total outstanding 7(a) loan portfolio. As part of SBA’s on-site reviews, reviewers judgmentally selected a sample of approximately 30 to 40 loan files using a risk-based approach. These loan files accounted for approximately 6 percent to 19 percent of each lender’s total gross SBA dollars in fiscal year 2016. For each lender, approximately 70 percent to 90 percent of the loan files in the sample were reviewed to evaluate compliance with the credit elsewhere requirement. According to SBA’s contractors, loans that were selected for other reasons, such as issues related to liquidation, were not required to be reviewed for credit elsewhere compliance. SBA requires lenders to provide a narrative to support the credit elsewhere determination in the credit memorandum included in each loan file. SBA’s standard operating procedures state that lenders must substantiate that credit is not available elsewhere by (1) discussing the criteria that demonstrate an identifiable weakness in a borrower’s credit and (2) including the specific reasons why the borrower does not meet the lender’s conventional loan policy requirements. In keeping with SBA’s documentation requirement, third-party contractors and SBA staff who conduct on-site reviews are supposed to assess whether lenders have adequately documented the credit elsewhere criteria and provided specific reasons supporting the criteria in the credit memorandum. According to SBA’s contractors, adequate documentation of the credit elsewhere determination in the credit memorandum would include not just which of the criteria a borrower met but also a discussion of the basis or justification for the decision. For example, if a lender determined that a borrower needed a longer maturity, the lender should explain in the credit memorandum the reasons why a longer maturity was necessary. SBA’s contractors also told us that they carefully review a lender’s loan policies in preparation for on-site reviews and refer to a lender’s policies throughout the reviews. Reviewers do not attempt to verify the evidence given in support of the credit elsewhere reason beyond the information provided in the credit memorandum. Based on our review of fiscal year 2016 reports, on-site reviews can result in three levels of noncompliance response: Finding: This is the most severe result and is associated with a corrective action for the lender to remedy the issue. Observation: This is a deficiency recorded in the review’s summary but may not warrant a corrective action for the lender. Deficiency Noted: This is the lowest level of response. It is a deficiency noted as part of the review that is not included in the review’s summary and also may not warrant a corrective action. According to SBA officials, SBA’s policy has been that any noncompliance with SBA loan program requirements results in a finding. However, according to SBA officials and our review of the fiscal year 2016 on-site review reports, if a single instance of noncompliance was identified in fiscal year 2016, SBA generally would not issue a finding. Instead, SBA’s contractors said they would attempt to determine whether that instance was an inadvertent error, such as by examining additional loan files. Lenders that are subject to corrective actions are generally required to submit a response to SBA within 30 days to document how they have addressed or plan to address the identified issues. SBA subsequently asks for documentation to show that the lender has remedied the issue, and in some cases will conduct another review that usually includes an assessment of 5 to 10 additional loan files to determine whether the credit elsewhere reason has been adequately documented. According to SBA officials, SBA may also review lenders’ compliance with corrective actions from recent on-site reviews during targeted reviews (discussed below) and delegated authority renewal reviews (for lenders with delegated authority). In addition to on-site reviews, SBA also monitors lenders’ compliance with the credit elsewhere requirement through targeted reviews (performed on- or off-site). Targeted reviews of a specific process or issue may be conducted for a variety of reasons at SBA’s discretion, including assessing a lender’s compliance with the credit elsewhere requirement. In fiscal year 2016, SBA conducted 24 targeted reviews that included an examination of lenders’ compliance with the credit elsewhere documentation requirement. For these reviews, SBA examined loan files for 5 judgmentally selected loans that were provided to SBA electronically, as well as copies of the credit elsewhere reasoning (among other underwriting documentation) for 10 additional recently-approved loans. SBA also conducts periodic off-site reviews that use loan- and lender- level portfolio metrics to evaluate the risk level of lenders’ 7(a) portfolios. According to agency officials, SBA also began using off-site reviews to evaluate lenders’ compliance with the credit elsewhere requirement in fiscal year 2016. In that year, SBA conducted off-site reviews of 250 lenders and required these lenders to report the credit elsewhere justification for a sample of 10 loans per lender that were identified by SBA’s selection process. Lenders were not required to provide supporting documentation, and SBA did not follow up with lenders or review loan files to ensure the validity of the self-reported reasons. According to SBA, off-site reviews followed the same procedures in fiscal year 2017 as in 2016 and that the agency planned to use the same procedures for these reviews in the future. According to the agency, it also routinely evaluates and revises its review processes and procedures. In addition, SBA’s Loan Guaranty Processing Center and National Guaranty Purchase Center conduct Improper Payments Elimination and Recovery Act and quality control reviews at the time of loan approval and at the time of guaranty purchase, respectively. These reviews examine the credit elsewhere requirement, among other issues. Lastly, since 2014 SBA’s Office of Inspector General has also examined whether high-dollar or early-defaulted 7(a) loans were made in accordance with rules; regulations; policies; and procedures, including the credit elsewhere requirement. Our review of the on-site reviews conducted in fiscal year 2016 found that 17 of the 40 reviews—more than 40 percent—identified compliance issues with the credit elsewhere documentation requirement. Of those 17 reviews, 10 reviews resulted in a Finding (all with associated corrective 3 reviews resulted in an Observation (none with associated corrective actions or requirements), and 4 reviews resulted in a Deficiency Noted (one with an associated requirement). For all of the 17 on-site reviews that identified an instance of noncompliance, the issue was related to the lender’s documentation of the credit elsewhere criteria or justification. For example, one review found that the lender’s “regulatory practices demonstrate material noncompliance with SBA Loan Program requirements regarding documentation of the Credit Elsewhere Test.” Another review found that the lender “failed to demonstrate with adequate documentation that credit was not available elsewhere on reasonable terms and conditions.” For 2 of the 17 reviews, the issue was partly related to a discrepancy between the credit elsewhere justification used for some of the sample loan files and the lender’s own loan policy limits. With regard to SBA’s targeted reviews, 7 of 24 reviews (29 percent) conducted in fiscal year 2016 found a compliance issue with the credit elsewhere requirement. Of those 7 reviews, 6 reviews resulted in a Finding (all with associated corrective actions), 1 review resulted in an Observation (without an associated corrective no reviews resulted in a Deficiency Noted. For all of the 7 targeted reviews that identified a compliance issue, the issue was wholly related to the lender’s documentation of the credit elsewhere reason or justification. For example, 4 reviews found that for at least one loan reviewed, “the Lender failed to document justification that credit was unavailable elsewhere.” Another review found that for “three SBA Express loans and one Small Loan Advantage loan reported “other factors relating to the credit that in the lender’s opinion cannot be overcome except for the guaranty’ without specific identification of the factors.” Based on our review of on-site review reports and an interview with one reviewer, the key factors underlying lenders’ high rate of noncompliance with the credit elsewhere documentation requirement were lenders’ lack of proper internal controls and procedures and lack of awareness of the credit elsewhere documentation requirement. In fiscal year 2016, SBA’s corrective actions related to the credit elsewhere requirement required the lenders to establish or strengthen their policies; procedures; underwriting processes; or internal controls. In addition, contractors conducting the on- site reviews with whom we spoke stated that some lenders appeared to be unfamiliar with SBA’s standard operating procedures or were unclear on how to interpret them. For the 11 on-site reviews conducted in 2016 that included corrective actions, SBA generally required lenders to improve controls or procedures. For example, one lender was required to “correct its policy, modify its procedure, and amend its internal controls to ensure that its consideration and documentation of credit unavailable elsewhere identifies the specific fact(s) which are applicable to the specific loan and the determination is rendered and accurate for each individual SBA loan that it originates.” Another lender was required to “improve underwriting processes and controls to ensure that the borrower meets the [credit elsewhere] requirement” and to “document the loan file with the reasons for the determination.” Similarly, for the six targeted reviews in 2016 that included corrective actions, SBA issued a general requirement for the lender to “identify the causes for the Findings and implement corrective actions.” Based on our review of these targeted reviews, lenders generally remedied or intended to remedy the issue by amending their internal controls or procedures. For example, one lender stated that the “Credit Elsewhere test will be incorporated into the Credit Department process.” Another lender stated that it would “centralize all SBA underwriting and has developed an SBA addendum that will be utilized for all SBA-guaranteed loans.” Although some of SBA’s on-site reviews for fiscal year 2016 identified factors leading to noncompliance, they generally did not document reviewers’ assessment of lenders’ policies and practices for complying with the credit elsewhere documentation requirement. SBA’s standard operating procedures state that the on-site reviewers should determine whether or not lenders’ policies and practices adhere to the requirement, but they do not require them to document their assessment of these policies and practices. Only 4 of the 40 fiscal year 2016 review reports that we examined included such an assessment. As a result, although SBA required corrective actions by the lender to address deficiencies, there usually was no record of the underlying factors that resulted in the lender’s noncompliance. Federal internal control standards state that management should design control activities to achieve objectives and respond to risks, including appropriate documentation of transactions and internal control. Because SBA does not require reviewers to document their assessment of lenders’ policies and practices for complying with the credit elsewhere documentation requirement, the agency does not have good information to help explain why so many lenders are not in compliance. This hinders SBA’s ability to take informed and effective actions to improve lender compliance with the requirement and ensure that the program is reaching its intended population. SBA does not routinely collect information on the criteria lenders use in their credit elsewhere justifications. As previously discussed, lenders are required to maintain documentation of borrower eligibility (including the credit elsewhere justification) in each loan file for loans approved through lenders’ delegated authority. However, SBA cannot readily aggregate information on lenders’ credit elsewhere justifications for both delegated and nondelegated loans: For delegated loans, lenders are required to certify the loan’s credit elsewhere eligibility on E-Tran, SBA’s online portal for origination of delegated and nondelegated loans. However, lenders are only required to check a box to certify that the loan file contains the required credit elsewhere justification and are not required to submit any additional information, including which of the criteria was used to make the determination. According to SBA officials, delegated loans account for loans approved by approximately 70 percent of lenders. For nondelegated loans, lenders are required to submit credit elsewhere documentation to be reviewed by SBA’s Loan Guaranty Processing Center. For these loans, which comprise loans approved by the remaining 30 percent of lenders, SBA might maintain paper records of data on borrowers’ eligibility but does not compile such data electronically and thus cannot readily aggregate the data for analysis. Instead, SBA relies on on-site reviews or lender-reported information to review lenders’ credit elsewhere justifications and collects limited data from these reviews. For its on-site reviews, SBA does not collect sample data on lenders’ use of the credit elsewhere criteria. For its off-site reviews, SBA collected sample data on lenders’ use of the credit elsewhere criteria based on 250 such reviews conducted in fiscal year 2016. For these reviews, SBA asked lenders to self-report a short description of the credit elsewhere justifications used for an SBA-selected sample of 10 loans. However, as discussed earlier, SBA did not request or examine loan files as part of these off-site reviews and did not follow up with lenders or review loan files to ensure the validity of the self- reported reasons. One reason why SBA does not routinely collect complete information on lenders’ use of the credit elsewhere criteria is that SBA’s loan origination system, E-Tran, is not equipped to record or tabulate this information. In addition, according to an SBA official, on-site reviews do not collect data on the credit elsewhere criteria because the loans reviewed are judgmentally selected and would not accurately represent the larger population. Federal internal control standards state that management should use quality information to achieve the entity’s objectives. To do so, management should identify the information needed to achieve the objectives and address the risks, obtain relevant data from reliable internal and external sources in a timely manner, and process the obtained data into quality information. More robust information on lenders’ credit elsewhere justifications, including the credit elsewhere criteria, would allow SBA to evaluate patterns in lender practices related to the credit elsewhere requirement and, in turn, help the agency ensure compliance with the requirement. In this context, generalizable data, which can be collected through random sampling, or complete data through required reporting for every loan would allow SBA to better understand patterns in lender practices across the 7(a) program. Further, nongeneralizable data, which are available through SBA’s current off- and on-site review processes, would allow SBA to examine specific groups of lenders and could help SBA determine if it is necessary to collect additional data. SBA does not analyze the limited data it collects to help it monitor lenders’ compliance with the credit elsewhere requirement. According to agency officials, SBA has not performed lender-level analyses of the criteria lenders use for their credit elsewhere justifications. Additionally, SBA has not analyzed 7(a) lenders’ use of the “other factors” criterion— that is, factors not specified in the other criteria that, in the lender’s opinion, cannot be overcome except for the guarantee—for example, by collecting data on the frequency of its use or examining why lenders rely on it. While some 7(a) lenders told us they avoided relying on the “other factors” criterion because it was vague and open to interpretation, some lenders have used it when a borrower’s profile did not meet any of the other criteria. For example, one lender stated that this criterion was used for a borrower who was no longer a start-up but had experienced fluctuations in cash flow due to relocation or change in ownership. Another lender stated that the criterion was used more frequently during the 2007-2009 financial recession to extend financing to borrowers whose owners had experienced a home foreclosure but were otherwise sound. Federal internal control standards state that management should establish and operate monitoring activities to monitor the internal control system and evaluate the results. Analyzing data on lenders’ use of the credit elsewhere criteria as part of its monitoring procedures could help SBA determine whether there are patterns in lender practices related to the criteria that could predict lender noncompliance. For example, SBA could analyze lenders’ use of the criteria along with lender review results and other data on loan characteristics and performance to determine whether certain patterns indicate that a lender might be applying the requirement inconsistently. Additionally, such analysis could inform SBA’s selection of which lenders to review by improving its ability to identify lenders at risk of noncompliance with the credit elsewhere requirement. Better selection criteria for its lender reviews could, in turn, improve identification and remediation of such noncompliance, helping ensure that the 7(a) program serves its target population. Representatives at 8 of the 11 lenders that we contacted said they believed that SBA’s current credit elsewhere criteria are adequate in targeting small business borrowers who cannot obtain credit at reasonable terms. Representatives of these lenders also agreed that the criteria generally serve the types of small businesses that would otherwise have trouble obtaining conventional credit, such as new businesses or those with a shortage of collateral. One lender representative told us its most commonly used criterion related to the overall time in business because of the higher risk of failure. Another lender representative cited the lack of collateral as the most common criterion used. Additionally, representatives at an industry association told us that one of the most commonly used criteria was the one related to loan maturity and many small businesses seek 7(a) loans because they offer repayment terms of up to 10 years, compared to 1 to 3 years for conventional loans. Representatives of two other lenders suggested that the credit elsewhere criteria should not be overly prescriptive, which could limit lenders’ ability to make 7(a) loans to some businesses. For example, one representative said the credit elsewhere criteria should remain flexible because banks have different lending policies. In addition, representatives at three lenders indicated that they were hesitant to use the “other factors” criterion. One lender believed the requirement was open to interpretation and could be used inappropriately with lenders determining their own individual conventional loan policies. Another lender commented that the criterion was vague and rarely used by his institution, noting that SBA should provide some additional guidance on its use. Lenders consider multiple factors in determining whether to offer small businesses a conventional loan or a 7(a) loan, according to stakeholders with whom we spoke. For example, representatives at an industry association stated that a bank goes through several analyses to determine what loan product to offer the borrower. These representatives stated that the credit elsewhere requirement is embedded in the analysis a bank performs, such as whether the borrower qualifies for a loan and has a financial need for an SBA loan and whether the 7(a) program is right for that borrower. Representatives at two other lenders also stated that many small businesses have already been turned down for conventional loans before they seek a 7(a) loan. One representative noted that the “reasonable rates and terms” component of the 7(a) program was important as it allows lenders to look more broadly at a borrower’s needs. For instance, the representative explained, lenders can assess whether repayment terms are reasonable given a particular borrower’s situation and the resources the borrower will have to repay the loan. Economic conditions also affect lending policies, including whether borrowers qualify for a conventional loan, according to representatives at seven lenders with whom we spoke. For example, during the recent economic downturn, banks tightened their underwriting standards for small businesses and were less willing to lend without a government guarantee, according to one lender representative. SBA has issued revised primary operational guidance for the 7(a) program, effective January 1, 2018. As discussed previously, lenders are required to make a determination that the desired credit is not available to the applicant from nonfederal sources. Under the previous guidance, the lender had to determine that some or the entire loan was not available from nonfederal sources or the resources of the applicant business. However under the revised guidance, the scope of nonfederal sources a lender must review was further defined to include sources both related and unrelated to the applicant. The updated guidance states that lenders must consider: Nonfederal sources related to the applicant, including the liquidity of owners of 20 percent or more of the equity of the applicant, their spouses and minor children, and the applicant itself; or Nonfederal sources unrelated to the applicant, including conventional lenders or other sources of credit. Representatives of five lenders told us they have been determining how to interpret the new procedures with a few stating they would like additional guidance, including what information to retain in the file. Representatives of two lenders stated that there is some ambiguity in how to determine nonfederal resources and how to assess whether small business owners have too many available liquid resources to qualify for a 7(a) loan. One representative said that lenders can have different interpretations of what constitutes “available resources,” which is not specified in the new SOP. As a result, he said, there may be some confusion about how to assess family members of the borrower who have high net worths and whether the borrower should decline a family member contribution to qualify for an SBA loan. A representative of one lender stated that lenders will not know what SBA expects until loans are approved under the new procedures, default, and are then reviewed. Another lender’s representative suggested additional guidance on documentation, such as whether the bank must obtain a personal financial statement for each owner of the business. A SBA staff told us SBA has provided multiple training presentations to SBA staff, lenders, and trade associations on the statutory changes to the credit elsewhere requirements and standard operating procedure updates. These have included a presentation at a trade association conference, four monthly conference calls for SBA staff, and two conference calls for SBA lenders. SBA staff said SBA also plans to hold monthly training sessions with SBA field offices, quarterly training sessions with the industry, and at least four training sessions in 2018 at lender trade conferences. Additionally, a representative from an industry association told us it is providing industry training on SBA’s revised procedures, including the credit elsewhere liquidity requirement. SBA’s 7(a) loan program is required to serve creditworthy small business borrowers who cannot obtain credit through a conventional lender at reasonable terms, and SBA largely relies on lenders with delegated authority to make credit elsewhere determinations. However, although there is a high rate of lender noncompliance with the credit elsewhere documentation requirement, SBA does not require its reviewers to document their assessment of the policies and procedures lenders use to meet the requirement. Without better information from lender reviews on how lenders are implementing the requirement to document their credit elsewhere decisions, SBA may be limited in its ability to promote compliance with requirements and, in turn, use such information to help ensure that 7(a) loans are reaching their target population. Furthermore, SBA does not routinely collect or analyze information on the criteria used for credit elsewhere justifications to evaluate patterns in lender practices. SBA recently began collecting some information on lenders’ use of the criteria, but this information is limited, and SBA does not analyze the information that it does collect to better understand lenders’ practices. Without more robust information and analysis, SBA may be limited in its ability to understand how lenders are using the credit elsewhere criteria and whether 7(a) loans are reaching borrowers who cannot obtain credit from other sources at reasonable terms. We are making the following three recommendations to SBA. The Administrator of SBA should require reviewers to consistently document their assessments of a lender’s policies and practices. (Recommendation 1) The Administrator of SBA should use its on-site and off-site reviews to routinely collect information on lenders’ use of credit elsewhere criteria as part of its monitoring of lender practices related to the credit elsewhere requirement. (Recommendation 2) The Administrator of SBA should analyze information on lenders’ use of credit elsewhere criteria obtained from its reviews to identify lenders that may be at greater risk of noncompliance and to inform its selection of lenders for further review for credit elsewhere compliance. (Recommendation 3) We provided a draft of this report for review and comment. SBA’s written comments are reprinted in appendix IV. SBA generally agreed with the recommendations. SBA also provided additional comments on certain statements in the draft report, which are summarized below with our responses. SBA noted that the draft Highlights did not discuss how credit elsewhere is determined for nondelegated loans. We have not revised the Highlights in response to this comment because our review focused on delegated lenders. In the body of the report we note that approximately 70 percent of 7(a) loans are approved under delegated authority. We also refer to SBA’s nondelegated loans in the report for additional context. According to SBA, the statement on our draft Highlights did not fully reflect its monitoring of lender compliance. SBA identified a variety of reviews it uses in addition to on-site reviews by third party contractors, which we discuss in the body of the report. We have modified the Highlights to reflect these other reviews. Also in reference to the draft Highlights, SBA stated that it provides oversight on every on-site lender review and that an SBA employee is present as a subject-matter expert on every review. We revised the Highlights by adding that SBA provides oversight to the on-site reviews conducted by third-party contractors. In response to a statement in our draft report that SBA guarantees loans to small businesses for working capital and other general business purposes, SBA commented that working capital generally is not the primary purpose for SBA-guaranteed loans. We did not revise the statement because SBA’s SOP 50 10 5 (version J) specifies that SBA’s 7(a) loan proceeds may be used for permanent working capital and revolving working capital, among other things. In relation to a footnote in our report that mentions two lender reviews for which we did not receive documentation, SBA stated that on February 15, 2018, it provided documentation to us related to the reviews and that we had confirmed its receipt. However, the text in the footnote in question refers to two targeted lender reviews from 2016 that included corrective actions. The information SBA provided to us on February 15, 2018, was related to on-site reviews conducted in 2016. As a result, we did not revise the footnote. SBA’s letter also contained technical comments that we incorporated as appropriate. We are sending copies of this report to congressional committees, agencies, and other interested parties. In addition, this report will be available at no charge on our website at http://www.gao.gov. If you or your staff members have any questions about this report, please contact me at (202) 512-8678 or shearw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix V. This report discusses (1) 7(a) lending to selected categories of small business borrowers from fiscal years 2007 through 2016; (2) how the Small Business Administration (SBA) monitors lenders’ compliance with the credit elsewhere requirement; (3) the extent to which SBA evaluates trends in lender practices related to the credit elsewhere requirement; and (4) lenders’ views on the criteria used to determine eligibility for 7(a) loans and other issues related to the 7(a) program. For background on the 7(a) program and the credit elsewhere requirement, we reviewed the legislative history of the 7(a) program and our previous reports. We also interviewed officials from SBA’s Office of Credit Risk Management on guidance provided to 7(a) lenders. For background on constraints in the small business credit market, we reviewed recent academic literature on the characteristics of small businesses that historically have had more difficulty accessing credit. In addition, we reviewed recent studies published by the Federal Reserve Banks of Atlanta; Cleveland; Kansas City; and New York. To describe the population of borrowers served by the 7(a) program, we selected characteristics (such as gender, minority status, and percentage of new business) that we used in our 2007 report and that were the subject of the recent studies by Federal Reserve Banks. We obtained and analyzed SBA loan-level data to describe 7(a) loans and borrowers. Specifically, SBA provided us with 581,393 records from its administrative data systems, which contained information on all loans approved and disbursed in fiscal years 2007 through 2016. The SBA data included various types of information describing each loan, including the total gross approval amount; the amount guaranteed by SBA; the loan term; and the interest rate; delivery method; and status of the loan. The SBA data also included information on borrower characteristics: Age of business. Firms were classified as new (less than 2 years in operation) or existing. Gender. Firms were classified as 100 percent male-owned; 50 percent or greater women owned; 50 percent or less women-owned; or “unknown.” Information on gender was voluntarily provided by borrowers. Economically distressed area. We identified borrowers in economically distressed areas by matching borrower zip codes provided by SBA to those in the 2011 through 2015 American Community Survey. We defined distressed areas as zip codes where at least 20 percent of households had incomes below the national poverty line. In about 1 percent of the cases, we were unable to classify a lender because a zip code had changed or had insufficient population to report a poverty rate. We consider 1 percent of unmatched cases to be low by data reliability standards. Race/ethnicity. Borrowers were placed in one of nine categories of race/ethnicity, including an “unknown” category. We aggregated these to create minority, nonminority, and undetermined categories. The minority category included all borrowers who reported being a race/ethnicity other than white. The nonminority category included borrowers who reported being white. Information on race was voluntarily provided by borrowers. Industry. Firms were assigned a North American Industrial Classification code. These six-digit codes begin with a two-digit sector code that we used to draw more general conclusions about industries. Geographic information. The data provided the state where the borrower is located. In addition, we obtained information from SBA on loan- and lender-level Small Business Risk Portfolio Solution scores (predictive scores) provided by Dunn & Bradstreet and Fair Isaac Company, for loans approved in fiscal year 2016, the latest available. We were able to obtain predictive scores for approximately 81 percent of the loans for which SBA had provided other information. According to SBA, some loans may not have been disbursed at the time we obtained the predictive scores and, as a result, we do not have scores associated with these loans. We analyzed the information to determine the range of predictive scores and the range of average predictive scores by lender. To assess the reliability of loan-level data on borrower and loan characteristics and predictive scores we received from SBA, we interviewed agency officials knowledgeable about the data and reviewed related documentation. We also conducted electronic testing, including checks for outliers, missing data, and erroneous values. We determined that the data were sufficiently reliable for the purposes of describing the characteristics of borrowers who received 7(a) loans and the distribution of predictive scores. To assess how SBA monitors lenders’ compliance with the credit elsewhere requirement and criteria, we reviewed SBA’s standard operating procedures and other guidance on 7(a) program regulations and lender oversight. Specifically, we reviewed SOP 50 10 5 (versions I and J) on Lender and Development Company Loan Programs, SOP 50 53(A) on Lender Supervision and Enforcement, and SOP 51 00, On-Site Lender Reviews/Examinations, as well as information and policy notices related to the credit elsewhere requirement. Additionally, we interviewed representatives including those at SBA’s Office of Capital Access and Office of Credit Risk Management on lender oversight and lender review processes. We reviewed all the on-site lender review reports (40 reviews), including corrective actions or requirements related to the credit elsewhere requirement (documentation for 11 lenders), and targeted review reports that had credit elsewhere findings (7 reviews) that SBA conducted in fiscal year 2016. We also interviewed officials and reviewed recent reports from SBA’s Office of Inspector General. To assess the extent to which SBA evaluates trends in lender practices related to the credit elsewhere requirement, we interviewed SBA officials and reviewed documentation for SBA’s online portal for loan origination. We also incorporated information from interviews with a nongeneralizable, nonrepresentative sample of 7(a) lenders, which we discuss below. To obtain lenders’ views on the criteria used to determine eligibility for 7(a) loans and other program-related issues, we interviewed SBA staff including from the Office of Capital Access, and representatives of the National Association of Government Guaranteed Lenders; American Bankers Association; Independent Community Bankers Association; and National Federation of Independent Businesses. We also interviewed 11 banks (one bank provided written responses) in order to obtain the lender perspective of credit elsewhere. Nine of the banks were selected by us using a random process that concentrated on larger lenders. These nine lenders selected by us represent about 13 percent of the loans approved and 16 percent of the dollars approved in 2016. In addition, we interviewed two additional banks that represented an industry group – one larger bank and one small bank. Although we partially selected at random, the lenders we interviewed should not be considered generalizable because of the small number. We conducted this performance audit from August 2017 to June 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. In this appendix, we provide information on the total amount and number of approved 7(a) loans and the top eight industry sectors receiving 7(a) loans. Data are also presented on fiscal year 2016 loan volume by state and per capita. As shown in figure 6 below, the total amount of approved 7(a) loans decreased during the period associated with the Great Recession (2007 through 2009). From fiscal year 2009 on, the total amount of approved 7(a) loans increased until a decline in fiscal year 2012. During this timeframe, the American Recovery and Reinvestment Act of 2009 and the Small Business Jobs Act of 2010 provided fee relief and higher guaranties. The Small Business Jobs Act of 2010 also provided a temporary increase in Small Business Administration (SBA) Express loan limits to $1 million (instead of $350,000). These programs have since expired. 7(a) Loans by North American Industry Classification System (NAICS) code. Table 1 shows the largest eight industrial sectors by proportion of the total amount of 7(a) loans approved, using the NAICS code. The combined share of the top eight sectors declined slightly from 85 percent to 80 percent of the total lending from fiscal years 2007 through 2016, with an average of 82 percent. During this period, the Accommodation and Food Services sector had the largest average share of total loan amount at 17 percent, followed by the Retail Trade sector at 15 percent. Approved loan amount and per capita dollars by state. As shown in figure 7, California; Texas; Florida; Georgia; and New York received the highest total of approved loan dollars in fiscal year 2016. The average approval amount across all loans was $380,619. Georgia and Arkansas had the largest average approval amount in 2016. Also, during this period, Utah; Colorado; Georgia; California; and Washington received the highest per capita approved loan dollars. In fiscal-year 2016, creditworthiness varied widely among 7(a) program borrowers. We analyzed creditworthiness using the Small Business Administration’s (SBA) Small Business Risk Portfolio Solution score (predictive score), which ranges from 70 to 300, with 300 indicating the least risky loan. According to SBA, loans with scores above 180 are considered “lower risk,” scores between 140 and 179 are considered “moderate risk,” and scores 139 and lower are considered “higher risk.” There did not appear be differences in score based on the gender of the borrower or the age of the business. While SBA relies on the Predictive Score data to identify lenders that may pose excessive risk to the SBA 7(a) portfolio, the data also provide potential insights related to lender implementation of the credit elsewhere requirement. Variation. We found that some 7(a) borrowers were much more creditworthy than others. In 2016, the only year for which we obtained data, the predictive score at origination varied widely among borrowers. In 2016, the scores of borrowers ranged from a low of 91 to a high of 246. However, most scores were between 171 and 203, and the median score was 188. Race/ethnicity. We found that there were slight differences in creditworthiness by race/ethnicity, with median scores ranging from 180 to 189 depending on the category. Specifically, loans to African Americans in 2016 had a median score of 180, and loans to Hispanics had a median score of 183. In contrast, loans to whites had a median score of 188, and loans to Asian and Pacific Islanders had a median score of 189. Lender size. We found that lenders with larger numbers of SBA loans tended to have slightly more creditworthy borrowers. The top 5 percent of lenders had a median average score of 187, whereas the bottom 75 percent of lenders had a median average score of 182.5. Among the top 5 percent of lenders (with 374 loans per lender on average, collectively representing about 70 percent of the loans approved), the average score ranged from 171 to 195. Among all lenders, the average score ranged from 116 to 233. However, because many lenders only approved one or two loans in 2016, the average may reflect very few borrowers for that lender, making it difficult to tell whether the scores reflect a real difference between lenders. In addition to the contact named above, Harry Medina (Assistant Director); Janet Fong (Analyst in Charge); Benjamin A. Bolitzer; Gita DeVaney; David S. Dornisch; Amanda D. Gallear (intern); Marc W. Molino; Jennifer W. Schwartz; and Tyler L. Spunaugle made key contributions to this report.

 

 Report 2:
Concerned that the federal government was more focused on program activities and processes than the results to be achieved, Congress passed the Government Performance and Results Act of 1993 (GPRA). GPRA sought to focus federal agencies on performance by requiring agencies to develop long-term and annual goals, and measure and report on progress towards those goals annually. Based on our analyses of the act’s implementation, we concluded in March 2004 that GPRA’s requirements had laid a solid foundation for results-oriented management. At that time, we found that performance planning and measurement had slowly yet increasingly become a part of agencies’ cultures. For example, managers reported having significantly more performance measures in 2003 than in 1997, when GPRA took effect government-wide. However, the benefit of collecting performance information is fully realized only when that information is actually used by managers to make decisions aimed at improving results. Although our 2003 survey found greater reported availability of performance information than in 1997, it also showed managers’ use of that information for various management activities generally had remained unchanged. Based on those results, and in response to a request from Congress, in September 2005, we developed a framework intended to help agencies better incorporate performance information into their decision making. As shown in figure 1, we identified five leading practices that can promote the use of performance information for policy and program decisions; and four ways agency managers can use performance information to make program decisions aimed at improving results. Our September 2005 report also highlighted examples of how agencies had used performance information to improve results. For example, we described how the Department of Transportation’s National Highway Traffic Safety Administration used performance information to identify, develop, and share effective strategies that increased national safety belt usage—which can decrease injuries and fatalities from traffic accidents— from 11 percent in 1985 to 80 percent in 2004. Subsequently, the GPRA Modernization Act of 2010 (GPRAMA) was enacted, which significantly expanded and enhanced the statutory framework for federal performance management. The Senate Committee on Homeland Security and Governmental Affairs report accompanying the bill that would become GPRAMA stated that agencies were not consistently using performance information to improve their management and results. The report cited the results of our 2007 survey of federal managers. That survey continued to show little change in managers’ use of performance information. The report further stated that provisions in GPRAMA are intended to address those findings and increase the use of performance information to improve performance and results. For example, GPRAMA requires certain agencies to designate a subset of their respective goals as their highest priorities—known as agency priority goals—and to measure and assess progress toward those goals at least quarterly through data-driven reviews. Our recent work and surveys suggest that data-driven reviews are having their intended effect. For example, in July 2015, we found that agencies reported that their reviews had positive effects on progress toward agency goals and efforts to improve the efficiency of operations, among other things. In addition, for those managers who were familiar with their agencies’ data-driven reviews, our 2013 and 2017 surveys showed that the more managers viewed their programs as being subject to a review, the more likely they were to report their agencies’ reviews were driving results and conducted in line with our leading practices. Recognizing the important role these reviews were playing in improving data-driven decision making, our management agenda for the presidential and congressional transition in 2017 included a key action to expand the use of data-driven reviews beyond agency priority goals to other agency goals. More broadly, our recent surveys of federal managers have continued to show that reported government-wide uses of performance information generally have not changed or in some cases have declined. As we found in September 2017, and as illustrated in figure 2, the 2017 update to our index suggests that government-wide use of performance information did not improve between 2013 and 2017. In addition, it is statistically significantly lower relative to our 2007 survey, when we created the index. Moreover, in looking at the government-wide results on the 11 individual survey questions that comprise the index, we found few statistically significant changes in 2017 when compared to (1) our 2013 survey or (2) the year each question was first introduced. For example, in comparing 2013 and 2017 results, two questions had results that were statistically significantly different: The percentage of managers who reported that employees who report to them pay attention to their agency’s use of performance information was statistically significantly higher (from 40 to 46 percent). The percentage of managers who reported using performance information to adopt new program approaches or change work processes was statistically significantly lower (from 54 to 47 percent). As we stated in our September 2017 report, the decline on the latter question was of particular concern as agencies were developing plans to improve their efficiency, effectiveness, and accountability, as called for by an April 2017 memorandum from OMB. In early 2017, the administration announced several efforts intended to improve government performance. OMB issued several memorandums detailing the administration’s plans to improve government performance by reorganizing the government, reducing the federal workforce, and reducing federal agency burden. As part of the reorganization efforts, OMB and agencies were to develop government-wide and agency reform plans, respectively, designed to leverage various GPRAMA provisions. For instance, the April 2017 memorandum mentioned above stated that OMB intends to monitor implementation of the reforms using, among other things, agency priority goals. While many agency-specific organizational improvements were included in the President’s fiscal year 2019 budget, released in February 2018, OMB published additional government-wide and agency reform proposals in June 2018. The President’s Management Agenda (PMA), released in March 2018, outlines a long-term vision for modernizing federal operations and improving the ability of agencies to achieve outcomes. To address the issues outlined in the PMA, the administration established a number of cross-agency priority (CAP) goals. CAP goals, required by GPRAMA, are to address issues in a limited number of policy areas requiring action across multiple agencies, or management improvements that are needed across the government. The PMA highlights several root causes for the challenges the federal government faces. Among them is that agencies do not consistently apply data-driven decision-making practices. The PMA states that smarter use of data and evidence is needed to orient decisions and accountability around service and results. To that end, in March 2018, the administration established the Leveraging Data as a Strategic Asset CAP goal to improve the use of data in decision making to increase the federal government’s effectiveness. Over the past 25 years, various organizations, roles, and responsibilities have been created by executive action or in law to provide leadership in federal performance management. At individual agencies and across the federal government, these organizations and officials have key responsibilities for improving performance, as outlined below. OMB: At least every four years, OMB is to coordinate with other agencies to develop CAP goals—such as the one described earlier on leveraging data as an asset—to improve the performance and management of the federal government. OMB is also required to coordinate with agencies to develop annual federal government performance plans to define, among other things, the level of performance to be achieved toward the CAP goals. Following GPRAMA’s enactment, OMB issued guidance for initial implementation, as required by the act, and continues to provide updated guidance in its annual Circular No. A-11, additional memorandums, and other means. Chief Operating Officer (COO): The deputy agency head, or equivalent, is designated as the COO, with overall responsibility for improving agency management and performance through, among other things, the use of performance information. President’s Management Council (PMC): The PMC is comprised of OMB’s Deputy Director for Management and the COOs of major departments and agencies, among other individuals. Its responsibilities include improving overall executive branch management and implementing the PMA. Performance Improvement Officer (PIO): Agency heads designate a senior executive as the PIO, who reports directly to the COO. The PIO is responsible for assisting the head of the agency and COO to ensure that agency goals are achieved through, among other things, the use of performance information. Performance Improvement Council (PIC): The PIC is charged with assisting OMB to improve the performance of the federal government. It is chaired by the Deputy Director for Management at OMB and includes PIOs from each of the 24 Chief Financial Officers Act agencies, as well as other PIOs and individuals designated by the chair. Among its responsibilities, the PIC is to work to resolve government-wide or cross-cutting performance issues, and facilitate the exchange among agencies of practices that have led to performance improvements. Previously, the General Service Administration’s (GSA) Office of Executive Councils provided analytical, management, and administrative support for the PIC, the PMC, and other government-wide management councils. In January 2018, the office was abolished and its functions, staff, and authorities, along with those of the Unified Shared Services Management Office, were reallocated to GSA’s newly created Shared Solutions and Performance Improvement Office. As at the government-wide level—where, as described earlier, the use of performance information did not change from 2013 to 2017—managers’ reported use of performance information at most agencies also did not improve since 2013 (illustrated in figure 3). At the agency level, 3 of the 24 agencies had statistically significant changes in their index scores—1 increase (National Science Foundation) and 2 decreases (Social Security Administration and the Office of Personnel Management). Also, in 2017, 6 agencies had results that were statistically significantly different—4 higher and 2 lower—than the government-wide average (see sidebar). Throughout the report, we highlight two different types of statistically significant results—changes from our last survey in 2013 and differences from the 2017 government-wide average. The former indicates when an agency’s reported use of performance information or leading practices has measurably improved or declined. The latter indicates when it is statistically significantly higher or lower than the rest of government. These results suggest agencies have taken actions that led to improvements in their use of performance information. For example, when a result is a statistically significant increase since 2013, as with the National Science Foundation index score in 2017, this suggests that the agency has adopted practices that led to a measurable increase in the use of performance information by managers. When a result is statistically significantly higher than the government-wide average, like GSA’s 2017 index score, this suggests that the agency’s use of performance information is among the highest results when compared to the rest of government. These agencies could also have insights into practices that led to relatively high levels of performance information use. Finally, when a result is a statistically significant decrease since 2013, as with the Social Security Administration’s index score in 2017, or statistically significantly lower than the government-wide average, like the Department of Homeland Security’s 2017 index score, this suggests the agencies face challenges that are hampering their ability to use performance information. Appendix III provides each agency’s index scores from 2007, 2013, and 2017 to show changes between survey years. When we disaggregated the index and analyzed responses from the 11 questions that comprise the index—which could help pinpoint particular actions that improved the use of performance information—we similarly found relatively few changes in agencies’ recent results. Specifically, we identified 16 instances where agency responses on individual questions were statistically significantly different from 2013 to 2017—10 increases and 6 decreases. This represents about 6 percent of the total possible responses to the 11 survey questions from each of the agencies. In addition, we found 12 instances where an agency’s result on a question was statistically significantly higher (11) or lower (1) than the government-wide average in 2017. For example, the percentage of Social Security Administration (SSA) managers reporting that their peers use performance information to share effective approaches was statistically significantly higher than the government-wide average. Although SSA’s index score had a statistically significant decline in 2017 compared to 2013, the agency’s index score remains relatively high, as it has in prior years. The scope of our work has not allowed us to determine definitively what factors caused the decline in SSA’s index score and whether the decline is likely to continue, although its result on this particular question may indicate a continued strength. Each agency’s results on the 11 questions that comprise the index are presented in appendix I. The agencies’ respective statistically significant results are identified in figure 4. While some agencies had statistically significant improvements on individual questions, and could point to actions that led to improvements in their use of performance information, these improvements should be considered in relation to the range of agency results and the government- wide average. In figure 4, there are five agencies with statistically significant increases on responses to individual questions, where those results were not statistically significantly higher than the government-wide average (see arrows without plus signs for the Departments of Agriculture, Defense, and Justice; the Environmental Protection Agency; and the National Science Foundation). While these represent improvements, they should be considered in relation to the range of agency results and the government-wide average (provided in detail in the agency summaries in appendix I). For example, in 2017, the percentage of managers at the Department of Agriculture who reported that upper management use performance information to inform decisions about program changes was statistically significantly higher than in 2013. However, the department’s 2017 result (37 percent) was relatively lower when compared to the maximum agency result on that question (60 percent). Appendix I presents the results on the index and the 11 questions that comprise it for each of the 24 agencies. When we compared government-wide and agency-level results on selected survey questions that reflect practices that promote the use of performance information, we found that results between 2013 and 2017 generally remained unchanged. As described earlier, there are 10 survey questions that both reflect the five leading practices identified in our past work and had statistically significant associations with higher index scores. As shown in figure 5, government-wide results on 2 of the 10 questions were statistically significantly different, both increases, from 2013 to 2017. Despite these two increases, the overall results suggest these practices are not widely followed government-wide. On most of the 10 questions, only about half (or fewer) of the managers reported their agencies were following them to a “great” or “very great” extent. When we analyzed agency-level responses to these 10 questions, we also found relatively few changes in recent results. Specifically, our analysis found 20 instances—16 increases and 4 decreases—where agencies’ responses on individual questions were statistically significantly different from 2013 to 2017. This represents about 8 percent of the total possible responses to the 10 survey questions from each of the agencies. In addition, we found 10 instances where an agency’s result on a question was statistically significantly higher (8) or lower (2) than the government-wide average in 2017. Each agency’s results on these 10 questions are presented in appendix I, and the statistically significant results are identified in figure 6. Those agencies with results on individual questions that are either statistically significantly higher than 2013, higher than the 2017 government-wide average, or both may have taken actions in line with our leading practices for promoting the use of performance information. For example, the National Science Foundation had both types of statistically significant results on a question about having sufficient information on the validity of their performance data. Here, the agency’s result increased 27 percentage points from 2013 to 2017. While the scope of our review does not allow us to definitively determine the reasons for the National Science Foundation’s higher results, they suggest the agency has taken recent actions that greatly improved the availability and accessibility of information on the validity of performance data. In both 2013 and 2017, our analyses found this particular question to be the strongest predictor of higher performance information use when we tested for associations between the questions that reflect leading practices and our index. Our 2017 survey results show that managers who reported their programs were subject to data-driven reviews also were more likely to report using performance information in decision making to a greater extent (see figure 7). For the 35 percent of managers who reported being familiar with data-driven reviews, those who reported their programs had been subject to data-driven reviews to a “great” or “very great” extent had index scores that were statistically significantly higher than those whose programs were subject to these reviews to a lesser extent. Similarly, we found that being subject to data-driven reviews to a greater extent was also related to greater reporting of agencies following practices that can promote the use of performance information. As figure 8 shows, managers who reported their programs were subject to these reviews to a “great” or “very great” extent more frequently reported that their agencies followed the five leading practices that promote the use of performance information, as measured by the 10 related survey questions associated with higher scores on the index. For example, of the estimated 48 percent of managers who reported their programs were subject to data-driven reviews to a “great” or “very great” extent, 72 percent also reported that managers at their level (peers) effectively communicate performance information on a routine basis to a “great” or “very great” extent. Conversely, for the 24 percent of managers who reported their programs were subject to data-driven reviews to a “small” or “no” extent, only 30 percent reported that managers at their level do this to a “great” or “very great” extent. Our past work has found that the Executive Branch has taken steps to improve the use of performance information in decision making by senior leaders at federal agencies. However, our survey results indicate those steps have not led to similar improvements in use by managers at lower levels. Through its guidance to implement GPRAMA, OMB developed a framework for performance management in the federal government that involves agencies setting goals and priorities, measuring performance, and regularly reviewing and reporting on progress. This includes expectations for how agency senior leaders should use performance information to assess progress towards achieving agency priority goals through data-driven reviews, and strategic objectives through strategic reviews. For example, GPRAMA requires, and OMB’s guidance reinforces, that data-driven reviews should involve the agency head, Chief Operating Officer, Performance Improvement Officer, and other senior officials responsible for leading efforts to achieve each goal. OMB’s guidance also identifies ways in which agency leaders should use the results of those reviews to inform various decision-making activities, such as revising strategies, formulating budgets, and managing risks. Our past work also found that agencies made progress in implementing these reviews and using performance information. In July 2015, we found that agencies generally were conducting their data-driven reviews in line with GPRAMA requirements and our related leading practices, including that agency leaders used the reviews to drive performance improvement. In addition, in September 2017, we reported on selected agencies’ experiences in implementing strategic reviews and found that the reviews helped direct leadership attention to progress on strategic objectives. Despite those findings, our survey results continue to show that the reported use of performance information by federal managers has generally not improved, and actually declined at some agencies. This could be because of the two different groups of agency officials covered by our work. GPRAMA’s requirements, and the federal performance management framework established by OMB’s guidance, apply at the agency-wide level and generally involve senior leaders. Our past work reviewing implementation of the act therefore focused on improvements in the use of performance information by senior leaders at the agency- wide level. In contrast, our surveys covered random samples of mid- and upper-level managers within those agencies, including at lower organizational levels such as component agencies. Their responses indicate that the use of performance information more broadly within agencies—at lower organizational levels—generally has not improved over time. The exception to this was managers whose programs were subject to the data-driven reviews required by GPRAMA. As described above, those managers were more likely to report greater use of performance information in their agencies. This reinforces the value of the processes and practices put in place by GPRAMA. Our survey results suggest that limited actions have been taken to diffuse processes and practices related to the use of performance information to lower levels within federal agencies, where mid-level and senior managers make decisions about managing programs and operations. Although OMB staff agreed that diffusing processes and practices to lower levels could lead to improved use of performance information, they told us they have not directed agencies to do so for a few reasons. First, OMB staff expressed concerns about potentially imposing a “one-size-fits- all” approach on agencies. They stated that agencies are best positioned to improve their managers’ use of performance information, given their individual and unique missions and cultures, and the environments in which they operate. We agree that it makes sense for agencies to be able to tailor their approaches for those reasons. OMB’s existing guidance provides an overarching framework that recognizes the need for flexibility and for agencies to tailor their approaches. Moreover, given the long- standing and cross-cutting nature of this challenge, a government-wide approach also would provide a consistent focus on improving the use of performance information more extensively within agencies. OMB staff also told us that they believed it would go beyond their mandate to direct agencies to extend GPRAMA requirements to lower levels. GPRAMA requires OMB to provide guidance to agencies to implement its requirements, which only apply at the agency-wide level. As noted earlier, however, GPRAMA also requires OMB to develop cross- agency priority (CAP) goals to improve the performance and management of the federal government. The President’s Management Agenda established a CAP goal to leverage data as a strategic asset, in part, to improve the use of data for decision making and accountability throughout the federal government. This new CAP goal presents an opportunity for OMB and agencies to identify actions to expand the use of performance information in decision making throughout agencies. As of June 2018, the action plan for implementing the Leveraging Data as a Strategic Asset CAP goal is limited. According to the President’s Management Agenda and initial CAP goal action plan, the goal primarily focuses on developing and implementing a long-term, enterprise-wide federal data strategy to better govern and leverage the federal government’s data. It is through this strategy that, among other things, the administration intends to improve the use of data for decision making and accountability. However, the strategy is under development and not expected to be released until January 2019, with a related plan to implement it expected in April 2019. The existing action plan, released in March 2018 and updated in June 2018, does not yet include specific steps needed to improve the use of data—including performance information—more extensively within agencies. According to the action plan for the goal, potential actions currently under consideration focus on establishing agency “learning agendas” that prioritize the development and use of data and other evidence for decision-making; building agency capacity to use data and other evidence; and improving the timeliness of performance information and other data, and making that information available to decision makers and the public. Although developing learning agendas and building capacity could help improve the use of performance information in agencies, improving availability of data may be less effective. For example, as our past survey results have shown, increasing the availability of performance information has not resulted in corresponding increases in its use in decision making. We recognize that the CAP goal was created in March 2018. Nonetheless, it is important that OMB and its fellow goal leaders develop the action plan and related federal data strategy consistent with all key requirements to better ensure successful implementation. The action plan does not yet include complete information related to the following GPRAMA requirements: performance goals that define the level of performance to be achieved each year for the CAP goal; the various federal agencies, organizations, programs, and other activities that contribute to the CAP goal; performance measures to assess overall progress towards the goal as well as the progress of each agency, program, and other activity contributing to the goal; and clearly defined quarterly targets. Consistent with GPRAMA, Standards for Internal Control in the Federal Government identifies information that agencies are required to include in their plans to help ensure they achieve their goals. The standards state that objectives—such as improving the use of data in decision making— should be clearly defined to enable the identification of risks. Objectives are to be defined in specific terms so they can be understood at all levels of the entity—in this case, government-wide as well as within individual agencies. This involves defining what is to be achieved, who is to achieve it, how it will be achieved, and the time frames for achievement. Ensuring that future updates to the new CAP goal’s action plan includes all required elements is particularly important, as our previous work has found that some past CAP goal teams did not meet all planning and reporting requirements. For example, in May 2016 we found that most of the CAP goal teams we reviewed had not established targets for all performance measures they were tracking. This limited the transparency of their efforts and the ability to track progress toward established goals. We recommended that OMB, working with the Performance Information Council (PIC), report on actions that CAP goal teams are taking, or plan to take, to develop such targets and performance measures. OMB staff generally agreed and, in July 2017, told us they were working, where possible, to assist the development of measures for CAP goals. However, the recommendation has not been addressed and OMB staff said the next opportunity to address it would be when the administration established new CAP goals (which took place in March 2018). Following the initial release of the new CAP goals, CAP goal teams are to more fully develop the related action plans through quarterly updates. Given the ongoing importance of meeting these planning and reporting requirements, we will continue to monitor the status of actions to address this recommendation as implementation of the new CAP goals proceeds. While the PIC, which is chaired by OMB, has contributed to efforts to enhance the use of performance information, our survey results identify additional opportunities to further those efforts. The PIC’s past efforts have included hosting various working groups and learning events for agency officials to provide performance management guidance, and developing resources with relevant practices. For example, the PIC created a working group focused on agency performance reviews, which was used to share recommendations for how agencies can implement reviews, along with a guide with practices for effectively implementing strategic reviews. In January 2018, staff supporting the PIC joined with staff from another GSA office to create a new group called Fed2Fed Solutions. This group consults with agencies and provides tailored support, such as data analysis and performance management training for agency officials, to help them address specific challenges related to organizational transformation, data-driven decision making, and other management improvement efforts. Our survey results identify useful information related to potential promising practices and challenges that OMB and the PIC could use to inform efforts to enhance the use of performance information more extensively within agencies (e.g., at lower levels). As was previously described, the PIC has responsibilities to (1) facilitate the exchange among agencies of proven practices, and (2) work to resolve government- wide or cross-cutting performance issues, such as challenges. Our analyses of 2017 survey results identified instances where agencies may have found effective ways to enhance the use of performance information by agency leaders and managers in decision making, as well as instances where agencies (and their managers) face challenges in doing so. Specifically, based on analyses of our survey responses, we identified 14 agencies that may have insights into specific practices that led to recent improvements in managers’ use of performance information, or ways that they maintain relatively high levels of use by their managers when compared to the rest of the government. Figure 9 summarizes the agencies identified earlier in the report that had statistically significant increases, or results higher than the government-wide average, on our index or individual survey questions. As the figure shows, several agencies had statistically significant results across all three sets of analyses and therefore may have greater insights to offer: the General Services Administration, National Aeronautics and Space Administration, and the National Science Foundation. In addition, our analyses identified nine agencies where results suggest managers face challenges that have hampered their ability to use performance information. Figure 10 summarizes the agencies identified earlier in the report that had statistically significant decreases, or results lower than the government-wide average, on our index or individual survey questions. As the figure shows, the Office of Personnel Management had statistically significant decreases in all three sets of analyses. Four agencies—the Departments of the Treasury and Veterans Affairs, the Nuclear Regulatory Commission, and the Social Security Administration—were common to both of the figures above. That is, they had results that indicate they may have insights on some aspects of using performance information and face challenges in other aspects. As was mentioned earlier, to provide proper context, these results should be considered in relation to the range of agency results and the government- wide average (provided in detail in the agency summaries in appendix I). Given the prioritization of other activities, such as the recent creation of the Fed2Fed Solutions program, the PIC has not yet undertaken a systematic approach that could improve the use of performance information by managers at lower levels within agencies. Such an approach would involve identifying and sharing practices that have led to improved use, as well as identifying common or cross-cutting challenges that have hampered such use. The results of our analyses could help the PIC do so, and in a more targeted manner. By identifying and sharing proven practices, the PIC could further ensure that agency leaders and managers are aware of effective or proven ways they can use performance information to inform their decisions across the spectrum of activities they manage within their agencies. Those proven practices also may help agency leaders and managers resolve any identified challenges. Furthermore, in September 2017, we found that, for the estimated 35 percent of managers who reported familiarity with data-driven reviews, the more they viewed their programs being subject to a review, the more likely they were to report the reviews were driving results and were conducted in line with our leading practices for using performance information. Despite the reported benefits of and results achieved through data-driven reviews, they were not necessarily widespread. As noted above, GPRAMA requires agencies to conduct such reviews for agency priority goals, which represent a small subset of goals, and they are required at the departmental level. These reasons may explain why most managers reported they were not familiar with the reviews. As a result, we recommended that OMB should work with the PIC to identify and share among agencies practices for expanding the use of data-driven reviews. OMB staff agreed with our recommendation but have yet to address it. In June 2018, OMB updated its annual guidance to agencies to explicitly encourage them to expand data-driven reviews to include other goals, priorities, and management areas as applicable to improve organizational performance. However, as of June 2018, OMB and the PIC have yet to take any steps to identify and share practices for expanding the use of these reviews in line with our recommendation. Given the additional analyses we conducted for this report—which show that being subject to data-driven reviews is related to greater reported use of performance information and leading practices that promote such use—we continue to believe these further actions would help agencies implement these reviews more extensively. We reiterate the importance of the September 2017 recommendation and will continue to monitor OMB’s progress to address it. For more than 20 years, our work has highlighted weaknesses in the use of performance information in federal decision making. While the Executive Branch has taken some actions in recent years, such as establishing a framework for performance management across the federal government, our survey results underscore that more needs to be done to improve the use of performance information more extensively within agencies and government-wide. The President’s Management Agenda and its related CAP goal to leverage data as a strategic asset present an opportunity to do so, as it aims to improve data-driven decision making. As OMB and its fellow goal leaders more fully develop the action plan for achieving this goal, providing additional details for its plans to improve data-driven decision making would help provide assurance that it can be achieved. As part of those initiatives, our survey results could provide a useful guide for targeting efforts. Officials at each agency could use these results to identify areas for additional analysis and potential actions that could help improve the use of performance information across the agency and at lower levels. Similarly, OMB and the PIC could use the results to identify broader issues in need of government-wide attention. It will also be important, however, for OMB and the PIC to go beyond this analysis and work with agencies to identify and share proven practices for increasing the use of performance information at lower levels within agencies, as well as challenges that may be hampering agencies’ ability to do so. We are making the following two recommendations to OMB: The Director of OMB should direct the leaders of the Leveraging Data as a Strategic Asset CAP Goal to ensure future updates to the action plan, and the resulting federal data strategy, provide additional details on improving the use of data, including performance information, more extensively within federal agencies. The action plan should identify performance goals; contributing agencies, organizations, programs, and other activities; those responsible for leading implementation within these contributors; planned actions; time frames; and means to assess progress. (Recommendation 1) The Director of OMB, in coordination with the PIC, should prioritize efforts to identify and share among agencies proven practices for increasing, and challenges that hamper, the use of performance information in decision making more extensively within agencies. At a minimum, this effort should involve the agencies that our survey suggests may offer such insights. (Recommendation 2) We provided a draft of this report to the Director of the Office of Management and Budget for review and comment. We also provided a draft of the report to the heads of each of the 24 federal agencies covered by our survey. OMB had no comments, and informed us that it would assess our recommendations and consider how best to respond. We are sending copies of this report to congressional requesters, the Director of the Office of Management and Budget, the heads of each of the 24 agencies, and other interested parties. This report will also be available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-6806 or mcneilt@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report. Key contributors to this report are listed in appendix IV. (USDA) (Goverment-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Commerce) (Goverment-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Goverment-wide) (DOD) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Education) (Goverment-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (Energy) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (HHS) (Goverment-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DHS) (Goverment-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (HUD) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Interior) (Government-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DOJ) (Government-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DOL) (Government-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (State) (Government-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (DOT) (Government-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Treasury) (Government-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (VA) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Goverment-wide) (USAID) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (EPA) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (GSA) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (NASA) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (NSF) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (NRC) (Government-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (OPM) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (SBA) (Government-wide) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” (Government-wide) (SSA) 10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great” This report responds to a request that we analyze agency-level results from our 2017 survey of federal managers at the 24 agencies covered by the Chief Financial Officers (CFO) Act of 1990, as amended, to determine the extent agencies are using performance information. This report assesses the extent to which: 1. the reported use of performance information and related leading practices at 24 agencies has changed compared to our prior survey in 2013; 2. being subject to data-driven reviews related to managers’ reported use of performance information and leading practices; and 3. the Executive Branch has taken actions to enhance agencies’ use of performance information in various decision-making activities. From November 2016 through March 2017, we administered our online survey to a stratified random sample of 4,395 individuals from a population of 153,779 mid- and upper-level civilian managers and supervisors at the 24 CFO Act agencies. The management levels covered general schedule (GS) or equivalent schedules at levels comparable to GS-13 through GS-15, and career Senior Executive Service (SES) or equivalent. We obtained the sample from the Office of Personnel Management’s Enterprise Human Resources Integration database as of September 30, 2015—the most recent fiscal year data available at the time. The sample was stratified by agency and whether the manager or supervisor was a member of the SES. To help determine the reliability and accuracy of the database elements used to draw our sample of federal managers for the 2017 survey, we checked the data for reasonableness and the presence of any obvious or potential errors in accuracy and completeness and reviewed our past analyses of the reliability of this database. We concluded in our September 2017 report that the data used to draw our sample were sufficiently reliable for the purpose of the survey. For the 2017 survey, we received usable questionnaires from about 67 percent of the eligible sample. The weighted response rate at each agency generally ranged from 57 percent to 82 percent, except the Department of Justice, which had a weighted response rate of 36 percent. The overall survey results are generalizable to the population of managers government-wide and at each individual agency. To assess the potential bias from agencies with lower response rates, we conducted a nonresponse bias analysis using information from the survey and sampling frame as available. The analysis confirmed discrepancies in the tendency to respond to the survey related to agency and SES status. The analysis also revealed some differences in response propensity by age and GS level; however, the direction and magnitude of the differences on these factors were not consistent across agencies or strata. Our data may be subject to bias from unmeasured sources for which we cannot control. Results, and in particular estimates from agencies with low response rates such as the Department of Justice, should be interpreted with caution. However, the survey’s results are comparable to five previous surveys we conducted in 1997, 2000, 2003, 2007, and 2013. To address the first objective, we used data from our 2017 survey to update agency scores on our use of performance information index. This index, which was last updated using data from our 2013 survey, averages managers’ responses on 11 questions related to the use of performance information for various management activities and decision making. Using 2017 survey data, we conducted statistical analyses to ensure these 11 questions were still positively correlated. That analysis confirmed that no negative correlations existed and therefore no changes to the index were needed. Figure 11 shows the questions that comprise the index. After calculating agency index scores for 2017, we compared them to previous results from 2007 and 2013, and to the government-wide average for 2017, to identify any statistically significant differences. We focus on statistically significant results because these indicate that observed relationships between variables and differences between groups are likely to be valid, after accounting for the effects of sampling and other sources of survey error. For each of the 11 questions that comprise the index, we identified individual agency results, excluding missing and no basis to judge responses, and determined when they were statistically significantly different from (1) the agency’s results on the same question in 2013, or (2) the government-wide average results on the question in 2017. In this report, we analyzed and summarized the results of our 2017 survey of federal managers. Due to the limited scope of the engagement, we did not conduct additional audit work to determine what may have caused statistically significant changes between our 2017 and past survey results. To further address this objective we completed several statistical analyses that allowed us to assess the association between the index and 22 survey questions that we determined relate to leading practices we previously found promote the use of performance information. See figure 12 for the 22 specific questions related to these five practices that we included in the analysis. When we individually tested these 22 survey questions (bivariate regression), we found that each was statistically significantly and positively related to the index in 2017. This means that each question, when tested in isolation from other factors, was associated with higher scores on the index. However, when all 22 questions were tested together (multivariate regression), we found that 5 questions continued to be positively and significantly associated with the index in 2017, after controlling for other factors. To conduct this multivariate analysis, we began with a base model that treated differences in managers’ views of agency performance management use as a function of the agency where they worked. We found, however, that a model based on agency alone had little predictive power (R-squared of 0.04). We next examined whether managers’ responses to these questions reflecting practices that promote the use of performance information related to their perceptions of agency use of performance information, independent of agency. The results of this analysis are presented in table 1 below. Each coefficient reflects the increase in our index associated with a one-unit increase in the value of a particular survey question. Our final multivariate regression model had an R-squared of 0.67, suggesting that the variables in this model explain approximately 67 percent of the variation in the use index. We also tested this model controlling for whether a respondent was a member of the SES and found similar results. As shown above in table 1, five questions related to three of the leading practices that promote agencies’ use of performance information were statistically significant in 2017. These results suggest that, when controlling for other factors, certain specific efforts to increase agency use of performance information—such as providing information on the validity of performance data—may have a higher return and lead to higher index scores. With respect to aligning agency-wide goals, objectives, and measures, we found that each increase in terms of the extent to which individuals felt that managers aligned performance measures with agencywide goals and objectives was associated with a 0.08 increase in their score on the use index. In terms of improving the usefulness of performance information, we found that having information on the validity of performance data for decision making was the strongest predictor in our model (0.18). As measured here, taking steps to ensure the performance information is useful and appropriate was associated with almost as large a change in a managers’ index score (0.16). In terms of developing agency capacity to use performance information, we found that having sufficient analytical tools to collect, analyze, and use performance information (0.07), and providing or paying for training that would help link their programs to achievement of agency strategic goals (0.10), were also statistically significantly related to a manager’s reported use of performance information. When we combined these results with what we previously found through a similar analysis of 2013 survey results in September 2014, we identified 10 questions that have had a statistically significant association with higher index scores. This reinforces the importance of the five leading practices to promote the use of performance information. For each of these questions, which are outlined in figure 13 below, we determined when agency results were statistically significantly different from 2013 results or the 2017 government-wide average. For the second objective, we examined, based on the extent they responded their programs had been subject to agency data-driven reviews, differences in managers’ use index scores and responses on questions related to practices that promote the use of performance information. We grouped managers based on the extent they reported their programs had been subject to these reviews, from “no extent” through “very great extent.” We then calculated the average index scores for the managers in each of those five categories. We also examined differences in how managers responded to the 10 questions reflecting practices that can promote the use of performance information based on the extent they reported their programs had been subject to data-driven reviews. We grouped managers into three categories based on the extent they reported their programs had been subject to these reviews (no-small extent, moderate extent, great-very great extent). We then compared how these groups responded to the ten questions. For the third objective, we reviewed our past work that assessed Executive Branch activities to enhance the use of performance information; various resources (i.e., guidance, guides, and playbooks) developed by the Office of Management and Budget (OMB) and the Performance Improvement Council (PIC) that could support agencies’ use of performance information; and the President’s Management Agenda, and related materials with information on cross-agency efforts to improve the use of data in federal decision making. Lastly, for the third objective we also interviewed OMB and PIC staff about any actions they have taken, or planned to take, to further support the use of performance information across the federal government. We conducted this performance audit from October 2017 to September 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. In addition to the above contact, Benjamin T. Licht (Assistant Director) and Adam Miles (Analyst-in-Charge) supervised this review and the development of the resulting report. Arpita Chattopadhyay, Caitlin Cusati, Meredith Moles, Dae Park, Amanda Prichard, Steven Putansu, Alan Rozzi, Shane Spencer, and Khristi Wilkins also made key contributions. Robert Robinson developed the graphics for this report. Alexandra Edwards, Jeff DeMarco, Mark Kehoe, Ulyana Panchishin, and Daniel Webb verified the information presented in this report. Results of the Periodic Surveys on Organizational Performance and Management Issues Managing for Results: Further Progress Made in Implementing the GPRA Modernization Act, but Additional Actions Needed to Address Pressing Governance Challenges. GAO-17-775. Washington, D.C.: September 29, 2017. Supplemental Material for GAO-17-775: 2017 Survey of Federal Managers on Organizational Performance and Management Issues. GAO-17-776SP. Washington, D.C.: September 29, 2017. Program Evaluation: Annual Agency-wide Plans Could Enhance Leadership Support for Program Evaluations. GAO-17-743. Washington, D.C.: September 29, 2017. Managing for Results: Agencies’ Trends in the Use of Performance Information to Make Decisions. GAO-14-747. Washington, D.C.: September 26, 2014. Managing for Results: Executive Branch Should More Fully Implement the GPRA Modernization Act to Address Pressing Governance Challenges. GAO-13-518. Washington, D.C.: June 26, 2013. Managing for Results: 2013 Federal Managers Survey on Organizational Performance and Management Issues, an E-supplement to GAO-13-518. GAO-13-519SP. Washington, D.C.: June 26, 2013. Program Evaluation: Strategies to Facilitate Agencies’ Use of Evaluation in Program Management and Policy Making. GAO-13-570. Washington, D.C.: June 26, 2013. Government Performance: Lessons Learned for the Next Administration on Using Performance Information to Improve Results. GAO-08-1026T. Washington, D.C.: July 24, 2008. Government Performance: 2007 Federal Managers Survey on Performance and Management Issues, an E-supplement to GAO-08-1026T. GAO-08-1036SP. Washington, D.C.: July 24, 2008. Results-Oriented Government: GPRA Has Established a Solid Foundation for Achieving Greater Results. GAO-04-38. Washington, D.C.: March 10, 2004. Managing for Results: Federal Managers’ Views on Key Management Issues Vary Widely Across Agencies. GAO-01-592. Washington, D.C.: May 25, 2001. Managing for Results: Federal Managers’ Views Show Need for Ensuring Top Leadership Skills.GAO-01-127. Washington, D.C.: October 20, 2000. The Government Performance and Results Act: 1997 Governmentwide Implementation Will Be Uneven. GAO/GGD-97-109. Washington, D.C.: June 2, 1997.

Question: Write a one-page summary of the report 1.

Answer: SBA's 7(a) program is required to serve creditworthy small business borrowers who cannot obtain credit through a conventional lender at reasonable terms. The Joint Explanatory Statement of the Consolidated Appropriations Act, 2017 includes a provision for GAO to review the 7(a) program. This report discusses, among other things, (1) how SBA monitors lenders' compliance with the credit elsewhere requirement, (2) the extent to which SBA evaluates trends in lender credit elsewhere practices, and (3) lenders' views on the credit elsewhere criteria for 7(a) loans. GAO analyzed SBA data on 7(a) loans approved for fiscal years 2007–2016, the latest available, and reviewed literature on small business lending; reviewed standard operating procedures, other guidance, and findings from SBA reviews performed in fiscal year 2016; and interviewed lender associations and a nonrepresentative sample of 7(a) lenders selected that concentrated on larger lenders. For its 7(a) loan program, the Small Business Administration (SBA) has largely delegated authority to lenders to make 7(a) loan determinations for those borrowers who cannot obtain conventional credit at reasonable terms elsewhere. To monitor lender compliance with the “credit elsewhere” requirement SBA primarily uses on-site reviews conducted by third-party contractors with SBA participation and oversight, and other reviews. According to SBA guidance, lenders making 7(a) loans must take steps to ensure and document that borrowers meet the program's credit elsewhere requirement. However, GAO noted a number of concerns with SBA's monitoring efforts. Specifically, GAO found the following: Over 40 percent (17 of 40) of the on-site lender reviews performed in fiscal year 2016 identified lender noncompliance with the requirement. On-site reviewers identified several factors, such as weakness in lenders' internal control processes that were the cause for lender noncompliance. Most on-site reviewers did not document their assessment of lenders' policies or procedures, because SBA does not require them to do so. As a result SBA does not have information that could help explain the high noncompliance rate. Federal internal control standards state that management should design control activities, including appropriate documentation, and use quality information to achieve the entity's objectives. Without better information on lenders' procedures for complying with the documentation requirement, SBA may be limited in its ability to promote compliance with requirements designed to help ensure that the 7(a) program reaches its target population. SBA does not routinely collect or analyze information on the criteria used by lenders for credit elsewhere justifications. SBA recently began collecting some information on lenders' use of the criteria, but this information is limited, and SBA does not analyze the information that it does collect to better understand lenders' practices. Federal internal control standards state that management should use quality information to achieve the entity's objectives. Without more robust information and analysis, SBA may be limited in its ability to understand how lenders are using the credit elsewhere criteria and identify patterns of use by certain lenders that place them at a higher risk of not reaching borrowers who cannot obtain credit from other sources at reasonable terms. In general, representatives from 8 of 11 lenders that GAO interviewed stated that SBA's credit elsewhere criteria are adequate for determining small business eligibility for the 7(a) program. These criteria help them target their lending to small businesses that would otherwise have difficulty obtaining conventional credit because they are often new businesses or have a shortage of collateral. However, they also said that other factors—such as lender policies and economic conditions—can affect their decisions to offer 7(a) loans. In January 2018, SBA issued revised guidance for the 7(a) program and has provided training on this new guidance to lenders and trade associations. Lenders told GAO they are still in the process of understanding the new requirements. GAO recommends that SBA (1) require its on-site reviewers to document their assessment of lenders' policies and procedures related to the credit elsewhere documentation requirement, (2) collect information on lenders' use of credit elsewhere criteria, and (3) analyze that information to identify trends. SBA generally agreed with the recommendations.