Small Business Administration: Steps Have Been Taken to Improve Administration of the 8(a) Program, but Key Controls for Continued Eligibility Need Strengthening
Published: 2010-03-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Small Business Administration's (SBA) 8(a) program helps eligible socially and economically disadvantaged small businesses compete in the economy by providing business development activities, such as counseling and technical assistance, and providing opportunities to obtain federal contracts on a set-aside basis. GAO was asked to review SBA's internal control procedures for determining 8(a) eligibility. Specifically, we (1) evaluated the procedures and processes that SBA has implemented to ensure that only eligible firms participate in the 8(a) program, and (2) assessed the extent to which SBA uses external mechanisms such as complaint information in helping to ensure that only eligible firms participate. To address these objectives, GAO reviewed SBA guidance and prior reports, interviewed SBA officials, and conducted site visits and file reviews of 123 randomly sampled 8(a) firms covering the most recent 2 years of annual reviews at five SBA locations.
section_title: What GAO Found
paragraphs: SBA relies primarily on its annual review of 8(a) firms to ensure their continued eligibility in the program, but inconsistencies and weaknesses in annual review procedures limit program oversight. GAO's review of a random sample of 8(a) firms identified an estimated 55 percent in which SBA staff failed to complete required annual review procedures intended to assess fundamental eligibility criteria, such as being economically disadvantaged. Multiple factors appear to have contributed to the inconsistencies identified, including the lack of specific criteria in SBA's current regulations and procedures that relate to some eligibility requirements such as determining whether firms exceed program thresholds for industry size averages, personal compensation, and personal asset limits. As a result, firms that may have outgrown the program continued to receive 8(a) program benefits. For example, GAO estimated that 17 percent of the firms we reviewed had exceeded one or more eligibility criteria for 2 consecutive years, but were recommended by SBA for retention. SBA has taken steps to clarify some, but not all, of these rules in recent proposed rule changes. SBA is required by statute to perform annual reviews on 100 percent of 8(a) firms but staff spent significant amounts of time trying to obtain annual review documents from firms--especially firms that did not have 8(a) contracts--which affected the timeliness of reviews. GAO identified a significant number of instances in which firms failed to submit annual review documents as required but still were recommended for retention. The Business Development Specialists' (BDS) dual role of advocacy for and monitoring of the firms may have contributed in part to the retention of ineligible firms. SBA has been addressing some data integrity and compatibility issues by enhancing its primary electronic system for annual review information. Finally, SBA did not maintain an accurate inventory of 8(a) Mentor-Prot?g? Program participant data, which limited the agency's ability to monitor these firms. SBA's program offices did not maintain comprehensive data on or have a system in place to track complaints on the eligibility of firms participating in the 8(a) program. District staff were not aware of the types and frequency of complaints across the agency. As a result, SBA staff lacked information that could be used with other information to help identify issues relating to program integrity and help improve the effectiveness of SBA oversight. Although complaint data are not a primary mechanism to ensure program eligibility, continuous monitoring is a key component in detecting and deterring fraud.

section_title: Letter
paragraphs: In fiscal year 2008, the federal government awarded $93.3 billion in  contracts to small businesses. The Small Business Administration (SBA)  plays an important role in ensuring that small businesses gain access to  federal contracting opportunities. Section 8(a) of the Small Business Act is  intended to help small businesses owned by socially and economically  disadvantaged individuals compete in the economy by providing business  development activities, such as counseling and technical assistance, and  providing access to federal contracting opportunities. The 8(a) program  does not guarantee that participating firms will obtain federal contracts,  but the firms are eligible for competitive (for which only 8(a) firms can  compete) and sole-source (for which awards are made without  competition) federal contracts when certain requirements are met. As of  fiscal year 2008, approximately 9,460 firms were in the program.
However, we and others have identified oversight weaknesses in SBA  controls that are intended to help ensure that only eligible small  businesses gain access to federal contracting opportunities. For example,  in congressional hearings on SBA’s Historically Underserved Business  Zone (HUBZone) Program that you held on July 17, 2008, we highlighted  fraud and eligibility control weaknesses in the program that allowed the  participation of fictitious and ineligible firms. In a November 2008 report  on the 8(a) program, we noted that SBA faced several challenges in its  overall administration of the 8(a) program, including competing demands  on a limited number of staff. For example, the amount of time staff spent  on statutorily mandated annual reviews of all 8(a) firms diminished the  amount of time they could devote to business development activities. Our  recommendations to SBA included assessing the workloads of business  development specialists (BDS) to ensure they could carry out their  responsibilities and improving processes to terminate firms. SBA agreed  with these recommendations in its written comments on the report.
You asked us to review SBA’s internal control procedures for determining  8(a) eligibility, including mentor-protégé arrangements (which provide  technical and management assistance as well as assistance in performing  federal contracts as a prime contractor through joint-venture  arrangements). Specifically, we (1) evaluated the procedures and  processes that SBA has implemented to ensure that only eligible firms  remain in the 8(a) program, and (2) assessed the extent to which SBA uses  external mechanisms, such as complaints by other 8(a) firms, to help  ensure that only eligible firms participate.
To address these objectives, we reviewed applicable statutes and the  legislative history of the 8(a) program, SBA’s regulations and guidance for  administering the program, our previous reports, and studies of the  program conducted by SBA, the SBA Inspector General, and external  organizations. To assess SBA’s compliance with its eligibility review  procedures, we visited 5 of the 68 SBA districts and reviewed files of 123  randomly sampled 8(a) firms and an additional 13 8(a) firms that had  mentor-protégé agreements. For each firm, we reviewed the most recent  2 years of annual reviews for the period 2007-2009, and any existing  mentor-protégé agreements, related documents, and correspondence. We  developed a data collection instrument to collect key annual review  information from each file, including SBA documentation and evidence  supporting eligibility criteria such as financial disadvantage. We selected  SBA districts based on the size of their 8(a) portfolios of firms with  contracts and for geographic diversity. These 5 districts represented 29  percent (or 672) of all active fiscal year 2008 8(a) firms with contracts and  37 percent (or about $2 billion) of contracting obligation dollars. The  results of the interviews and sample results cannot be generalized to all 68  district offices; however, the results of our file review sample can be  generalized to all files managed by the 5 district offices we included in our  review. We also interviewed SBA officials in the Office of Business  Development, Division of Program Certification and Eligibility, and district  staff to discuss their procedures for determining initial and continuing  eligibility, oversight efforts, technical assistance offered, and mechanisms  to help identify ineligible firms in the program. Appendix I discusses our  scope and methodology in further detail.
We conducted our work in Boston, Massachusetts; Denver, Colorado; San  Antonio, Texas; San Francisco, California; and Washington, D.C., between  May 2009 and March 2010 in accordance with generally accepted  government auditing standards. Those standards require that we plan and  perform the audit to obtain sufficient, appropriate evidence to provide a  reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: A firm must meet several initial eligibility requirements to qualify for the  8(a) program (a process known as certification), and then meet other  requirements to continue participation. In general, a concern meets the  basic requirements for admission to the program if it is a small business  that is unconditionally owned and controlled by one or more socially and  economically disadvantaged individuals who are of good character and  U.S. citizens, and which demonstrates the potential for success. Table 1  summarizes the key requirements.
Participation in the 8(a) program lasts 9 years, and once it is completed, a  firm and the individual cannot reapply. The 9-year program tenure is  divided into two stages—a developmental stage covering years 1 through  4, and a transitional stage covering years 5 through 9. During the  transitional years, firms are required to meet certain activity targets for  non-8(a) contracts to ensure they do not develop an unreasonable reliance  on the program. Additionally, firms in the 8(a) program are eligible to  receive sole-source and competitively awarded set-aside federal  contracts.
As part of the 8(a) program, SBA developed the Mentor-Protégé Program,  in which experienced firms mentor 8(a) firms to enhance the capabilities  of the protégé, provide various forms of business developmental  assistance, and improve the protégé’s ability to successfully compete for  contracts. To qualify initially as a protégé, an 8(a) firm must meet one of  three conditions: (1) be in the developmental stage of the 8(a) program, or  (2) never have received an 8(a) contract, or (3) be of a size that is less than  half the size standard corresponding to its primary standard industry code.  The mentor and protégé enter into a written agreement that sets forth the  protégé’s needs and details the assistance the mentor commits to provide  to address those needs. SBA must review and approve the initial  agreement and annually evaluate specific mentor-protégé requirements.
SBA’s 8(a) program is delivered collaboratively by two departments of  SBA. The Office of Business Development (OBD) is responsible for policy  formation and the certifications of 8(a) applications, approval of mentor- protégé applications, as well as the approval of existing 8(a) firms that are  exiting the program (early graduations, approval of changes of ownership,  approval of voluntary withdrawals, approval of terminations, and  suspensions). OBD is also responsible for the virtual training and relevant  policy briefings provided to SBA staff across the country responsible for  executing the 8(a) program on an ongoing basis throughout the year. The  Office of Field Operations (OFO) is responsible for supporting the  business development specialists, tasked with executing the 8(a) program,  who are located in 68 district offices across the country. Selected BDSs  will have 8(a) firms assigned to them. The BDSs work directly with 8(a)  firms to help prepare business plans; provide technical assistance; review  continuing eligibility; coordinate with resource partners that provide  counseling, training, loans, and other assistance to small businesses; and  coordinate additional assistance and training for firms through another  SBA program. BDS staff also conduct annual reviews of the firms’  progress in implementing business plans and analyze firms’ year-end  financial statements, income tax returns, and records of contracting  activity for certain compliance requirements, including program eligibility.  The purpose of the annual reviews is to determine if firms continue to  meet eligibility requirements and to identify business development needs.  SBA long has been required by statute to complete annual reviews of all  firms. As of fiscal year 2008, SBA had 182.5 full-time-equivalent BDS staff.
section_title: While SBA Has Made Improvements to Its 8(a) Annual Reviews, Internal Control Weaknesses and Other Challenges Limit Program Oversight
paragraphs: SBA relies primarily on its annual reviews of 8(a) firms to ensure the  continued eligibility of firms enrolled in the program, but we observed  inconsistencies and weaknesses in annual review procedures related to  determining continued eligibility for the program. For example, we found  that SBA did not consistently notify or graduate 8(a) firms that exceeded  industry averages for economic success or graduate firms that exceeded  the net worth threshold of $750,000. The lack of specific criteria in the  current regulations and procedures may have contributed to the  inconsistencies that we observed, and SBA has taken steps to clarify some,  but not all, of these requirements in a recent proposed rule change.  Although BDSs have been challenged to perform all their  responsibilities—in particular the statutory requirement to perform annual  reviews on 100 percent of 8(a) firms—SBA has not yet assessed its  workload to ensure it could carry out its responsibilities as we  recommended in our 2008 report. SBA recently has implemented new  procedures intended to streamline terminations that may address some of  these inconsistencies that we identified with the lack of termination  actions taken against firms that did not submit annual review documents  as required. Finally, we found that SBA did not maintain an accurate  inventory of Mentor-Protégé Program participants and did not document  some annual oversight activities of these firms. As a result of these  inconsistencies and weaknesses, there is increased potential that firms  that no longer meet SBA 8(a) continuing eligibility requirements could be  allowed to continue in the program and receive 8(a) contracts.
subsections: section_title: SBA’s Staff Did Not Follow Required Annual Review Procedures Related to Continuing Program Eligibility in about Half of the Files We Reviewed
paragraphs: In a substantial number of cases we reviewed, SBA staff failed to complete  required annual review procedures intended to assess fundamental  eligibility conditions, such as the firm’s net worth, used to determine if  participants continue to meet the criteria for being economically  disadvantaged. SBA may terminate firms found to be ineligible based on  several conditions, including failure to submit required documentation for  the annual review process or failure to maintain ownership and control by  a disadvantaged individual. SBA may also graduate firms that have  successfully completed the program by substantially achieving the targets,  objectives, and goals in their business plans prior to the expiration of their  program terms, and demonstrated their ability to compete in the  marketplace without assistance from the program, or where one or more  of the disadvantaged owners no longer are economically disadvantaged (a  process known as early graduation). Criteria used to determine continuing  eligibility and associated conditions such as economic disadvantage  include factors such as personal assets, income, and net worth, while  criteria used to determine if a firm successfully met targets and objectives  include exceeding industry averages for economic success and owners  making excessive withdrawals of company funds or other assets.
We selected a random sample of files from each of the five district offices  we visited to determine if district offices’ practices for monitoring 8(a)  firms were consistent with requirements in regulations, policies, and  procedures. Specifically, we estimated that for the five district offices,  SBA failed to complete one or more annual required review procedures 55  percent of the time. Our estimates were based on a statistical sample of  123 annual review files from a population of 672 files. Of the 123 files  sampled, we identified 67 instances where SBA failed to complete one or  more annual review procedures related to eligibility determinations (a 55  percent rate). We tested seven specific annual review requirements  relating to continuing eligibility: (1) notifying 8(a) firms that they had  exceeded industry averages for economic success, (2) reviewing or  graduating 8(a) firms or providing an explanation for retention if they had  exceeded industry averages for 2 consecutive years, (3) reviewing net  worth or graduating firms in which individuals exceeded the net worth  threshold of $750,000, (4) performing eligibility reviews when required for  such cases as a change in the firm’s ownership, (5) completing the  required annual reviews, (6) obtaining required supervisory reviews (and  signatures), and (7) imposing remedial actions or obtaining waivers for  firms not meeting business activity targets. Table 2 shows information on  the extent to which SBA did not complete these annual review  requirements.
Taking action when a firm exceeded industry averages for economic  success by  notifying firms that exceeded four of seven industry averages for 1  year  graduating or explaining retention of firms that exceeded four of seven  industry averages for 2 consecutive years  Reviewing net worth or graduating firms in which individuals exceeded  adjusted net worth limitations   Performing required eligibility reviews because of a change in the firms’  ownership  Imposing remedial actions or obtaining waivers for firms not meeting  business activity targets    Exceeding industry averages: Officials from two of the five district offices  told us that while the guidance requires notifying 8(a) firms when they  have exceeded industry averages for economic success, in practice the  districts have been using discretion in notifying the firm after the first year  in which this condition occurs. SBA procedures identify exceeding  industry averages as a criterion for considering that the firm has met its  goals and therefore may no longer be economically disadvantaged. The  notification is intended to make participants aware that they may be  subject to early graduation proceedings if they exceed industry averages  for 2 consecutive years. SBA procedures state that if the firm exceeds  industry averages for 2 consecutive years, the participant no longer can be  considered economically disadvantaged unless the BDS provides evidence  that early graduation is not warranted because of compelling reasons.  Officials from these district offices explained that they did not follow  these procedures, even though they were required, because they did not  think that exceeding industry averages always indicated that participants  no longer were economically disadvantaged. The level of staff knowledge  about calculations for industry averages and the way in which staff  entered the calculations into information systems also may have  contributed to failures to meet this requirement. One district office told us  it was not clear how the ratios were calculated. We also found errors in  the calculations of industry averages at another district office. As we  discuss in more detail later in the report, the industry ratio calculations  require the BDS to manually enter data into a template that will then  calculate the ratio of the firm’s performance against that of industry. As  shown in table 2, we estimate that staff failed to complete this requirement  in about 26 percent of the cases in which a notification letter was required,  and in about 4 percent of cases in which industry averages were exceeded  for 2 consecutive years.
Reviewing net worth or graduating firms in which individuals exceeded  adjusted net worth limitations: One of the clearest indicators of  economic disadvantage that SBA uses is the net worth requirement. The  regulations specifically state that for continued eligibility after admission  into the program, adjusted net worth must be less than $750,000. Our file  review shows that SBA retained an estimated 7 percent of the firms we  sampled, in which there was no evidence that staff reviewed the firms’ net  worth, or retained firms in the program despite their exceeding the net  worth limits. Similarly, in our companion report investigating the  potential for 8(a) program fraud and abuse, we identified cases in which  SBA’s files clearly indicated that the firms were not eligible for the 8(a)  program, yet SBA staff failed to terminate or graduate the firms from the  program. Later in this report we discuss different factors that may have  contributed to the retention of firms that clearly appeared to be no longer  eligible, including the BDSs’ dual role of advocacy for and monitoring of  the firms and workload constraints.
Completing eligibility reviews: We estimated that about 4 percent of our  file sample contained no evidence that SBA staff had performed a separate  required eligibility review. Eligibility reviews are required in cases in  which the BDS has reason to question a participant’s eligibility, including a  change in the firm’s ownership (the factor we used for our analysis).  Eligibility reviews are critical because they could uncover program  participants that no longer met control and ownership eligibility  requirements. Representatives from one district office we visited  explained that these reviews were a low priority compared with other  responsibilities, such as completing annual reviews and initial  certifications.
Completing annual reviews: Although SBA is statutorily required to  perform annual reviews of 100 percent of 8(a) firms, we estimated that in  about 2 percent of our sample, the files contained no evidence that SBA  had performed the annual reviews. For example, in two cases, a district  office had no record on file that annual reviews had been performed, and  in three other cases it had bundled 2 years of reviews because of a change  in the internal deadline for completing annual reviews (it skipped an  annual review). Our sample of 123 files included only firms that received  contracts. As a result, SBA could be unaware that a potentially ineligible  firm had received contracts because it had not performed an annual  review.
We also identified a few instances in which SBA failed to follow  procedural requirements related to the annual reviews, including not  consistently documenting supervisory reviews in one district and failing to  take remedial actions for firms not meeting their business activity targets.
Documenting supervisory reviews: One district office did not always have  the required supervisory signatures on the BDSs’ annual review  recommendations. Of the 64 files that we sampled in that district, 20  lacked evidence of supervisory review signatures. That is, it appeared that  only a BDS recommended a firm’s retention or dismissal from the  program. Overall, we estimated that SBA did not meet this requirement for  about 23 percent of the files in the five district offices. The  noncompliance rate in this district may be attributable to the large size of  its 8(a) portfolio—about 20 percent of all active fiscal year 2008 8(a) firms.  According to district officials, the office also had competing priorities,  such as the need to review applications for the Mentor-Protégé Program.  Nevertheless, SBA officials were not properly monitoring their staff in  these cases. Without the quality controls intended by the supervisory  reviews, SBA has limited assurances that the annual reviews are fulfilling  their intended purpose.
Imposing remedial actions or obtaining waivers for firms not meeting  business activity targets: In about 10 percent of the files we reviewed,  district offices did not submit required documentation of remedial actions  or a waiver when a firm in the transitional phase of the program did not  meet its business activity targets. The remedial action is intended as an  incentive for firms to obtain non-8(a) contracts so that they will be  prepared to compete in the marketplace without the assistance of the 8(a)  program upon graduation. Firms are required to achieve their targets or  otherwise are not eligible to receive 8(a) sole-source contracts. By not  notifying firms and setting up a remedial plan when required, the BDSs’  actions did not appear to be consistent with a key business development  activity intended to help firms develop and exit from the program.  Furthermore, SBA could be providing opportunities for potentially  ineligible firms to receive sole-source contracts.
section_title: Although SBA Has Proposed Changes to Its Regulations to Improve Eligibility Determinations, It Still Lacks Specific Guidance for Some Criteria
paragraphs: Our file review results and interviews with district office officials  identified numerous instances in which staff did not consistently apply  objective standards relating to eligibility determinations. SBA lacks  specific criteria in its current regulations and procedures that relate to  some of the eligibility requirements such as determining whether a firm  should be graduated from the program when it exceeds size standards,  industry averages (such as total assets, net sales, working capital, or  pretax profit), limits for personal compensation and assets, and excessive  withdrawals. Furthermore, SBA guidance directs staff to rely on Office of  Hearings and Appeals (OHA) decisions to use as thresholds for eligibility  criteria, such as total assets and total compensation, in order to make  eligibility determinations. However, as we noted in our related  investigation, agency staff did not follow case law consistently.
More specifically, we estimate that 17 percent of the firms had exceeded  one or more eligibility criteria for 2 consecutive years, indicating that the  firms may have been outgrowing the program, but were recommended by  SBA for retention. Although each criterion in and of itself may not be a  determinant for early graduation based on the current regulations, each is  an important factor in determining if these firms continue to meet  eligibility requirements and if they should remain in the program. SBA  considers the totality of circumstances to determine whether a firm has  met its goals and objectives and should be recommended for early  graduation.
In two cases in one district office, firms had exceeded both average  compensation limits and the limits for excessive withdrawals for 2  consecutive years, and still were recommended for retention. The District  Director and staff at the district office agreed that the two cases were red  flags and that the firms should have been recommended for early  graduation or termination.
In another example, at a different district office, one firm that, over its 8- year tenure in the 8(a) program. had exceeded (1) industry averages for 5  years (in 2 of these years, the firm could have been considered for early  graduation because it exceeded industry averages for 2 consecutive years),  (2) compensation limits by having an average salary of more than $200,000  for 2 years, (3) the size standard for its primary North American Industry  Classification System code, (4) and made excessive withdrawals in 1 year,  but in each year was recommended for retention. This firm had more than  $16 million in contracts by its sixth year in the program.
We also found inconsistencies in the use of third-party sources to verify  firm-reported data. For instance, two districts told us they reviewed third- party sources such as Internal Revenue Service (IRS) tax transcripts,  debarments, and bank information such as withdrawals more routinely as  part of their annual review, while two other districts told us they had not  performed any third-party verification. At least in part, these  inconsistencies can be attributed to lack of specific guidance or criteria  regarding the need for third-party verification. Overall, the regulations  state that SBA may terminate a firm on the basis of discovering false  information, but contain few specific requirements to consult third-party  sources for continuing eligibility. For example, participants must submit  the IRS 4506-T transcript request form as part of the annual review  requirements, which allows SBA to request tax return information.  Additionally, the regulations suggest that staff should consult the federal  list of debarred and suspended firms, since such firms are ineligible for  admission to the 8(a) program. However, we found little evidence of  regulatory requirements to obtain other third-party verifications. As noted  in our report on the potential for 8(a) program fraud and abuse, validating  data against other government or third-party sources is a fraud preventive  control meant to keep ineligible firms from entering the program.   However, we found that SBA relied heavily on self-reported information  from the firms during the initial certification and annual reviews, with  limited data validation performed after the firms had entered the program.  Additionally, in that report we make a recommendation to assess the  feasibility of using additional third-party data sources and site visits, based  on random or risk-based criteria, to allow more independent verification  of firm-reported data.
SBA recently proposed changes to its Small Business Size and 8(a)  Business Development Regulations to address technical issues as well as  make more substantive changes resulting from its experience in  implementing the current regulations. The agency last updated most of  these regulations in 1998. According to a senior SBA official, these  changes are intended to help SBA administer the program more  effectively. The proposed rules would introduce more detailed guidance  and allow for less staff judgment, particularly for the standards that  appeared to be associated with the inconsistencies in the annual review  procedures in our review of 8(a) case files. For example, the proposed  regulations define more specific thresholds for considering an individual’s  personal assets and compensation, and whether a firm has exceeded size  standards. However, the proposed regulations do not introduce more  specific requirements relating to exceeding industry averages, and would  increase staff flexibility to make judgments relating to excessive  withdrawals. Furthermore, the proposed rule changes do not address  under what circumstances or to what extent staff should verify firm- reported information with third-party sources. According to SBA, the  proposed rules attempt to address areas where the current regulations  needed more clarity to ensure consistency with SBA policy as well as  areas where the current regulations may unreasonably restrict  participants. For example, the proposed rule changes allow for flexibility  in judgment regarding excessive withdrawals because SBA believes that it  is important that SBA look at the totality of the circumstances in  determining whether to include a specific amount as a withdrawal in an  effort to prevent some firms from circumventing excessive withdrawal  limitations. However, the lack of specific criteria in the current regulations  and procedures reduces assurances that the BDSs are making consistent  and objective determinations about 8(a) firms’ continued eligibility in the  program.
section_title: SBA Has Not Assessed BDS Workload, a Significant Portion of Which Is Devoted to Annual Reviews, a Fact That Limits Other 8(a) Activities
paragraphs: BDSs devote significant time and resources to complying with the  statutory requirement to perform annual reviews on 100 percent of 8(a)  firms, a fact that affects the time and resources they can devote to other  8(a) activities. Monitoring the firms’ continuing eligibility for the 8(a)  program is just one of many responsibilities of the BDS. The BDS also has  an advocacy role—maintaining an ongoing responsibility to assist the  participant in developing the business to the fullest extent possible. This  includes striving to increase both the dollar value and the percentage of  8(a) contracts through communication of procurement activities, training,  and counseling. SBA guidance requires the BDS to be the primary provider  in helping firms develop business plans, seek loans, and receive  counseling on finances, marketing, and management practices.
Officials in all five of the district offices we visited indicated that they met  the 100 percent annual review goal for fiscal year 2008 but stated it was a  time- and resource-intensive process. For example, district staff estimated  that the annual review process consumed from about 40 to 70 percent of  their time. BDSs in the district offices told us their individual portfolios  ranged from about 30 to 140 firms, depending on their experience level.  Three districts noted that BDS turnover resulted in newer staff initially  taking more time to process reviews and having smaller portfolios while  they were learning their job. One of the districts told us that all available  staff in the district office, including staff not assigned to the 8(a) program,  had to assist in completing and processing annual reviews in order to meet  the review goal. District office staff also told us that they spent a  significant amount of time and resources following up with 8(a) firms to  have them submit required documentation such as tax and business  financial information, which also slowed the review process. District  offices indicated that firms that did not have contracts were especially  prone to submitting documents late because annual reviews were not a  priority for them. These delays, in turn, reduced the amount of time that  the BDSs had to spend on firms that exhibited a high risk of  misrepresentation or noncompliance with 8(a) eligibility requirements— monitoring necessary for effective program oversight.
Furthermore, in our November 2008 report we noted that demands of the  annual review process and resource constraints affected SBA’s ability to  conduct other program activities. For this report, some districts noted  that the annual review goal affected their ability to perform site visits;  follow up on issues that warranted more attention, such as red flags  identified in the prior year’s annual review; and conduct other core  business development activities. For instance, the frequency of site visits  varied in the five offices we visited. One district office told us that staff  were able to conduct site visits for all firms, but another district conducted  site visits for about half of its firms, and the remaining districts performed  site visits on a limited basis, citing circumstances such as a firm  transferring into the district or confirming that a firm was operating at a  bona fide place of business. Another district stated that staff do not have  time to follow up on red flags such as concerns identified in prior annual  reviews because of the emphasis on meeting the annual review goal.  Another district also told us that meeting the 100 percent annual review  goal has limited the district’s ability to get out and educate agencies and  firms. This included providing outreach and awareness training. Finally,  another district told us the annual reviews have affected its ability to  provide developmental assistance and services to address the 8(a) firm’s  needs. The officials also stated that it was hard to develop working  relationships with the firms because of the amount of work reports,  projects, and other duties assigned.
Although BDSs have been challenged to perform all their responsibilities,  SBA has not yet assessed their workload to ensure they could carry out  their responsibilities, as we recommended in our 2008 report. As we  reported, SBA did recognize specifically that staffing constraints affected  its ability to perform annual reviews. For example, according to its 2006  Performance and Accountability Report, a main contributing factor in the  agency’s inability to complete annual reviews of all 8(a) firms was a lack  of staff resources in the district offices. However, since our previous work  in 2008, the emphasis on meeting annual review compliance requirements  has strained staff capacity to conduct other core activities for the 8(a)  program.
By not assessing BDS workloads, SBA may be bypassing opportunities to  better support the mission of the 8(a) program—that is, to develop and  prepare small disadvantaged firms for procurement and other business  opportunities. In addition, the lack of time to follow up on issues of  concern identified in prior-year reviews also undermines SBA’s ability to  carry out its monitoring responsibilities.
section_title: Noncompliant Firms Remain in the 8(a) Program, although Termination Rates Have Increased and New Procedures Should Streamline the Termination Process
paragraphs: On the basis of our file review, we observed instances in which firms were  not compliant with 8(a) continuing eligibility requirements related to  document submission, but remained in the program. Failure to submit  documentation as required is the primary source of noncompliance in the  8(a) program, and is listed in the regulations as an example of good cause  for termination. Our file review showed that business development staff  frequently accepted incomplete, incorrect, and late documentation from  firms and in many cases recommended the noncompliant firms for  retention.
subsections: section_title: Most Firms Submitted Documents Late, but Few Were Terminated for This Cause
paragraphs: Of the 123 firms we tested, 61 percent were noncompliant because of  failures to submit documents as required, but staff recommended 3  percent for termination. According to the regulations and procedures,  unless participants are also suspended in conjunction with termination  proceedings, 8(a) firms remain eligible to receive program benefits and to  compete for contracts during termination proceedings, a fact that affords  them the opportunity for notice and an opportunity to appeal a  termination decision.
During interviews with district office staff, SBA officials acknowledged  that some firms took more time than allowed to submit documents. One  district office official stated that some firms did not take deadlines  seriously and would delay the annual review process. District staff  estimated that despite a 30-day deadline, most firms submitted documents  within 30 to 45 days and in some cases, up to 60 days after their  anniversary date. As mentioned earlier, our file review of 123 firms  showed that 49 percent submitted late documentation. In one case, a firm  failed to provide documents on time and SBA staff waited 4 months before  recommending the firm for termination. After receiving the letter of intent  to terminate, the firm took another 2 months to submit the requested  documents. SBA then reinstated the firm after a total of 6 months’  delinquency. In another case, a firm failed to submit financial information,  and business development staff sent the letter of intent to terminate  shortly after the firm’s deadline passed. SBA waited another 6 months for  the firm to submit the required documentation, which turned out to be  incomplete, but upon receipt SBA chose to reinstate the firm. The next  year, the firm submitted a personal financial statement identical to the  previous year’s (including dates), but SBA did not take action.
section_title: Advocacy Role of BDSs May Contribute to Decisions to Retain Firms
paragraphs: As previously discussed, the BDS’s role as an advocate for 8(a) firms may  have contributed to a reluctance to terminate firms even if the BDSs had a  basis for doing so. Staff in one district office explained they worked with  firms before initiating the termination process, in an attempt to avoid  termination and to achieve the program mission of preparing  disadvantaged firms to compete in the market. Similarly, as noted in our  companion report, SBA staff responsible for annually assessing the  eligibility of participants were not actively looking for fraud and abuse in  the program—and in some cases, staff supported firms despite eligibility  concerns that we raised. Furthermore, our file review provides examples  of reluctance to terminate noncompliant firms.
An 8(a) firm sent an unsigned annual update form 3 months after its  deadline. One month later, SBA recommended retention pending receipt of  the firm’s remaining documents, such as the personal financial statement  and tax returns required to demonstrate economic disadvantage. More  than 2 months later, the firm provided partial financial documentation.  Although SBA’s recommendation to retain the firm was based on  expecting to eventually receive the firm’s remaining documents, these  required documents still were outstanding at the time of our file review— which occurred approximately 1.5 years after the initial annual review  deadline. As a result, it is unknown whether the firm was eligible to  continue participating in the 8(a) program because SBA did not have the  needed information to fully assess the financially disadvantaged status of  the firm.
We also have observed instances in which the BDS recommended  termination but higher levels of management retained the noncompliant  firms in the program. For example, one firm did not submit any annual  review documentation and the BDS subsequently recommended it for  termination. SBA headquarters disagreed with the determination and  chose to retain the firm. However, there was no documentation in the file  to explain the basis for this decision.
section_title: The Overall Trend in Recent Years Showed an Increase in Terminations
paragraphs: In contrast to these cases, there has been an overall upward trend of firms  exiting the 8(a) program through termination or voluntary withdrawal.  According to headquarters officials, this trend is a result of the agency’s  emphasis in recent years on fully meeting its statutory requirements to  conduct annual reviews of all firms. By requesting the annual update from  the firm in anticipation of completing the annual review, business  development staff provide the firms an opportunity to demonstrate basic  program compliance.
Table 3 shows exit data trends over the past several years. The most  recent data indicate a sharp increase in overall terminations and voluntary  withdrawals from the 8(a) program. For example, from 2007 to 2008, the  number of terminations increased more than threefold. Firms are given the  option to withdraw from the program when faced with termination  proceedings. SBA headquarters officials explained that some firms prefer a  withdrawal instead of a termination on their record, and that the increase  in annual reviews also increased this opportunity.
Effective September 2009, SBA revised its 8(a) program procedures to  shorten the termination process and improve internal controls. The  procedural change shortens the termination process by 30 days to 135  days. While this falls short of the 75-day reduction SBA officials planned  at the time of our November 2008 report, it may succeed in removing more  ineligible firms from the program. It remains to be seen what effect this  time reduction will have on termination as an eligibility control.
To create the 30-day reduction in the termination procedure, SBA gave the  district offices responsibility for sending letters of intent to terminate  directly to the firms. Previously, district offices had to submit termination  information to headquarters before an intent letter could be mailed.  Because of this change, the district office primarily will be in charge of  handling new documents the firm submits after receiving the intent letter.  By giving the district offices direct responsibility for tracking  documentation and communicating with the firm during this phase, SBA  intends the process to be more streamlined and straightforward.
While the new procedures reaffirm that firms may be terminated for good  cause (as outlined in the program regulations), they provide no additional  discussion of what factors or conditions would warrant termination. The  8(a) regulations to which the program procedures refer do provide  examples of “good cause,” including a “pattern of failure” to make required  submissions in a timely manner. However, they provide no examples or  criteria for staff to use in determining what constitutes a pattern of failure.  The lack of guidance may have contributed to staff decisions to retain or  reinstate noncompliant firms.
section_title: SBA Plans to Address Compatibility and Functionality Issues in the 8(a) Program’s Management Information System to Improve the Efficiency of Annual Reviews
paragraphs: Issues such as data integration, compatibility, and functionality associated  with SBA’s Business Development Management Information System  (BDMIS) for the 8(a) program present challenges that affect effective  program management. The agency has been planning to address some data  integrity and compatibility issues with BDMIS and E-8(a), a database that  provides business status and business contract activity for each  participant in the 8(a) program. District office officials indicated that  information discrepancies existed between the two systems and required  dual data entry of some firm information. SBA officials stated they were  reconciling the information in E-8(a) and BDMIS to address discrepancies.  Additionally, the officials explained that some information had to be  entered separately into the two systems but that they were moving toward  a single data feed. The officials expected this change to occur by the end  of the third quarter of fiscal year 2010.
As of October 2008, BDMIS was operational in all district offices, allowing  8(a) participants to submit their annual review data electronically and the  BDSs to review the documentation electronically. District staff identified  benefits and challenges with the implementation of the online annual  review process in BDMIS. For example, one district told us that learning  the BDMIS system was challenging initially for some 8(a) participants and  depended on participants’ skills and abilities to enter information into the  system. Another district noted that a calculator that assesses a firm’s  performance in its respective industry was a positive addition to the  BDMIS system, allowing the BDS to move through reviews more quickly  and efficiently. But the BDS still had to enter firm financial data manually  into the calculator, a fact that could increase the likelihood for data entry  errors. (The industry ratio calculations require the BDS to manually enter  data into a template that calculates the ratio between the firm’s  performance and that of industry.) For example, at one district we visited,  we observed BDS staff manually entering industry performance ratios.  District staff also told us that BDMIS’s functionality has been limited  because the system did not allow staff to access complete firm  information, such as contract and historical information, and develop  reports. Some district offices also told us that the BDSs’ overall workload  has not improved and that BDSs spend a significant amount of their time  following up with 8(a) firms to submit relevant annual review documents.  Despite these challenges, district staff with whom we spoke said BDMIS  has been helping to achieve better organization and tracking and  anticipated that when fully operational, it could save time and increase  transparency.
SBA officials also told us that they have been planning to upgrade BDMIS,  which currently is operating in its first version. SBA expects to complete  three upgrades by the end of the fiscal year 2010. As part of the upgrades,  SBA plans to integrate an existing federal database, the Federal  Procurement Data System-Next Generation, that contains contracting  information that could help SBA staff to verify firms’ contracting  information and enable district staff to run reports on their 8(a) firms.  District staff told us they rely on the 8(a) firm and federal agencies to  provide contract information that is used in the annual review to  determine a firm’s ratio of 8(a) and non-8(a) contracts. As the firm  matures, the goal for 8(a) firms is to increase the amount of non-8(a)  contract work and decrease reliance on 8(a) contracts. However, one  district explained that contract information such as contracts pending and  awarded is recorded in E8(a) but the information is not complete because  it does not contain obligation data.
SBA’s planned system upgrades could improve the efficiency of annual  reviews, particularly because they would likely address duplicative data  entry, make more information readily available to staff, and decrease the  amount of time spent on annual reviews. However, it is too early to tell  whether these changes, once implemented and fully operational, would  achieve their intended purposes.
section_title: SBA Did Not Have an Accurate Inventory of Mentor-Protégé Firms and Did Not Follow Important Procedures to Properly Monitor the Program
paragraphs: SBA did not maintain an accurate list of Mentor-Protégé Program  participants. Specifically, the headquarters office has had difficulty  verifying which firms actively participate in the program. An SBA  headquarters official responsible for the program stated that staff added  firms to a working list based on agreements once they were approved at  headquarters. However, this list is not systematically updated when  mentor-protégé agreements are extended or dissolved, which occurs at the  district office level instead of at headquarters. While the list constituted  the agency’s only central participation roster for the program, officials  stated it was not meant to be used as an eligibility control. Most district  offices that we visited kept their own lists, which occasionally were used  to verify the headquarters list. One district office we visited did not  compile a list of its mentor-protégé participants, but instead relied on  individual program files and the list from headquarters for information.  When we followed up with other district offices, we found contradictory  or inconsistent data in comparison with those of headquarters. For  example, the headquarters list showed two active mentor-protégé  agreements for a district office that stated it had no active participants.
Because there is no list of active mentor-protégé agreements, SBA may not  be able to properly monitor 8(a) protégé firms that submit agreements  with more than one mentor, or mentors that submit agreements with more  than one 8(a) protégé. Currently, mentors may have more than one  protégé if specially approved by SBA. At least 28 mentor firms appeared to  have more than one protégé firm, but SBA was unable to confirm whether  5 of these mentors were authorized to do so. SBA has proposed new  regulations that would limit mentor firms to a maximum of 3 protégé firms  at a time. SBA also has proposed changes to the regulations that would  allow protégé firms to have more than 1 mentor under limited  circumstances. To date, the regulations have prohibited protégé firms  from having more than one mentor at a time. However, we identified 12  protégé firms that appeared to have 2 mentors at the same time. SBA  indicated that some of these relationships had been dissolved, but these  firms remained on its list of approved mentor-protégé agreements. The  current lack of data limits the agency’s ability to fully monitor the Mentor- Protégé Program. As a result, unauthorized partnerships could receive 8(a)  set-aside contracts.
Maintaining an accurate list of firms participating in SBA’s Mentor-Protégé  Program is an important control mechanism to ensure participation only  by eligible firms and that the agency has relevant and reliable information  for management. Monitoring eligibility for the Mentor-Protégé Program is  especially important because participants were more successful in earning  proceeds from federal contracts in fiscal year 2008 than the larger pool of  8(a) firms.  Mentor-protégé participants averaged $4.1 million in sales  compared with $2.4 million for other 8(a) firms. As a group, these  participants earned $638 million in fiscal year 2008.
In addition to finding high-level data inconsistencies, including  unverifiable participation lists and mentors and protégés with multiple  agreements, we found cases in which SBA failed to properly document  analysis and monitoring of the Mentor-Protégé Program. As part of our file  review across five district offices, we tested 20 8(a) firms with mentor- protégé agreements. We focused on initial agreement information, annual  updates, and recommendations. Our file review results showed that SBA  staff failed to comply with certain initial review and annual review  procedures for participants in 6 of the 20 mentor-protégé cases that we  reviewed. These procedures include providing a written eligibility analysis  and ensuring a signed supervisory review of the BDS’s recommendation.  In our interviews with district office officials, we also found that Mentor- Protégé eligibility information had not been incorporated into BDMIS.  District offices were not able to integrate initial approval  recommendations and annual review monitoring with the firm’s general  8(a) eligibility information held electronically in BDMIS. As a result of the  lack of documentation and the data limitations discussed above, SBA has  not been able to properly oversee this program.
section_title: SBA Lacks a Formal Mechanism to Collect and Analyze Complaint Data Related to 8(a) Eligibility
paragraphs: SBA can receive information and complaints from other 8(a) firms,  disgruntled 8(a) employees, and anonymous sources, but SBA does not  maintain comprehensive data about complaints such as allegations that  certain 8(a) firms may not comply with eligibility requirements.  Although  complaint information is not the primary mechanism for ensuring  continuing program eligibility, it can be an additional tool for identifying  fraud or wrongdoing. As we noted in our other GAO investigative report  on the 8(a) program, detection and monitoring are crucial elements in a  well-designed fraud prevention system. Complaints and other allegations  regarding the eligibility of firms in the program can serve as red flags for  SBA staff to take additional steps to ensure that firms continue to meet  program requirements.
District office officials told us that complaints received at the district  receive an initial review (to determine if they warrant follow-up), which  may include follow- up with other agencies and the specific firm to gather  more information. SBA’s standard operating procedures instruct staff to  refer to SBA’s Office of Inspector General (OIG) any possible criminal  violations and other wrongdoing involving SBA programs, such as  knowingly making or using a statement or document that is false,  fictitious, or fraudulent. If warranted, complaints are to be referred to  SBA’s OIG for possible investigation. One district told us that the district  counsel reviews the evidence, and if the case has merit, the information is  referred to the SBA OIG for further investigation. Two other districts told  us the BDS will seek more information by checking with the contracting  agency involved regarding the nature of the complaint or contacting the  8(a) firm for clarification before making a referral to OIG.
However, because district staff do not collect and maintain comprehensive  complaint information involving 8(a) firms, staff are not aware of the types  and frequency of complaints across the agency, including potential  eligibility concerns. Specifically, none of the five districts that we visited  were able to provide us with a list of complaints or allegations that they  received over the past year regarding the potential ineligibility of 8(a)  firms in their districts. While OIG maintains general complaint information  such as the name of the 8(a) firm and type of complaint, a senior OIG  official told us that 8(a) complaints involving a single company generally  did not rank high in priority for a review because of resource limitations  and other priorities but that it might be considered in the OIG’s work- planning effort. OIG officials explained that the OIG ultimately also could  refer a case to the U.S. Attorney for prosecution, but that the threshold for  prosecutions was high and many cases did not meet that threshold.
As a result, it appears that complaint data involving 8(a) firms are not  being utilized to the full extent as a means to identify potential areas of  concern such as program eligibility issues. Without a standard process for  collecting and analyzing complaints, SBA staff—and the agency as a  whole—lack information that could be used to help identify issues relating  to program integrity and help improve the effectiveness of SBA oversight.
section_title: Conclusions
paragraphs: SBA’s 8(a) program provides opportunities for participating firms to  collectively receive billions of dollars in federal contracts on a competitive  or noncompetitive basis. As a result, it is critical that SBA’s annual reviews  of 8(a) firms are performed effectively to help ensure that only eligible  firms are allowed to continue to participate in and benefit from the  program. However, our file review at five district offices found  inconsistencies in the annual review policies and procedures followed by  SBA staff related to program eligibility. This suggests a need for greater  monitoring by SBA and potentially a need for more guidance and training  to ensure greater consistency in the performance of required annual  review procedures.
Furthermore, the lack of specific criteria in the current regulations related  to eligibility determinants such as size standards and industry averages  and the dual roles of the BDSs—providing oversight and being an advocate  for the firm—may have contributed to the variation in annual review  practices we observed. By clarifying guidance, further detailing or  expanding procedures, and emphasizing the importance of quality  controls, SBA could help eliminate ambiguities, improve the quality of  reviews, and provide clearer criteria against which to judge eligibility and  ensure that only intended recipients benefit from program participation.
Workload constraints of BDS staff may have been a contributing factor to  the inconsistencies and deficiencies identified in our review of annual  review files in the five districts that we visited. While the annual review  process is central to ensuring program integrity, SBA’s statutory  requirement to conduct annual reviews of 100 percent of 8(a) firms also is  time- and resource-intensive. The workload demands associated with the  annual review process likely have affected the quality of these reviews as  well as detracted from the time staff have been able to devote to other  core 8(a) program responsibilities, ranging from technical assistance to  mentoring. As we previously recommended and continue to believe, an  assessment of the BDS workload could help ensure the BDSs can carry  out their responsibilities and determine what mechanisms can be used to  prioritize or redistribute their workload. Such an assessment also would  be helpful in assessing the multiple roles and responsibilities of BDS staff,  including ways to mitigate the conflicting roles of business development,  and ensuring that only eligible firms are allowed to participate in the  program. In a fiscally challenged environment and with workload  constraints as a constant, it is important that the agency review staff and  resource allocations and identify process efficiencies wherever possible.
Changes that SBA recently made to termination procedures, coupled with  the increase in terminations overall, may help to alleviate workload  constraints for district office staff. As we noted in our November 2008  report, the inefficient termination process consumed scarce SBA  resources and may have affected business development activities. District  staff could take advantage of the revised, more efficient termination  process to minimize time spent waiting for documents from firms and free  up time for business development and other activities. However, SBA  retained some firms that repeatedly did not submit required  documentation for annual reviews. By monitoring the implementation of  regulations relating to documentation requirements, SBA could help staff  more readily identify firms for termination, reduce the time staff spent  “chasing” documentation, and help improve the timeliness of annual  reviews. Additionally, by providing specific examples in the regulations or  procedures of what is considered to be a pattern of failure, staff would be  able to better justify termination decisions.
The agency also faces a number of challenges in effectively monitoring  and managing the Mentor-Protégé Program, which is an important subset  of the 8(a) program. For example, SBA headquarters and district offices  could not agree or provide current and basic information on the total  number of mentor-protégé agreements. Maintaining accurate information  on participants is a basic and important control mechanism to monitor  8(a) protégé firms that submit agreements with more than one mentor, or  mentors that submit agreements with more than one 8(a) protégé. By  developing a centralized process to collect and maintain information on  program participants, SBA would have a critical tool necessary to properly  monitor and oversee the program.
Finally, SBA also has an opportunity to develop another tool that could  enhance its oversight of the 8(a) program. Currently, SBA lacks  comprehensive data on complaints involving 8(a) firms because it does  not systematically collect and analyze information on the nature of the  complaints and their disposition. Although complaint data are not a  primary mechanism to ensure program eligibility, continual monitoring is  a key component in detecting and deterring fraud. By developing an  agencywide process for documenting and analyzing complaints, SBA  would have an information resource that could be used with other efforts  to provide reasonable assurance that only eligible firms are participating  in the program.
section_title: Recommendations for Executive Action
paragraphs: To improve the monitoring of and procedures used in assessing the  continuing eligibility of firms to participate in and benefit from the 8(a)  program, we recommend that the Administrator of SBA take the following  six actions:    To help ensure greater consistency in carrying out annual review  procedures and improve the overall quality of these reviews, we  recommend that the SBA Administrator monitor, and provide additional  guidance and training to, district offices on the procedures used to  determine continuing eligibility, including  taking appropriate action when firms exceed four of seven industry  size averages, including notifying firms the first year and enforcing  procedures relating to early graduation of firms that exceed industry  averages for 2 consecutive years;    obtaining appropriate supervisory signatures to finalize annual review    submitting remedial action or a waiver for firms in the transition phase  that did not meet business activity targets;    graduating firms that exceed the net worth threshold of $750,000;    performing timely eligibility reviews in required cases; and    completing required annual reviews.
To help reduce inconsistencies between districts and BDS staff in annual  review procedures requiring judgment, we recommend that SBA review its  existing 8(a) program regulations and its proposed changes with the intent  of providing additional criteria and examples for staff when assessing key  areas of program eligibility and determining whether a firm should be  graduated from the program when it exceeds size standards, industry  averages (such as total assets, net sales, working capital, or pretax profit),  and limits for personal compensation and assets, and excessive  withdrawals.
To help address competing demands on 8(a) resources, SBA should assess  the workload of business development specialists to ensure that they can  carry out all their responsibilities. As part of this assessment, SBA should  review the roles and responsibilities of the BDSs to minimize or mitigate to  the extent possible the potentially conflicting roles of advocacy for firms  in the program with the responsibility of ensuring that only eligible firms  are allowed to continue to participate in the program. In addition, SBA  should review the size of the 8(a) portfolio for all business development  specialists and, if necessary, determine what mechanisms should be used  to prioritize or redistribute their workload.
To reduce the practice of retaining firms that fail to submit annual review  documentation as required, SBA should monitor the implementation of  regulations relating to termination to see if they are achieving their  purpose or whether business development staff need further guidance in  interpreting the regulations. SBA should consider providing specific  examples of what might be considered a pattern of failure to submit  documentation as required.
To better manage and monitor participation in the Mentor-Protégé  Program, including compliance with the number of allowable mentor and  protégé firms, SBA should develop a centralized process to collect and  maintain up-to-date and accurate data on 8(a) firms participating in the  Mentor-Protégé Program. SBA should consider incorporating information  on Mentor-Protégé approvals, extensions, and dissolutions in existing  electronic data systems used for the annual review process.
To more fully utilize and leverage third-party complaints to identify  potentially ineligible firms participating in the 8(a) program, design and  implement a standard process for documenting and analyzing complaint  data.
section_title: Agency Comments and Our Evaluation
paragraphs: We requested SBA’s comments on a draft of this report, and SBA’s  Associate Administrator of the Office of Government Contracting and  Business Development provided written comments that are presented in  appendix II. SBA agreed with each of the six recommendations and stated  that some corrective measures have already been implemented and  additional actions are planned to be implemented in the near future. For  example, SBA stated it has implemented a comprehensive training  curriculum, revised guidance for annual review procedures, and will  provide additional examples that will assist staff in assessing key areas in  making annual review determinations. SBA also indicated that it had  begun to develop a routine centralized process to collect and maintain  accurate data related to the Mentor-Protégé Program. Finally, SBA stated  that it plans to assess BDS workload and develop a central repository for  third-party complaints.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to other  interested congressional committees and the Administrator of the Small  Business Administration. The report will also be available at no charge on  the GAO Web site at http://www.gao.gov.
If you or your office have any questions about this report, please contact  me at (202) 512-8678 or shearw@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Key contributors to this report are listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: Our objectives were to (1) evaluate the procedures and processes that the  Small Business Administration (SBA) has implemented to ensure that only  eligible firms remain in the 8(a) program, and (2) assess the extent to  which SBA used external mechanisms, such as complaints by other 8(a)  firms, to help ensure that only eligible firms participate in the program.
To evaluate the procedures and processes that SBA has implemented to  help to ensure that only eligible firms participate in the 8(a) program, we  reviewed applicable statutes and the legislative history of the 8(a)  program, SBA’s regulations and guidance for administering the program,  our previous reports, and studies of the program conducted by SBA, SBA’s  Office of Inspector General (OIG), and external organizations.  Additionally, we randomly sampled files for review at 5 selected district  offices to assess SBA’s compliance with its eligibility review procedures  for the 8(a) and Mentor-Protégé programs. We selected the 5 district  offices based on the high dollar value of contract obligations in these  districts and geographic diversity. Our sample population included firms  that were active in the 8(a) program in fiscal year 2008 and had 8(a)  contracts in fiscal year 2008. We identified these firms by using SBA’s list  of active fiscal year 2008 8(a) firms and matching these data to the Federal  Procurement Data System-Next Generation (FPDS-NG) to determine  which of those firms had obligations. For our review, we excluded those  firms that joined the program during calendar year 2008, because these  firms would not yet have been in the program long enough to have an  annual review on file. We also excluded Alaska Native Corporations,  tribally owned, Native Hawaiian Organization-owned, other Native  American-owned, and Community Development Corporation-owned firms  because of the different 8(a) eligibility requirements applied to these  entities. The results of our sample are generalizable only to the 5 district  offices. We randomly sampled 123 8(a) firms from our population, and an  additional 13 8(a) firms that had mentor-protégé agreements, which we  judgmentally selected from SBA’s list of Mentor-Protégé firms as of  September 2009. For each firm, we reviewed its most recent 2 years of  annual reviews for the period 2007-2009, and any existing mentor-protégé  agreements, related documents, and correspondence. We developed a data  collection instrument (DCI) to collect key annual review data from each  file. The DCI was pretested in 2 district offices and modified based on  these tests. We also analyzed mentor- protégé data to identify protégé  firms that may have multiple mentors, which are against regulation, and  mentor firms that may have multiple protégés, which is allowable only  when specially authorized by SBA. To identify these cases, we sorted the  data by firm name and searched for duplicate matches. A total of 672 firms  met our study criteria and are shown in table 4.
We randomly selected the indicated number of cases within each regional  office. We treated this as a stratified random sample and weighted the  sample cases accordingly for our analysis. Our estimates are statistically  representative for all files maintained in these 5 SBA regional offices.
Because we treated our file review as a stratified random sample, we  assumed our sample was only one of a large number that could have been  drawn. Because each sample could have provided different estimates, we  expressed our confidence in the precision of our particular sample’s  results as a 95 percent confidence interval. This is the interval that would  contain the actual population value for 95 percent of the samples we could  have drawn. As a result, we are 95 percent confident that each of the  confidence intervals based on the file review includes the true values in  the sample population. The 95 percent confidence intervals for each of the  estimates are summarized in table 5.
We performed appropriate data reliability procedures for our sample  testing at the 5 district offices and analysis of inappropriate mentor-  protégé relationships. We compared SBA data with data from other  sources such as FPDS-NG and the Central Contractor Registry, performed  electronic testing, reviewed related documentation and internal controls,  and performed interviews with knowledgeable agency officials. We  determined that the data were sufficient to perform our sample testing and  project our results to the 5 district offices in our population of 8(a) firms.  We also determined through these methods that data relating to mentor- protégé participants were sufficient to report on descriptive statistics of  mentor-protégé firms with contracts. The discrepancies we found in the  general list of mentor-protégé participants are documented within the  report.
To assess the extent that external mechanisms exist, such as complaints  by other 8(a) firms, to help ensure that only eligible firms participate, we  interviewed agency and SBA Office of Inspector General officials, and we  reviewed SBA OIG complaint data. We also interviewed officials in SBA’s  Office of Business Development, Division of Program Certification and  Eligibility, and district office staff to discuss their procedures for  determining initial and continuing eligibility, oversight efforts, technical  assistance offered, and mechanisms to help identify ineligible firms in the  program.
We conducted our work in Boston, Massachusetts; Denver, Colorado; San  Antonio, Texas; San Francisco, California; and Washington, D.C., between  May 2009 and March 2010 in accordance with generally accepted  government auditing standards. Those standards require that we plan and  perform the audit to obtain sufficient, appropriate evidence to provide a  reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Small Business Administration
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Harry Medina (Assistant Director),  Carl Barden, Tania Calhoun, Janet Fong, Cindy Gilbert, Julia Kennon, Amy  Moran Lowe, Barbara Roesmann, Verginie Tarpinian, and William Woods  made key contributions to this report.