Federal agencies conduct a variety of procurements that are reserved for small business participation (through small business set-aside and sole- source opportunities, hereafter called set-asides). The set-asides can be for small businesses in general or be specific to small businesses meeting additional eligibility requirements in the Service-Disabled Veteran-Owned Small Business Concern (SDVOSBC), Historically Underutilized Business Zone (HUBZone), 8(a) Business Development, and WOSB programs. The WOSB program, which started operating in 2011, has requirements that pertain to the sectors in which set-asides can be offered as well as eligibility requirements for businesses. That is, set-aside contracts under the WOSB program can only be made in certain industries in which WOSBs were substantially underrepresented and EDWOSBs underrepresented, according to the program regulation. Additionally, only certain businesses are eligible to participate in the WOSB program. The business must be at least 51 percent owned and controlled by one or more women. The owner must provide documents demonstrating that the business meets program requirements, including submitting a document in which the owner attests to the business’s status as a WOSB or EDWOSB. The program’s authorizing statute directs that each business either be certified by a third party, or self-certified by the business owner. SBA’s final rule includes these two methods. Self-certification is free and businesses pay a fee for third-party certification. A third-party certifier is a federal agency, state government, or national certifying entity approved by SBA to provide certifications of WOSBs or EDWOSBs. To be approved as certifiers, interested organizations submit an application to SBA that contains information on the organization’s structure and staff, policies and procedures for certification, and attestations that they will adhere to program requirements. SBA has approved four organizations to act as third-party certifiers: El Paso Hispanic Chamber of Commerce; National Women Business Owners Corporation; U.S. Women’s Chamber of Commerce; and Women’s Business Enterprise National Council. The most active certifier is the Women’s Business Enterprise National Council (WBENC), which completed about 76 percent of all WOSB third- party certifications performed from August 2011 through May 2014. To conduct the certifications, WBENC uses 14 regional partner organizations. The fees for certification vary depending on a WOSB’s gross annual sales, membership status in the certifying organization, and geographic location (see table 1). In the case of businesses that seek a WOSB program certification through WBENC’s partner organizations, businesses that pay for a Women’s Business Enterprise certification (used for private- sector or some local, state, and federal procurement, but not for the WOSB program) can receive WOSB program certifications at no additional cost. We discuss the WOSB certification process in greater detail later in this report. SBA’s Office of Government Contracting administers the WOSB program by publishing regulations for the program, conducting eligibility examinations of businesses that received contracts under the WOSB or EDWOSB set-aside, deciding protests related to eligibility for a WOSB program contract award, conducting studies to determine eligible industries, and working with other federal agencies in assisting WOSBs and EDWOSBs. According to SBA officials, the agency also works at the regional and local levels with its Small Business Development Centers, district offices, and other organizations (such as Procurement Technical Assistance Centers) to assist WOSBs and EDWOSBs to receive contracts with federal agencies. The services SBA coordinates with these offices and organizations include training, counseling, mentoring, access to information about federal contracting opportunities, and business financing. According to the program regulation, businesses may use self- or third- party certification to demonstrate they are eligible for WOSB or EDWOSB status. Both certification processes require signed representations by businesses about their WOSB or EDWOSB eligibility. For this reason, SBA has described all participants in the program as self-certified. When using the self-certification option, businesses must provide documents supporting their status to the online document repository for the WOSB program that SBA maintains. Required submissions include copies of citizenship papers (birth or naturalization certificates or passports) and, depending on business type, items including copies of partnership agreements or articles of incorporation. Businesses must submit a signed certification on which the owners attest that the documents and information provided are true and accurate. Moreover, businesses must register and attest to being a WOSB in the System for Award Management (SAM), the primary database of vendors doing business with the federal government. Businesses also must make representations about their status in SAM before submitting an offer on a WOSB or EDWOSB solicitation. For third-party certification, businesses submit documentation to approved certifiers. According to third-party certifiers we interviewed, they review documents (and some may conduct site visits to businesses) and make determinations of eligibility. If approved, businesses will receive a document showing receipt of third-party certification. Business then can upload the certificate to the WOSB program repository along with documents supporting their EDWOSB or WOSB status. SBA does not track the number of businesses that self certify and could not provide information on how many self-certified businesses obtained contracts under the WOSB program. While SBA can look at an individual business profile—which lists the documents the business has uploaded to support its eligibility—in the repository to determine if a certificate from a third- party certifier is present, it has no corresponding mechanism to determine if a business lacking such a certificate was self-certified. That is, there are no data fields for certification type in any of the systems used in the program and SBA cannot generate reports to isolate information on certification type by business. According to SBA officials, such information on certification type is not needed because both certification options are treated equally under the program and, because all businesses make an attestation of status as a WOSB whether or not the business uses a third- party certifier. Therefore, SBA considers this a self-certification program. Contracting officers obtain a solicitation and conduct market research to identify businesses potentially capable of filling contract requirements. Once a contracting officer has determined that a solicitation can be set aside under the WOSB program, the officer obtains bids and selects an awardee for the contract. Only after selecting an awardee, does the agency obtain access to the business’s profile in the WOSB program repository, which lists the documents the business has uploaded to support its eligibility (the business must grant the contracting agency access). SBA’s Contracting Officer’s Guide to the WOSB Program states that contracting officers must determine that specified documents have been uploaded by the business to the program repository, but the guide does not require contracting officers to assess the validity of those documents. Only after viewing the uploaded documents would the contracting officer be able to determine if the business was likely self- certified or had a certificate from a third-party certifier. Two groups we interviewed that represent the interests of WOSBs said that contracting officers prefer third-party over self-certified businesses when selecting an awardee. A representative of one organization thought that contracting officers tended to select businesses with third-party certifications because they did not have to review as many documents in the program repository as for self-certified businesses. However, the certification method does not appear to influence contract awards. According to officials from all contracting agencies with whom we spoke and SBA officials, contracting staff are unaware of the certification method used by a business until after an awardee is selected. SBA generally has not overseen third-party certifiers and lacks reasonable assurance that only eligible businesses receive WOSB set- aside contracts. SBA has not put in place formal policies to review the performance of third-party certifiers, including their compliance with a requirement to inform businesses of the no-cost, self-certification option. The agency has not developed formal policies and procedures for reviewing required monthly reports submitted to SBA by certifiers or standardized reporting formats for the certifiers, or addressed most issues raised in the reports. Although SBA examinations have found high rates of ineligibility among a sample of businesses that previously received set- aside contracts, SBA has not determined the causes of ineligibility or made changes to its oversight of certifications to better ensure that only eligible businesses participate in the program. To date, SBA generally has not conducted performance reviews of third- party certifiers and does not have procedures in place for such reviews. According to federal standards for internal control, agencies should conduct control activities such as performance reviews and clearly document internal controls. Third-party certifiers agree to be subject to performance reviews by SBA at any time to ensure that they meet the requirements of the agreement with SBA and program certification regulations—including requirements related to the certification process, obtaining supporting documents, informing businesses about the no-cost option for WOSB program certification, and reporting to SBA on certifier activities. Before beginning the certification process, SBA requires third-party certifiers to inform businesses in writing (on an SBA-developed form) that they can self certify under the program at no cost. Certifiers, a WOSB advocacy group, and WOSBs had perspectives on fees for third-party certification. Representatives of all three certifiers with whom we spoke stated that fees their organization charged for certifications were reasonable and affordable for a small business. Staff from one WOSB advocacy organization told us that such fees could deter some businesses from participating in the program, but owners of WOSBs with which we spoke generally did not concur with this view. Certifiers with whom we spoke told us that they inform businesses about their option to self certify, but SBA does not have a method in place to help ensure that certifiers are providing this information to businesses and agency officials told us that they do not monitor whether certifiers fulfilled the requirement. SBA officials said that they believe that the no-cost option ameliorates the risk of excessive fees charged to businesses or the risk that fees would deter program participation and that because all certifiers must provide national coverage, businesses can seek lower fees. Officials also told us that they believed that businesses and advocacy groups would inform the agency if certifiers were not providing this information. However, they were not able to describe how SBA would learn from businesses that certifiers had failed to provide this information. The requirement is part of SBA’s agreement with third-party certifiers, but SBA has not described the requirement on the program web-site or made it part of informational materials to businesses. Thus, businesses may not know of this requirement without being informed by the certifier or know to inform SBA if the certifier had not fulfilled the requirement. The largest certifier, WBENC, has delegated the majority of certification activity to other entities that SBA also has not reviewed. WBENC has conducted about 76 percent of third-party certifications through May 2014. However, WBENC delegates WOSB certification responsibilities to 14 regional partner organizations. SBA neither maintains nor reviews information about standards and procedures at WBENC, including a compliance review process for each of its 14 partner organizations that WBENC told SBA it uses. SBA officials told us that they rely on information available on public websites to determine the fee structures set by WBENC’s partner organizations. SBA also does not have copies of compliance reviews that WBENC told SBA it annually conducts for each partner organization. SBA requested documents from WBENC, which included information about WBENC’s oversight of its 14 partner organizations. WBENC’s response was incomplete; WBENC referenced but did not provide its standards and procedures to oversee partner organizations. SBA told us it recognized that WBENC’s response was incomplete, and indicated it had not followed up on WBENC’s response. Without this information SBA cannot determine how WBENC has been overseeing the 14 entities to which it has delegated certification responsibilities. Although SBA has not developed or conducted formal performance reviews of certifiers, officials described activities they consider to be certifier oversight. For example, when a business is denied third-party certification but wishes to self-certify, it must subject itself to an eligibility examination by SBA before doing so. In this case, or during a bid protest, SBA conducts its own review of documentation the business submitted to the certifier. SBA officials stated that these reviews were not intended as a form of certifier oversight but described them as de facto reviews of third-party certifier performance. However, such reviews do not involve a comprehensive assessment of certifiers’ activity or performance over time. An SBA official acknowledged that the agency could do more to oversee certifiers. SBA plans to develop written procedures for certifier oversight to be included in the standard operating procedure (SOP) for the program, which remains under development. But SBA has not yet estimated when it would complete written procedures for certifier oversight or the SOP. Without ongoing monitoring and oversight of the activities and performance of third-party certifiers, SBA cannot reasonably ensure that certifiers have fulfilled the performance requirements of their agreement with SBA—including informing businesses about no-cost certification. SBA has not yet developed written procedures to review required monthly reports from certifiers and does not have a consistent format for reports. In SBA’s agreement with third-party certifiers, the agency requires each certifier to submit monthly reports that must include the number of WOSB and EDWOSB applications received, approved, and denied; identifying information for each certified business, such as the business name; concerns about fraud, waste, and abuse; and a description of any changes to the procedures the organizations used to certify businesses as WOSBs and EDWOSBs. Internal control should include documented procedures and monitoring or review activities that help ensure that review findings and deficiencies are brought to the attention of management and resolved promptly. Based on our review of each monthly report submitted from August 2011 through May 2014 (135 in total), not all reports contained consistent information. Some monthly reports were missing the owner names and contact information for businesses that had applied for certification. One certifier regularly identified potential fraud among businesses to which it had denied certification, about one or two per month for 16 of the 34 reporting months included in our review. This certifier provided detailed narrative information in its reports to SBA about its concerns. The reporting format and level of detail reported also varied among certifiers. One certifier listed detailed information on its activities in a spreadsheet. Another described its activities using narrative text and an attached list of applicants for certification. One certifier included dates for certification, recertification, and the expiration of a certification, while other certifiers did not include this information. According to SBA officials, the agency did not have consistent procedures for reviewing monthly reports, including procedures to identify and resolve discrepancies in reports or oversee how certifiers collect and compile information transmitted to the agency. SBA officials said that one official, who recently retired, was responsible for reviewing all certifier monthly reports. Current officials and staff were not able to tell us what process this official used to assess the reports. Finally, with one person responsible for reviewing monthly reports until recently, SBA generally has not followed up on issues raised in reports. Agency officials told us that early in the program they found problems with the monthly report of one of the certifiers that indicated that the certifier did not understand program requirements and they contacted the certifier to address the issue. We found additional issues that would appear to warrant follow up from SBA. For example, two businesses were denied certification by one third-party certifier and approved shortly after by another. SBA stated that it had not identified these potential discrepancies but that it was possible for businesses to be deemed ineligible, resolve the issue preventing certification, and become eligible soon after. However, according to the program regulation, if a business was denied third-party certification and the owner believed the business eligible, the owner would have to request that SBA conduct an examination to verify its eligibility to represent the business as a WOSB. According to SBA officials, the agency was unaware of this business or its certification. And, as discussed previously, one certifier regularly identified potential fraud among businesses to which it had denied certification. SBA officials told us that they had not identified or investigated this certifier’s concerns about potential fraud. When we asked SBA officials how the agency addressed such concerns, an official responded that fraudulently entering into a set-aside contract was illegal and the business would be subject to prosecution. However, without SBA following up on these types of issues, it is unclear how businesses committing fraud in the program would be prosecuted. According to an SBA official, the agency has been developing written procedures to review the monthly reports, but has not yet estimated when the procedures would be completed. The procedures will be included in SBA’s SOP for the program, which also remains under development. As noted earlier, SBA could not estimate when it would complete the SOP. Without procedures in place to consistently review monthly reports and respond to problems identified in those reports, SBA lacks information about the activities and performance of third-party certifiers and leaves concerns raised by certifiers unaddressed. SBA’s methods to verify the eligibility of businesses in its WOSB program repository include annual examinations of businesses that received set- aside contracts. SBA’s program responsibilities include conducting eligibility examinations of WOSBs and EDWOSBs, according to SBA’s compliance guide for the WOSB program and its regulation. Section 8(m) of the Small Business Act sets forth eligibility criteria businesses must meet to receive a contract under the WOSB program set-aside. SBA examines a sample of businesses with a current attestation in SAM and that received a contract during SBA’s examination year. SBA does not include in its sample businesses that had not yet obtained a WOSB program contract. According to SBA officials, staff conducting the eligibility examination review the documents each business owner uploaded to the WOSB program repository to support the representation in SAM of eligibility for WOSB or EDWOSB status. For example, agency officials said that reviewers ensure that all documents required have been uploaded and review the contents of the documents to ensure that a business is eligible. SBA said staff conducting the examination then determine that the business has met the requirements to document its status as a WOSB, or determine that information is missing or not consistent with the program requirements and the business is not eligible at the time of SBA’s review to certify itself as a WOSB. SBA officials said the agency also uses the same process to investigate the eligibility of businesses on an ad hoc basis in response to referrals from contracting agencies or other parties, such as other businesses, that question the eligibility of a business. If a business has not sufficiently documented its eligibility representation, SBA sends a letter directing the business to enter required information or documents into the repository or remove its attestation of program eligibility in SAM within 15 days. If SBA receives no response after 15 days, it sends a second letter instructing the business to remove its WOSB attestation in SAM within 5 days. In 2012 and 2013, SBA sent final 5-day letters to 44 businesses identified through annual examinations or examinations following a referral. If the business does not do so, it may be subject to enforcement actions including suspension or debarment from federal contracting or criminal penalties, according to SBA officials. An SBA official said that the agency is unaware of any such enforcement actions as part of the WOSB program. SBA also decides protests from contracting agency staff or any other interested parties relating to a business’s eligibility. SBA considers protests if there is sufficient, credible evidence to show that the business may not be at least 51 percent owned and controlled by one or more women, or if the business has failed to provide documents required to establish eligibility for the program. Once SBA has obtained a protest, it examines documents submitted in the case, makes a determination of program eligibility based on the content of these documents and notifies relevant parties—typically, the contracting officer, protester (if not the same), and the business—of the determination. If eligible for the set- aside, the contracting officer may make an award to the business. Otherwise, the contracting officer may not award a contract to the business in question. From program implementation in April 2011 through July 2, 2014, SBA responded to 27 protests, and in 7 protests the businesses involved were found to be ineligible for the WOSB program. In the remaining protests, the businesses were found eligible, the party that filed the protest withdrew it, or SBA dismissed the protest. As described earlier in the report, contracting officers check for the presence of documents in the repository when making a WOSB program award. This could be considered part of SBA’s framework to oversee certifications, but the requirement for contracting officers to review documents is limited to ensuring that businesses have uploaded documents listed in the regulation. Representatives from some of the contracting offices we interviewed believed that they had to assess the validity of the documents or did not think they had the necessary qualifications to assess the documents. However, program guidance does not require contracting officers to assess the validity of these documents, and SBA officials told us contracting officers are not expected to evaluate the eligibility of businesses. SBA activities relating to eligibility verifications, particularly examinations, have several weaknesses. For instance, SBA has not yet developed procedures to conduct annual eligibility examinations although such efforts are in process, according to officials; has not evaluated the results of the eligibility examinations in the context of how the actions of businesses, contracting agencies, and third-party certifiers may have contributed to the high levels of incomplete and inaccurate documentations found in examinations; and has not assessed its internal controls or made procedural changes in response to the findings of its eligibility examinations. According to federal standards for internal control, agencies should have documented procedures, conduct monitoring, and ensure that any review findings and deficiencies are brought to the attention of management and are resolved promptly. Corrective action is to be taken or improvements made within established time frames to resolve the matters brought to management’s attention. Also, management needs to comprehensively identify risks the agency faces from both internal and external sources, and management should consider all significant interactions between the agency and all other parties. SBA conducted annual eligibility examinations in 2012 and 2013 on a sample of businesses that received contracts under the WOSB program and found that 42 percent of businesses in the 2012 sample were ineligible for WOSB program contract awards on the date of its review, and 43 percent in the 2013 sample were ineligible. According to SBA officials, both self- and third-party certified businesses were found ineligible at the time of review. SBA staff reviewed the documents that each business in its sample had posted to the program repository to ensure the businesses had sufficiently supported their attestations as required in program regulations. However, SBA could not provide documentation of a consistent procedure to examine each business. SBA staff reviewing documentation in the repository did not have guidelines describing how to conduct each review. SBA officials told us that they have been developing written procedures to conduct annual eligibility examinations, estimated a completion date that the agency did not meet, and that the agency does not have an estimation of completion. SBA officials explained that they determined the eligibility of businesses on a given date after the business received a contract. According to SBA officials, a finding of ineligibility does not mean the business was ineligible at the time of contract award because the status of the business might have changed. Although SBA officials did not know whether businesses examined were eligible at the time of award, the high rate of ineligibility it found raises questions about whether contracts may have been awarded to ineligible businesses. According to SBA officials, information in its repository constantly changes and SBA has yet to determine how or if a business was eligible when it received a WOSB set-aside contract. SBA officials told us that they believe they may be able to make such a determination but could not describe exactly how they would conduct the review or confirm that the business was an eligible WOSB or EDWOSB at the time of award. As part of its annual examination, SBA only examines businesses at some time after the business received a contract and, therefore, SBA’s examination is limited in its ability to identify potentially ineligible businesses prior to a contract award. SBA officials said that after the annual examinations they did not institute new controls to guard against ineligible businesses receiving program contracts because they described the examinations and the results as a method to gain insight about the program—specifically, that WOSBs may lack understanding of program eligibility requirements—and not a basis for change in oversight procedures. According to SBA officials, the levels of ineligibility found during the examinations were similar to those found in examinations of its other socioeconomic programs. SBA officials said businesses were deemed ineligible because they did not understand the documentation requirements for establishing eligibility and also attributed the ineligibility of third-party certified businesses to improper uploading of documents by the businesses themselves. SBA officials said they needed to make additional efforts to train businesses to properly document their eligibility. However, SBA officials could not explain how they had determined lack of understanding was the cause of ineligibility among businesses and have not made efforts to confirm that this was the cause. As a result, they have missed opportunities to obtain meaningful insights into the program. SBA regarded the bid protest as means of identifying ineligibility. SBA officials referred to the program as a self-policing program, because of the bid protest function through which competing businesses, contracting officers, or SBA can protest a business’s claim to be a WOSB or EDWOSB and eligible for contract awards under the program. In addition, an SBA official stated that business owners affirm their status when awarded a contract and are subject to prosecution if they had done so and later were found to have been ineligible at the time of contract award—which the official considered a program safeguard. However, without (1) developing program eligibility controls that include procedures for conducting annual eligibility examinations; (2) analyzing the results of the examinations to understand the underlying causes of ineligibility; (3) developing new procedures for examinations, including expanding the sample of businesses to be examined to include those that did not receive contracts; and (4) investigating businesses based on examination results, SBA may continue to find high rates of ineligibility among businesses registered in the WOSB program repository. In turn, this would continue to expose the program to the risk that ineligible businesses may receive set-aside contracts. Also, by reviewing the eligibility of businesses that have not received program contracts, SBA may improve the quality of the pool of potential program award recipients. Set-asides under the WOSB program to date have had a minimal effect on overall contracting obligations to WOSBs and attainment of WOSB contracting goals. WOSB program set-aside obligations increased from fiscal year 2012 to fiscal year 2013. The Department of Defense (DOD), the Department of Homeland Security (DHS), and the General Services Administration (GSA) accounted for the majority of these obligations. The WOSB program set-asides represented less than 1 percent of total federal awards to women-owned small businesses. Contracting officers, WOSBs, and others with whom we spoke suggested a number of program changes that might increase use of the WOSB program, including increasing awareness, allowing for sole-source awards, and expanding the list of eligible industries for the set-aside program. WOSB program set-aside obligations increased from fiscal year 2012 to fiscal year 2013. Obligations to WOSBs under the WOSB set-aside program increased from $33.3 million in 2012 to $39.9 million in 2013, and obligations to EDWOSBs increased from $39.2 million in 2012 to $60.0 million in 2013. The National Defense Authorization Act for Fiscal Year 2013 removed the dollar cap on contract awards eligible under the WOSB set-aside program, which may account for some of the increase in obligations from 2012 to 2013. SBA officials told us that they expect increased use of the program in the future as a result of this change. As shown in table 2, three federal agencies—DOD, DHS, and GSA— collectively accounted for the majority of the obligations awarded under the set-aside program. DOD (Air Force, Army, Navy, and all other defense agencies) accounted for 62.2 percent of obligations, DHS for 10.7 percent, and GSA for 4.0 percent of obligations. No other individual agency accounted for more than 3.4 percent of obligations awarded under the program. From April 2011 through May 2014, WOSB program set-asides constituted a very small percentage (0.44 percent) of all the contracting obligations awarded to WOSBs (see fig. 1). The majority of obligations awarded to WOSBs were made under other, longer-established set-aside programs. For example, if eligible, a WOSB could receive a contracting award under the 8(a), HUBZone, or SDVOSBC programs, or through a general small business set-aside. WOSBs also can obtain federal contracts without set-asides (through open competition). Based on our analysis of FPDS-NG data of federal contracting agencies, contract obligations awarded through the WOSB set-aside totaled $228.9 million, or 0.44 percent, of the $52.6 billion in contract obligations awarded to WOSBs from April 2011 through May 2014. Additionally, the WOSB set-aside has had relatively little impact on federal agency achievement of goals for contracting to WOSBs, because the program set-asides represent a very small percentage of all contracting awards to WOSBs. Since 2011, the overall percentage of contracting obligations awarded to WOSBs (through any program or open competition) has remained below the government-wide goal of 5 percent (see table 3). Goal achievement by the three contracting agencies with the highest amount of obligations through the set-aside program varied. For example, DOD did not meet its 5 percent goal for contracting obligations to WOSBs in any of the 3 years. DHS and GSA met their goals in all 3 years. Excluding obligations made by DOD, about 5.7 percent of total federal contracting obligations to small businesses included in SBA’s fiscal year 2013 Small Business Goaling Report were awarded to WOSBs. For the 24 agencies subject to the Chief Financial Officers Act listed in SBA’s scorecards, 19 met their WOSB contracting goal in fiscal year 2012 and 20 met their goal in fiscal year 2013. One agency missed its goal in fiscal year 2012 but met its goal in fiscal year 2013. Four agencies (the same four each year) did not meet their goal for either year. Selected federal contracting officials, businesses that received a WOSB or EDWOSB set-aside, third-party certifiers, and a WOSB advocacy organization with which we spoke gave their perspectives on existing challenges and possible changes to increase program usage. Complexity and burdensome requirements. Contracting officers described challenges to using the WOSB set-aside. Some contracting officers noted that generally, all contracts awarded to WOSBs count for the purposes of meeting agencies’ 5 percent goal and that from their perspective it does not matter whether a contract is awarded to a WOSB using the WOSB program, another set-aside program, or open competition. Some contracting officers said that WOSB program requirements were burdensome or complex relative to other SBA programs with set-asides. Unlike the other programs, the WOSB program requires the use of a separate electronic repository, maintained by SBA, to collect and store certification documents. One contracting officer noted that the contracting process slowed when officials had to seek information from the repository. Another contracting officer told us the role of the contracting officer included confirming that businesses had uploaded required documents in the SBA repository based on a list of required documents in the program regulation—but noted this task was not required under other contracting programs. Lack of awareness and agency commitment. Representatives from advocacy groups also identified awareness of and commitment to the program as another area for improvement. An advocacy group representative told us that some of their member WOSBs had encountered confusion and reluctance on the part of contracting officers to use the program. Another advocacy group said that SBA should engender more commitment to the program among contracting officers and agencies. Another representative noted that there are no consequences for agency leaders for failure to meet contracting goals for WOSBs or use the set-aside program. SBA officials described to us consequences that included a low rating in the publicly available SBA contracting scorecard, which may draw negative attention to the agency. Also, the National Defense Authorization Act for Fiscal Year 2013 includes the extent to which agencies meet contracting goals as a competency by which members of the senior executive service are rated. All of the businesses we interviewed that received WOSB program contracts cited the need for increased agency outreach or awareness of the program. For example, one participant advocated increasing contracting officer awareness and understanding of how an agency could benefit from using the WOSB set-aside program. Changes to increase use of program. Contracting officers also identified changes they believe could increase use of the WOSB set- aside. For example, some noted that allowing sole-source contracts could increase program use. Currently, contract officers can establish a set- aside only if there is a reasonable expectation that at least two eligible WOSBs will submit a bid for the contract. Some contracting officers suggested expanding the list of North American Industry Association Classification System (NAICS) codes eligible for use under the WOSB set-aside. For example, one contracting office said that the designated NAICS for the set-aside program did not meet their procurement needs. One representative pointed out that SBA had designated some NAICS codes just for EDWOSB and others for WOSBs. SBA officials told us the agency does not have the authority to change the list of industry sectors eligible for program set-asides without conducting a study of industries in which WOSBs were underrepresented or substantially underrepresented. Representatives from all of the WOSB advocacy groups, three of which are also third-party certifiers, said that expanding the NAICS codes would improve the program. For example, one advocacy group said that certain WOSBs would like to obtain WOSB or EDWOSB set-asides but did not have NAICS codes that were listed as eligible. Another said that they would not limit the number of eligible industries under the program. Finally, the businesses we interviewed also believed that allowing sole- source awards or adding more NAICS codes would increase program use. Six participants commented on the limitations for awarding sole- source contracts through the WOSB set-aside. Five participants felt that the NAICS codes under the program were limited. One program participant mentioned that she felt that limiting set-asides for the WOSB program to certain NAICS codes was inconsistent with other SBA programs with set-asides, such as 8(a), HUBZone, and SDVOSBC. She gave an example of an agency that issued a draft solicitation that sought to award two contracts each to WOSB set-asides, HUBZone, and SDVOSBC businesses. However when it became clear that the contract was not in an eligible NAICS code for the WOSB program, the agency converted the two contracts intended for WOSB set-aside to a general small business category. Some program participants also mentioned positive aspects of the program. Five participants believed that the program provided greater opportunities for their businesses and WOSBs in general. Furthermore, five of the six businesses with whom we spoke that received only one or two contracts felt that the program improved their ability to compete for a federal contract. For example, one participant noted that while she has not seen many set-aside solicitations for the NAICS code under which her business primarily operates, the existence of the program prompted her to bid on set-asides under other NAICS codes. As the only federal procurement set-aside specifically for women-owned businesses, the WOSB program could play an important role in limiting competition to certain federal contracts for WOSBs and EDWOSBs that are underrepresented in their industries. However, weaknesses in multiple areas of SBA’s management of the program hinder effective oversight of the WOSB program. Specifically, SBA has limited information about the performance of its certifiers and does not use what information is available to help ensure certifiers adhere to program requirements, a deficiency exacerbated by the highest-volume certifier’s—about 76 percent of third-party certifications—delegation of duties to 14 partner organizations. An incomplete response to SBA’s request for information on WBENC’s certification process demonstrates the need for an oversight framework to ensure that certifiers adhere to agreements with SBA. SBA did not follow up on the incomplete response from WBENC, which raises questions about SBA’s commitment to oversight of the certifiers. Furthermore, the lack of procedures for review and analysis of monthly certifier reports means that SBA has forgone opportunities to oversee certifiers and pursue concerns about fraud of individual businesses identified by one certifier. According to federal standards for internal control, agencies should conduct control activities such as performance reviews and clearly document internal controls. Formalizing existing ad hoc processes (by developing procedures) will help SBA obtain the information necessary to better ensure that third-party certifiers fulfill the requirements of their agreements with SBA—an effort SBA said it plans to undertake, although it has not estimated a completion date. Additionally, SBA could use results and insights from reviews of certifier reports— which are to include concerns about businesses—to inform its processes for eligibility verification, particularly examinations. Weaknesses related to SBA’s examination of program participants and approach to enforcement mean that the agency cannot offer reasonable assurance that only eligible businesses participate in the program. Although the agency’s examinations found high rates of ineligibility, SBA has not yet formalized examination guidance for staff or followed up on examination results to determine the status of ineligible businesses at the time of contract award. SBA also has not focused on identifying factors that may be causing businesses to be found ineligible; rather, the agency appears to have determined that more training for businesses about eligibility requirements could address the issue. However, training alone would be a limited response to examination results, and SBA officials could not say what analysis determined training to be the relevant response. Additionally, the sample of businesses that SBA examines includes only those businesses that received WOSB set-aside contracts. All these factors limit SBA’s ability to better understand the eligibility of businesses before applying for and being awarded contracts. Rather than gather and regularly analyze information related to program eligibility, SBA relies on other parties to identify potential misrepresentation of WOSB status (through bid-protest filings and less formal mechanisms)—a reactive and limited approach to oversight. Federal standards for internal control state that agencies should have documented procedures, conduct monitoring, and ensure that any review findings and deficiencies are brought to the attention of management and are resolved promptly. Additionally, the standards state that management needs to comprehensively identify risks the agency faces from both internal and external sources. By expanding its examination of firms and analyzing and following up on the results, SBA could advance the key program goal of restricting competition for set-aside contracts to WOSBs and EDWOSBs. We make the following recommendations to improve management and oversight of the WOSB program. To help ensure the effective oversight of third-party certifiers, the Administrator of SBA should establish and implement comprehensive procedures to monitor and assess performance of certifiers in accord with the requirements of the third-party certifier agreement and program regulations. To provide reasonable assurance that only eligible businesses obtain WOSB set-aside contracts, the Administrator of SBA should enhance examination of businesses that register to participate in the WOSB program, including actions such as: promptly completing the development of procedures to conduct annual eligibility examinations and implementing such procedures; analyzing examination results and individual businesses found to be ineligible to better understand the cause of the high rate of ineligibility in annual reviews, and determine what actions are needed to address the causes; and implementing ongoing reviews of a sample of all businesses that have represented their eligibility to participate in the program. We provided a draft of this report to SBA, DHS, DOD, and GSA for review and comment. SBA provided written comments that are described below and reprinted in appendix II. The other agencies—DHS, DOD, and GSA—did not provide comments on this report. SBA generally agreed with our recommendations and said that the agency is already in the process of implementing many of our recommendations. While SBA generally agreed with our recommendations, the agency stated that the report could be clearer about the program examination process. Specifically, SBA stated that the agency has authority to conduct eligibility examinations at any time for any firm asserting eligibility to receive WOSB program contracts. We have added information to the draft to clarify this point. The draft report we sent to SBA for comment discussed the agency’s process of conducting annual eligibility examinations and provided a description of SBA’s current process. SBA also stated that “the report recommends that SBA conduct ongoing annual eligibility examinations and implement such procedures.” However, our report recommends that SBA complete the development of procedures to conduct annual eligibility examinations (which SBA has conducted for the past 2 years) and implement such procedures. We separately recommend implementing ongoing reviews of a sample of all businesses that have represented their eligibility to participate in the program. We do not specify that these eligibility reviews, which are eligibility examinations, should be annual. SBA could choose to conduct these reviews more frequently if deemed appropriate. Whether SBA conducts eligibility examinations annually or more frequently, examinations should be consistently conducted by following written procedures and the results assessed to determine the causes of ineligibility. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to appropriate congressional committees and members, the Secretary of DOD, the Secretary of DHS, the Administrator of GSA, the Administrator of SBA, and other interested parties. This report will also be available at no charge on our website at http://www.gao.gov. If you or your staff have any questions concerning this report, please contact me at (202) 512-8678 or shearw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. This report examines the Women-Owned Small Business (WOSB) program of the Small Business Administration (SBA). More specifically, the report (1) describes how WOSBs and economically disadvantaged WOSBs (EDWOSBs) are certified as eligible for the program, (2) examines the extent to which SBA has implemented internal control and oversight procedures of WOSB program certifications, and (3) discusses the effect the program has had on federal contracting opportunities available to WOSBs or EDWOSBs. To describe how businesses are certified as eligible for the program, we reviewed SBA policies and procedures to establish program eligibility including the responsibilities of businesses, third-party certifiers, contracting officers, and SBA. We interviewed SBA officials from the Office of Government Contracting. To evaluate how certification procedures may affect program participation, we obtained from SBA monthly reports (from September 2011 through May 2014) from each of the four third-party certifiers. We took steps to develop a dataset we could use for our analyses, including creating and merging monthly spreadsheets, identifying missing business names, and clearing the list of duplicate entries. We compared this dataset with Federal Procurement Data System-Next Generation (FPDS-NG) data for businesses that received a WOSB program set-aside contract. We determined that the data on how many third-party certified businesses received contracts as part of the WOSB program were sufficiently reliable for our purposes by corroborating a sample of businesses we identified as third-party certified with documentation for the businesses in the WOSB program repository. We were not able to determine how many self-certified businesses obtained contracts under the program, because the format of the documentation maintained in the SBA repository does not include a record of documents that were present at the time of contract award. We also interviewed a sample of contracting officers from selected components in the Department of Defense (DOD), Department of Homeland Security (DHS), and the General Services Administration (GSA). We selected these three agencies to represent a range of program participation based on the number and total obligation amounts of active set-aside contracts awarded in 2011 through 2013. Within DOD and DHS, we selected two components from each that demonstrated high- and mid-level program participation (based on number of contracts and obligation amounts). For DOD, we selected the U.S. Army and Defense Logistics Agency. For DHS, we selected the U.S. Coast Guard, and Customs and Border Protection. Within each of the components and GSA, we compared FPDS-NG data on program activity by obligation amount, contract number, and North American Industry Classification System (NAICS) codes for 2011 through 2013. For each, we selected two contracting offices using the same criteria we used to select agencies, which included identifying a high- and mid-level program obligation amount and offices with multiple contracts and under multiple NAICS codes. We excluded one Customs and Border Patrol office because only one office awarded multiple contracts under multiple NAICS codes. We also interviewed three of the four SBA-approved third-party certifiers (the El Paso Hispanic Chamber of Commerce, the National Women Business Owners Corporation, and the U.S. Women’s Chamber of Commerce). We were unable to interview the Women’s Business National Enterprise Council (WBENC). SBA requested documentation of WBENC’s oversight procedures for the certification activity and fee structures of its regional partner organizations. WBENC provided a written response to SBA, which was not fully responsive to the request, as discussed in the report. We conducted semi-structured interviews with a sample of 10 businesses that were certified for the program, 9 of which had received a set-aside contract. To evaluate SBA’s oversight of certification, we reviewed the program regulation and program documents, agreements with third-party certifiers, 135 monthly reports submitted by all four third-party certifiers, and letters SBA sends to inform businesses when their WOSB or EDWOSB status is in question, among other documents. We discussed the agency’s procedures to monitor certifiers and ensure participant eligibility with SBA officials from the Office of Government Contracting. We compared officials’ descriptions of their oversight activities with federal internal control standards. We inquired about documentation and eligibility examinations conducted in 2012 and 2013, and a planned examination for 2014, and reviewed reports of the 2012 and 2013 examination results. We also inquired about ongoing plans to develop a standard operating procedure, and future plans to evaluate the program. To determine what effect, if any, the WOSB program has had on federal contracting opportunities available to WOSBs, we identified set-aside contract obligations in FPDS-NG from April 2011 through May 2014 to identify trends in program participation by contracting agencies included in both FPDS-NG and SBA goaling reports. Using a review of FPDS-NG documentation and electronic edit checks, we deemed these data sufficiently reliable for our purposes. We also analyzed SBA goaling reports from 2011 through 2013 to describe progress made towards meeting the 5 percent goal for federal contracting to WOSBs. We conducted semi-structured interviews with a sample of 10 businesses there were certified for the program, 9 of which had received a set-aside contract. We selected this nongeneralizable sample of businesses to reflect whether they had been certified by a third-party entity, or had self- certified. While the results of these interviews could not be generalized to all WOSB program participants, they provided insight into the benefits and challenges of the program. We interviewed SBA officials and contracting agency officials about the extent to which the program has met its statutory purpose of increasing contracting opportunities for WOSBs. Finally, we interviewed industry advocates, including three of the four third-party certifiers (the El Paso Hispanic Chamber of Commerce, the National Women Business Owners Corporation, and U.S. Women’s Chamber of Commerce) and one other industry advocate (Women Impacting Public Policy) actively involved in promoting the program with WOSBs. We conducted this performance audit from August 2013 to October 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. In addition to the contact named above, Andrew Pauline (Assistant Director), Julie Trinder-Clements (analyst-in-charge), Pamela Davidson, Daniel Kaneshiro, Julia Kennon, Barbara Roesmann, Jessica Sandler, and Jena Sinkfield made key contributions to this report.

Summary: In 2000, Congress authorized the WOSB program to increase contracting opportunities for WOSBs by allowing contracting officers to set aside procurements to such businesses. SBA, which administers the program, issued implementing regulations that became effective in 2011. GAO was asked to review the WOSB program. This report examines (1) how businesses are certified as eligible for the WOSB program, (2) SBA's oversight of certifications, and (3) the effect the program has had on federal contracting opportunities available to WOSBs or EDWOSBs. GAO reviewed relevant laws, regulations, and program documents; analyzed federal contracting data from April 2011 through May 2014; and interviewed SBA, officials from contracting agencies selected to obtain a range of experience with the WOSB program, third-party certifiers, WOSBs, and organizations that represent their interests. Businesses have two options to certify their eligibility for the women-owned small business (WOSB) program. Whether self-certifying at no cost or using the fee-based services of an approved third-party certifier, businesses must attest that they are a WOSB or an economically disadvantaged WOSB (EDWOSB). Businesses also must submit documents supporting their attestation to a repository the Small Business Administration (SBA) maintains (required documents vary depending on certification type), and, if they obtain a third-party certification, to the certifier. SBA performs minimal oversight of third-party certifiers and has yet to develop procedures that provide reasonable assurance that only eligible businesses obtain WOSB set-aside contracts. For example, SBA generally has not reviewed certifier performance or developed or implemented procedures for such reviews, including determining whether certifiers inform businesses of the no-cost self-certification option, a requirement in the agency's agreement with certifiers. SBA also has not completed or implemented procedures to review the monthly reports that third-party certifiers must submit. Without ongoing monitoring and oversight of the activities and performance of third-party certifiers, SBA cannot reasonably assure that certifiers fulfill the requirements of the agreement. Moreover, in 2012 and 2013, SBA found that more than 40 percent of businesses (that previously received contracts) it examined for program eligibility should not have attested they were WOSBs or EDWOSBs at the time of SBA's review. SBA officials speculated about possible reasons for the results, including businesses not providing adequate documentation or becoming ineligible after contracts were awarded, but SBA has not assessed the results of the examinations to determine the actual reasons for the high numbers of businesses found ineligible. SBA also has not completed or implemented procedures to conduct eligibility examinations. According to federal standards for internal control, agencies should have documented procedures, conduct monitoring, and ensure that any review findings and deficiencies are resolved promptly. As a result of inadequate monitoring and controls, potentially ineligible businesses may continue to incorrectly certify themselves as WOSBs, increasing the risk that they may receive contracts for which they are not eligible. The WOSB program has had a limited effect on federal contracting opportunities available to WOSBs. Set-aside contracts under the program represent less than 1 percent of all federal contract obligations to women-owned small businesses. The Departments of Defense and Homeland Security and the General Services Administration collectively accounted for the majority of the $228.9 million in set-aside obligations awarded under the program between April 2011 and May 2014. Contracting officers, business owners, and industry advocates with whom GAO spoke identified challenges to program use and suggested potential changes that might increase program use, including allowing sole-source contracts rather than requiring at least two businesses to compete and expanding the list of 330 industries in which WOSBs and EDWOSBs were eligible for a set-aside. GAO recommends that SBA, among other things, establish and implement procedures to monitor certifiers and improve annual eligibility examinations, including by analyzing examination results. SBA generally agreed with GAO's recommendations.