Federal agencies conduct a variety of procurements that are reserved for small business participation through small business set-asides. The set- asides can be for small businesses in general, or they can be specific to small businesses that meet additional eligibility requirements in the Service-Disabled Veteran-Owned Small Business (SDVOSB), Historically Underutilized Business Zone (HUBZone), 8(a) Business Development (8(a)), and WOSB programs. The WOSB program enables federal contracting officers to identify and establish a sheltered market, or set-aside, for competition among WOSBs and EDWOSBs in certain industries. To determine the industries eligible under the WOSB program, SBA is required to conduct a study to determine which NAICS codes are eligible under the program and to report on such studies every 5 years. WOSBs can receive set-asides in industries in which SBA has determined that women-owned small businesses are substantially underrepresented. EDWOSBs can receive set-asides in WOSB-eligible industries as well as in an additional set of industries in which SBA has determined that women-owned small businesses are underrepresented but not substantially so. As of February 2019, there were a total of 113 four-digit NAICS codes (representing NAICS industry groups) eligible under the WOSB program—92 eligible NAICS codes for WOSBs and 21 for EDWOSBs. Additionally, businesses must be at least 51 percent owned and controlled by one or more women who are U.S. citizens to participate in the WOSB program. The owner must provide documents demonstrating that the business meets program requirements, including a document in which the owner attests to the business’s status as a WOSB or EDWOSB. EDWOSBs are WOSBs that are controlled by one or more women who are citizens and who are economically disadvantaged in accordance with SBA regulations. According to SBA, as of early October 2018, there were 13,224 WOSBs and 4,488 EDWOSBs registered in SBA’s online certification database. SBA’s Office of Government Contracting administers the WOSB program by promulgating regulations, conducting eligibility examinations of businesses that receive contracts under a WOSB or EDWOSB set-aside, deciding protests related to eligibility for a WOSB set-aside, conducting studies to determine eligible industries, and working with other federal agencies in assisting WOSBs and EDWOSBs. According to SBA officials, the Office of Government Contracting also works at the regional and local levels with SBA’s Small Business Development Centers and district offices, and with other organizations (such as Procurement Technical Assistance Centers), to help WOSBs and EDWOSBs obtain contracts with federal agencies. The services SBA coordinates include training, counseling, mentoring, facilitating access to information about federal contracting opportunities, and business financing. According to SBA, as of October 2018, there were two full-time staff within the Office of Government Contracting whose primary responsibility was the WOSB program. Initially, the program’s statutory authority allowed WOSBs to be self- certified by the business owner or certified by an approved third-party national certifying entity as eligible for the program. Self-certification is free, but some third-party certification options require businesses to pay a fee. Each certification process requires businesses to provide signed representations attesting to their WOSB or EDWOSB eligibility. Businesses must provide documents supporting their status before submitting an offer to perform the requirements of a WOSB set-aside contract. In August 2016, SBA launched certify.sba.gov, which is an online portal that allows firms to upload required documents and track their submission and also enables contracting officers to review firms’ eligibility documentation. According to the Federal Acquisition Regulation (FAR), contracting officers are required to verify that all required documentation is present in the online portal when selecting a business for an award. In addition, businesses must register and attest to being a WOSB in the System for Award Management, the primary database of vendors doing business with the federal government. In 2011, SBA approved four organizations to act as third-party certifiers: El Paso Hispanic Chamber of Commerce, NWBOC (previously known as the National Women Business Owners U.S. Women’s Chamber of Commerce, and Women’s Business Enterprise National Council. These organizations have been the WOSB program’s third-party certifiers since 2011. According to SBA data, the Women’s Business Enterprise National Council was the most active third-party certifier in fiscal year 2017—performing 2,638 WOSB certification examinations. The U.S. Women’s Chamber of Commerce, NWBOC, and El Paso Hispanic Chamber of Commerce—completed 644, 105, and 12 certifications, respectively. As discussed previously, in 2014 we reviewed the WOSB program and found a number of deficiencies in SBA’s oversight of the four SBA- approved third-party certifiers and in SBA’s eligibility examination processes and we made related recommendations for SBA. In addition, in 2015 and 2018 the SBA OIG reviewed the WOSB program and also found oversight deficiencies, including evidence of WOSB contracts set aside for ineligible firms. In both reports, the SBA OIG also made recommendations for SBA. Further, in July 2015, we issued GAO’s fraud risk framework, which provides a comprehensive set of key components and leading practices that serve as a guide for agency managers to use when developing efforts to combat fraud in a strategic, risk-based way. In July 2016, the Office of Management and Budget issued guidelines requiring executive agencies to create controls to identify and respond to fraud risks. These guidelines also affirm that managers should adhere to the leading practices identified in GAO’s fraud risk framework. As of February 2019, SBA had implemented one of the three changes that the 2015 NDAA made to the WOSB program—sole-source authority. The two other changes—authorizing SBA to implement its own certification process for WOSBs and requiring SBA to eliminate the WOSB self-certification option—have not been implemented. The 2015 NDAA did not require a specific time frame for SBA to update its regulations. SBA officials have stated that they will not eliminate self- certification until the new certification process for the WOSB program is in place, which they expect to be completed by January 1, 2020. In September 2015, SBA published a final rule to implement sole-source authority for the WOSB program (effective October 2015). Among other things, the rule authorized contracting officers to award a contract to a WOSB or EDWOSB without competition, provided that the contracting officer’s market research cannot identify two or more WOSBs or EDWOSBs in eligible industries that can perform the requirements of the contract at a fair and reasonable price. In the final rule, SBA explained that it promulgated the sole-source rule before the WOSB certification requirements for two reasons. First, the sole-source rule could be accomplished by simply incorporating the statutory language into the regulations, whereas the WOSB certification requirements would instead require a prolonged rulemaking process. Second, SBA said that addressing all three regulatory changes at the same time would delay the implementation of sole-source authority. SBA described the sole-source mechanism as an additional tool for federal agencies to ensure that women-owned small businesses have an equal opportunity to participate in federal contracting and to ensure consistency among SBA’s socioeconomic small business procurement programs. According to SBA, most of the 495 comments submitted about the sole- source rule supported the agency’s decision to implement the authority quickly. However, the SBA OIG’s June 2018 audit report cautioned that allowing sole-source contracting authority while firms can still self-certify exposes the WOSB program to unnecessary risk of fraud and abuse, and the report recommended that SBA implement a new certification process for the WOSB program per the 2015 NDAA. In addition, our previous report identified risks of program participation by ineligible firms associated with deficiencies in SBA’s oversight structure. As we discuss in detail later, SBA has still not addressed these risks, which may be exacerbated by the implementation of sole-source authority without addressing the other changes made by the 2015 NDAA, including eliminating the self-certification option. As of February 2019, SBA had not published a proposed rule for public comment to establish a new certification process for the WOSB program. Previously, in October 2017, an SBA official stated that SBA was about 1–2 months away from publishing a proposed rule. However, in June 2018, SBA officials stated that a cost analysis would be necessary before the draft could be sent to the Office of Management and Budget for review. Certain stages of the rulemaking process have mandated time periods, such as the required interagency review process for certain rules. In June 2017, we reported that SBA officials said that an increase in the number of statutorily mandated rules in recent years had contributed to delays in the agency’s ability to promulgate rules in a more timely fashion. As of February 2019, SBA had not provided documentation or time frames for issuing a proposed rule or completing the rulemaking process. However, in response to the SBA OIG recommendation that SBA implement the new certification process, SBA stated that it would fulfill the recommendation (meaning implement a new certification process) by January 1, 2020. In December 2015, SBA published an advance notice of proposed rulemaking to solicit public comments to assist the agency with drafting a proposed rule to implement a new WOSB certification program. In the notice, SBA stated that it intends to address the 2015 NDAA changes, including eliminating the self-certification option, through drafting regulations to implement a new certification process. Previously, in its September 2015 final rule implementing sole-source authority, SBA stated that there was no evidence that Congress intended that the existing WOSB program, including self-certification, be halted before establishing the infrastructure and new regulations for a new certification program. The advance notice requested comments on various topics, such as how well the current certification processes were working, which of the certification options were feasible and should be pursued, whether there should be a grace period for self-certified WOSB firms to complete the new certification process, and what documentation should be required. Three third-party certifiers submitted comments in response to the advance notice of proposed rulemaking, and none supported the option of SBA acting as a WOSB certifier. One third-party certifier commented that such an arrangement is a conflict of interest given that SBA is also responsible for oversight of the WOSB program, and two certifiers commented that SBA lacked the required resources. The three third-party certifiers also asserted in their comments that no other federal agency should be allowed to become an authorized WOSB certifier, with one commenting that federal agencies should instead focus on providing contracting opportunities for women-owned businesses. All three certifiers also proposed ways to improve the current system of third-party certification—for example, by strengthening oversight of certifiers or expanding their number. The three certifiers also suggested that SBA move to a process that better leverages existing programs with certification requirements similar to those of the WOSB program, such as the 8(a) program. In the advance notice, SBA asked for comments on alternative certification options, such as SBA acting as a certifier or limiting WOSB program certifications to the 8(a) program and otherwise relying on state or third-party certifiers. Further, in June 2018, SBA officials told us that they were evaluating the potential costs of a new certification program as part of their development of the new certification rule. SBA has not fully addressed deficiencies in its oversight of third-party certifiers that we identified in our October 2014 report. We reported that SBA did not have formal policies for reviewing the performance of its four approved third-party certifiers, including their compliance with their agreements with SBA. Further, we found that SBA had not developed formal policies and procedures for, among other things, reviewing the monthly reports that certifiers submit to SBA. As a result, we recommended that SBA establish comprehensive procedures to monitor and assess the performance of the third-party certifiers in accordance with their agreements with SBA and program regulations. While SBA has taken some steps to address the recommendation, as of February 2019 it remained open. In response to our October 2014 recommendation, in 2016 SBA conducted compliance reviews of the four SBA-approved third-party certifiers. According to SBA, the purpose of the compliance reviews was to ensure the certifiers’ compliance with regulations, their signed third- party certifier certification form (or agreement) with SBA, and other program requirements. The compliance reviews included an assessment of the third-party certifiers’ internal certification procedures and processes, an examination of a sample of applications from businesses that the certifiers deemed eligible and ineligible for certification, and an interview with management staff. SBA officials said that SBA’s review team did not identify significant deficiencies in any of the four certifiers’ processes and found that all were generally complying with their agreements. However, one compliance review report described “grave concerns” that a third-party certifier had arbitrarily established eligibility requirements that did not align with WOSB program regulations and used them to decline firms’ applications. SBA noted in the report that if the third-party certifier failed to correct this practice SBA could terminate the agreement. As directed by SBA, the third-party certifier submitted a letter to SBA outlining actions it had taken to address this issue, among others. The final compliance review reports for the other third-party certifiers also recommended areas for improvement, including providing staff with additional training on how to conduct eligibility examinations and reviewing certification files to ensure they contain complete documentation. In addition, two of the three compliance review reports with recommendations (including the compliance review report for the certifier discussed above) required the certifier to provide a written response within 30 days outlining plans to correct the areas. SBA officials said that they reviewed the written responses and determined that no further action was required. In January 2017, SBA’s Office of Government Contracting updated its written Standard Operating Procedures (SOP) to include policies and procedures for the WOSB program, in part to address our October 2014 recommendation. The 2017 SOP discusses what a third-party-certifier compliance review entails, how often the reviews are to be conducted, and how findings are to be reported. The 2017 SOP notes that SBA may initiate a compliance review “at any time and as frequently as the agency determines is necessary.” In September 2018, SBA officials told us that they were again updating the SOP, in part to address deficiencies we identified in our prior work and during this review. However, as of February 2019, SBA had not provided an updated SOP. In addition, in April 2018, SBA finalized a WOSB Program Desk Guide that, according to SBA, is designed to provide program staff with detailed guidance for conducting oversight procedures, including compliance reviews of third-party certifiers. For example, the Desk Guide discusses how staff should prepare for a compliance review of a third-party certifier, review certification documents, and prepare a final report. However, the Desk Guide does not describe specific activities designed to oversee third-party certifiers on an ongoing basis. In November 2017, SBA officials told us that they planned to conduct additional compliance reviews of the third-party certifiers. However, in June 2018, officials said there were no plans to conduct further compliance reviews until the final rule implementing the new certification process was completed. Further, SBA officials said that the 2016 certifier compliance reviews did not result in significant deficiencies. However, as noted previously, one of the compliance review reports described a potential violation of the third-party certifier’s agreement with SBA. Per written agreements with SBA, third-party certifiers are required to submit monthly reports that include the number of WOSB and EDWOSB applications received, approved, and denied; identifying information for each certified business, such as the business name; concerns about fraud, waste, and abuse; and a description of any changes to the procedures the organizations used to certify businesses as WOSBs or EDWOSBs. In our October 2014 report, we noted that SBA had not followed up on issues raised in the monthly reports and had not developed written procedures for reviewing them. At that time, SBA officials said that they were unaware of the issues identified in the certifiers’ reports and that the agency was developing procedures for reviewing the monthly reports but could not estimate a completion date. In our interviews for this report, SBA officials stated that SBA still does not use the third-party certifiers’ monthly reports to regularly monitor the program. Specifically, SBA does not review the reports to identify any trends in certification deficiencies that could inform program oversight. Officials said the reports generally do not contain information that SBA considers helpful for overseeing the WOSB program, although staff sometimes use the reports to obtain firms’ contact information. SBA officials also said that staff very rarely receive information about potentially fraudulent WOSB firms from the third-party certifiers—maybe three firms per year—and that this information is generally received via email and not as part of the monthly reports. SBA officials said that when they receive information about potentially fraudulent firms, WOSB program staff conduct an examination to determine the firm’s eligibility and report the results back to the certifier. However, a third-party certifier told us it has regularly reported firms it suspected of submitting potentially fraudulent applications in its monthly reports and that SBA has not followed up with them. In addition, two third-party certifiers said that if SBA is not cross-checking the list of firms included in their monthly reports, a firm deemed ineligible by one certifier may submit an application to another certifier and obtain approval. The three third-party certifiers we spoke with said that SBA generally had not communicated with them about their implementation of the program since the 2016 compliance reviews. However, SBA officials noted that three of the four third-party certifiers attended an SBA roundtable in March 2017 to discuss comments on the proposed rulemaking. In addition, SBA officials said that the third-party certifiers may contact them with questions about implementing the WOSB program, but SBA generally does not reach out to them. Although SBA has taken steps to enhance its written policies and procedures for oversight of third-party certifiers, it does not have plans to conduct further compliance reviews of the certifiers and does not intend to review certifiers’ monthly reports on a regular basis. SBA officials said that third-party certifier oversight procedures would be updated, if necessary, after certification options have been clarified in the final WOSB certification rule. However, ongoing oversight activities, such as regular compliance reviews, could help SBA better understand the steps certifiers have taken in response to previous compliance review findings and whether those steps have been effective. In addition, leading fraud risk management practices include identifying specific tools, methods, and sources for gathering information about fraud risks, including data on fraud schemes and trends from monitoring and detection activities, as well as involving relevant stakeholders in the risk assessment process. Without procedures to regularly monitor and oversee third-party certifiers, SBA cannot provide reasonable assurance that certifiers are complying with program requirements and cannot improve its efforts to identify ineligible firms or potential fraud. Further, it is unclear when SBA’s final rule will be implemented. As a result, we maintain that our previous recommendation should be addressed—that is, that the Administrator of SBA should establish and implement comprehensive procedures to monitor and assess the performance of certifiers in accordance with the requirements of the third-party certifier agreement and program regulations. SBA also has not fully addressed deficiencies found in our 2014 review related specifically to eligibility examinations. We found that SBA lacked formalized guidance for its eligibility examination processes and that the examinations continued to identify high rates of potentially ineligible businesses. As a result, we recommended that SBA enhance its examination of businesses that register for the WOSB program to ensure that only eligible businesses obtain WOSB set-asides. Specifically, we suggested that SBA consider (1) completing the development of procedures to conduct annual eligibility examinations and implementing such procedures; (2) analyzing examination results and individual businesses found to be ineligible to better understand the cause of the high rate of ineligibility in annual reviews and determine what actions are needed to address the causes, and (3) implementing ongoing reviews of a sample of all businesses that have represented their eligibility to participate in the program. SBA has taken some steps to implement our recommendation—such as by completing its 2017 SOP and its Desk Guide, both of which include written policies and procedures for WOSB program eligibility examinations. The 2017 SOP includes a brief description of what activities are entailed in the examinations, the staff responsible for conducting them, and how firms are selected. In addition, as noted previously, SBA officials told us in September 2018 that a forthcoming update to the SOP would address deficiencies we identified regarding WOSB eligibility examinations. However, as of February 2019, SBA had not provided an updated SOP. The Desk Guide contains more detailed information on eligibility examinations. It notes that a sample of firms is to be examined annually and it provides selection criteria, which can include whether the agency has received information challenging the firm’s eligibility for the program. The Desk Guide also provides specific instructions on how to determine whether a firm meets the WOSB program’s ownership, control, and financial requirements and what documentation should be consulted or requested. SBA does not collect reliable information on the results of its annual eligibility examinations. According to SBA officials, SBA has conducted eligibility examinations of a sample of businesses that received WOSB program set-aside contracts each year since fiscal year 2012. However, SBA officials told us that the results of annual eligibility examinations— such as the number of businesses found eligible or ineligible—are generally not documented. As a result, we obtained conflicting data from SBA on the number of examinations completed and the percentage of businesses found to be ineligible in fiscal years 2012 through 2018. For example, based on previous information provided by SBA, we reported in October 2014 that in fiscal year 2012, 113 eligibility examinations were conducted and 42 percent of businesses were found to be ineligible for the WOSB program. However, during this review, we received information from SBA that 78 eligibility examinations were conducted and 37 percent of businesses were found ineligible in fiscal year 2012. We found similar disparities when we compared fiscal year 2016 data provided by SBA for this report with a performance memorandum summarizing that fiscal year’s statistics. Regardless of the disparity between the data sources, the rate of ineligible businesses has remained significant. For example, according to documentation SBA provided during this review, in fiscal year 2017, SBA found that about 40 percent of the businesses in its sample were not eligible. In addition, SBA continues to have no mechanism for evaluating examination results in aggregate to inform the WOSB program. In 2014, we reported that SBA officials told us that most businesses that were deemed ineligible did not understand the documentation requirements for establishing eligibility. However, we also reported that SBA officials could not explain how they knew a lack of understanding was the cause of ineligibility among businesses and had not made efforts to confirm that this was the cause. In June 2018, SBA officials told us they did not analyze the annual examinations in aggregate for common eligibility issues because the examination results are unique to each WOSB firm. They noted that this was not necessary as WOSB program staff are familiar with common eligibility issues through the annual eligibility examinations. As we noted in 2014, by not analyzing aggregate examination results, the agency is missing opportunities to obtain meaningful insights into the program, such as the reasons many businesses are deemed ineligible. Also, SBA still conducts eligibility examinations only of firms that have already received a WOSB award. In 2014, we concluded that this sampling practice restricts SBA’s ability to identify potentially ineligible businesses prior to a contract award. Similarly, during this review, SBA officials said that while some aspects of the sample characteristics have changed since 2012, the samples still generally consist only of firms that have been awarded a WOSB set-aside. In addition, officials said that the sample size of the eligibility examinations has varied over time and is largely based on the workload of WOSB program staff. Restricting the samples in this way limits SBA’s ability to better understand the eligibility of businesses before they apply for and are awarded contracts, as well as its ability to detect and prevent potential fraud. SBA officials said that their other means of reducing participation by ineligible firms and mitigating potential fraud is through WOSB or EDWOSB status protests—that is, allegations that a business receiving an award does not meet program eligibility requirements. A federal contractor can file a status protest against any firm receiving an award that represents itself as a WOSB in the System for Award Management for grounds that include failure to provide all required supporting documentation. The penalties for misrepresenting a firm’s status, per regulation, include debarment or suspension. However, one third-party certifier expressed in its comments to the advance notice of proposed rulemaking on certification that status protests alone are not a viable option for protecting the integrity of the WOSB program. The certifier questioned how a firm could have sufficient information about a competitor firm to raise questions about its eligibility. According to SBA officials, 11 status protests were filed under the WOSB program in fiscal year 2018. Of these, four firms were deemed ineligible for the WOSB program, four were deemed eligible, and three status protests were dismissed. In fiscal year 2017, 9 status protests were filed; of these, three firms were found ineligible, two were found eligible, and four status protests were dismissed. We recognize that SBA has made some effort to address our previous recommendation by documenting procedures for conducting annual eligibility examinations of WOSB firms. However, leading fraud risk management practices state that federal program managers should design control activities that focus on fraud prevention over detection and response, to the extent possible. Without maintaining reliable information on the results of eligibility examinations, developing procedures for analyzing results, and expanding the sample of businesses to be examined to include those that did not receive contracts, SBA limits the value of its eligibility examinations and its ability to reduce ineligibility among businesses registered to participate in the WOSB program. These deficiencies also limit SBA’s ability to identify potential fraud risks and develop any additional control activities needed to address these risks. As a result, the program may continue to be exposed to the risk of ineligible businesses receiving set-aside contracts. In addition, in light of these continued oversight deficiencies, the implementation of sole-source authority without addressing the other changes made by the 2015 NDAA could increase program risk. For these reasons, we maintain that our previous recommendation that SBA enhance its WOSB eligibility examination procedures should be addressed. In 2015 and 2018, the SBA OIG reported instances in which WOSB set- asides were awarded using NAICS codes that were not eligible under the WOSB program, and our analysis indicates that this problem persists. In 2015, the SBA OIG reported on its analysis of a sample of 34 WOSB set- aside awards and found that 10 awards were set aside using an ineligible NAICS code. The SBA OIG concluded that this may have been due to contracting officers’ uncertainty about NAICS code requirements under the program and recommended that SBA provide additional, updated training and outreach to federal agencies’ contracting officers on the program’s NAICS code requirements. In response, SBA updated WOSB program training and outreach documents in March 2016 to include information about the program’s NAICS code requirements. In 2018, the SBA OIG issued another report evaluating the WOSB program, with a focus on the use of the program’s sole-source contract authority. Here, the SBA OIG identified additional instances of contracting officers using inaccurate NAICS codes to set aside WOSB contracts. Specifically, the SBA OIG reviewed a sample of 56 awards and found that 4 were awarded under ineligible NAICS codes. The report included two recommendations for SBA aimed at preventing and correcting improper NAICS code data in FPDS-NG: (1) conduct quarterly reviews of FPDS- NG data to ensure contracting officers used the appropriate NAICS codes and (2) in coordination with the Office of Federal Procurement Policy and GSA, strengthen controls in FPDS-NG to prevent contracting officers from using ineligible NAICS codes. SBA disagreed with both of these recommendations. In its response to the first recommendation, SBA stated that it is not responsible for the oversight of other agencies’ contracting officers and therefore is not in a position to implement the corrective actions. With respect to the second recommendation, SBA stated that adding such controls to FPDS-NG would further complicate the WOSB program and increase contracting officers’ reluctance to use it. SBA also stated its preference for focusing its efforts on ensuring that contracting officers select the appropriate NAICS code at the beginning of the award process. In our review, we also found several issues with WOSB program set- asides being awarded under ineligible NAICS codes. Our analysis of FPDS-NG data on all obligations to WOSB program set-asides from the third quarter of fiscal year 2011 through the third quarter of fiscal year 2018 found the following: 3.5 percent (or about $76 million) of WOSB program obligations were awarded under NAICS codes that were never eligible for the WOSB program; 10.5 percent (or about $232 million) of WOSB program obligations made under an EDWOSB NAICS code went to women-owned businesses that were not eligible to receive awards in EDWOSB- eligible industries; and 17 of the 47 federal agencies that obligated dollars to WOSB program set-asides during the period used inaccurate NAICS codes in at least 5 percent of their WOSB set-asides (representing about $25 million). According to SBA officials we spoke with during this review, WOSB program set-asides may be awarded under ineligible NAICS codes because of human error when contracting officers are inputting data in FPDS-NG or because a small business contract was misclassified as a WOSB program set-aside. They characterized the extent of the issue as “small” relative to the size of the FPDS-NG database and said that such issues do not affect the program’s purpose. Rather than review FPDS-NG data that are inputted after the contract is awarded, SBA officials said that they have discussed options for working with GSA to add controls defining eligible NAICS codes for WOSB program set-aside opportunities on FedBizOpps.gov—the website that contracting officers use to post announcements about available federal contracting opportunities. Adding controls to this system, officials said, would help contracting officers realize as they are writing the contract requirements that they should not set aside contracts under the WOSB program without reviewing the proper NAICS codes. However, SBA officials said that the feasibility of this option was still being discussed and that the issue was not a high priority. For these reasons, according to officials, SBA’s updated oversight procedures described in the 2017 SOP and the Desk Guide do not include a process for reviewing WOSB program set-aside data in FPDS-NG to determine whether they were awarded under the appropriate NAICS codes. Further, as of November 2018, the WOSB program did not have targeted outreach or training that focused on specific agencies’ use of NAICS codes. As noted previously, in March 2016, SBA updated its WOSB program training materials to address NAICS code requirements in response to a 2015 SBA OIG recommendation. In fiscal year 2018, SBA conducted three WOSB program training sessions for federal contracting officers, including (1) a virtual learning session, (2) a session conducted during WOSB Industry Day at the Department of Housing and Urban Development, and (3) a session conducted during a Department of Defense Small Business Training Conference. However, with the exception of the virtual learning session, these training sessions were requested by the agencies. SBA officials did not identify any targeted outreach or training provided to specific agencies to improve understanding of WOSB NAICS code requirements (or other issues related to the WOSB program). Congress authorized SBA to develop a contract set-aside program specifically for WOSBs and EDWOSBs to address the underrepresentation of such businesses in specific industries. In addition, federal standards for internal control state that management should design control activities to achieve objectives and respond to risks and to establish and operate monitoring activities to monitor and evaluate the results. Because SBA does not review whether contracts are being awarded under the appropriate NAICS codes, it cannot provide reasonable assurance that WOSB program requirements are being met or identify agencies that may require targeted outreach or additional training on eligible NAICS codes. As a result, WOSB contracts may continue to be awarded to groups other than those intended, which can undermine the goals of and confidence in the program. Federal dollars obligated for contracts to all women-owned small businesses increased from $18.2 billion in fiscal year 2012 to $21.4 billion in fiscal year 2017. These figures include contracts for any type of good or service awarded under the WOSB program, under other federal programs, or through full and open competition. Contracts awarded to all women-owned small businesses within WOSB-program-eligible industries also increased during this period—from about $15 billion to $18.8 billion, as shown in figure 1. However, obligations under the WOSB program represented only a small share of this increase. In fiscal year 2012, WOSB program contract obligations were 0.5 percent of contract obligations to all women-owned small businesses for WOSB-program- eligible goods or services (about $73.5 million), and in fiscal year 2017 this percentage had grown to 3.8 percent (about $713.3 million) (see fig. 1). From fiscal years 2012 through 2017, 98 percent of total dollars obligated for contracts to all women-owned small businesses in WOSB-program- eligible industries were not awarded under the WOSB program. Instead, these contracts were awarded without a set-aside or under other, longer- established socioeconomic contracting programs, such as HUBZone, the SDVOSB, and 8(a). For example, during this period, dollars obligated to contracts awarded to women-owned small businesses without a set-aside represented about 34 percent of dollars obligated for contracts to all women-owned small businesses in these industries (see fig. 2). As shown in table 1, six federal agencies—DOD, DHS, Department of Commerce, Department of Agriculture, Department of Health and Human Services, and GSA—collectively accounted for nearly 83 percent of the obligations awarded under the WOSB program from the third quarter of fiscal year 2011 through the third quarter of fiscal year 2018, with DOD accounting for about 49 percent of the total. Contracting officers’ use of sole-source authority was relatively limited, representing about 12 percent of WOSB program obligations from January 2016 through June 2018. In fiscal year 2017—the only full fiscal year for which we have data on sole-source authority—about $77 million were obligated using sole-source authority. The share of sole-source awards as a percentage of total WOSB program set-asides also varied considerably by quarter—from as low as 5 percent in the third quarter of 2016 to as high as 21 percent in the first quarter of 2017 (see fig. 3). We spoke with 14 stakeholder groups to obtain their views on usage of the WOSB program. These groups consisted of staff within three federal agencies (DHS, DOD, and GSA), eight contracting offices within these agencies, and three third-party certifiers. Issues stakeholders discussed included the impact of sole-source authority and program-specific NAICS codes on program usage. Stakeholders also noted the potential effect of other program requirements on contracting officers’ willingness to use the program, and some suggested that SBA provide additional guidance and training to contracting officers. Sole-source authority. Participants in 12 of the 14 stakeholder groups commented on the effect of sole-source authority on WOSB program usage. Staff from 4 of the 12 stakeholder groups—including three contracting offices—said that sole-source authority generally had no effect on the use of the WOSB program. One of these stakeholders believed contracting officers seldom use the authority because they lack an understanding of how and when to use it; therefore, in this stakeholder’s opinion, use of the WOSB program has not generally changed since the authority was implemented. However, staff from two contracting offices and one third-party certifier said that sole-source authority was a positive addition because, for example, it can significantly reduce the lead time before a contracting officer can offer a contract award to a firm. Staff from one of these two contracting offices stated that the award process can take between 60 to 90 days using sole-source authority, compared to 6 to 12 months using a competitive WOSB program set-aside. These staff also said that negotiating the terms of a sole-source contract is easier, from a contracting officer’s perspective, because they can communicate directly with the firm. As discussed previously, SBA officials we interviewed said that adding sole-source authority to the WOSB program made the program more consistent with other existing socioeconomic set-aside programs, such as 8(a) and HUBZone. The remaining five stakeholder groups that discussed the effects of WOSB sole-source authority described difficulties with implementing it. Specifically, representatives from DHS, DOD, and one third-party certifier said that executing sole-source authority under the WOSB program is difficult for contracting officers because rules for sole-source authority under WOSB are different from those under other SBA programs, such as 8(a) and HUBZone. For example, the FAR’s requirement that contracting officers justify, in writing, why they do not expect other WOSBs or EDWOSBs to submit offers on a contract is stricter under the WOSB program than it is for the 8(a) program. Further, staff from one contracting office noted that justifications for WOSB set-asides must then be published on a federal website. In contrast, contracting officers generally do not need to prepare and publish a justification under the 8(a) program. According to staff from another contracting office, it may be difficult to find more than one firm qualified to do the work under some WOSB-eligible NAICS codes, but contracting officers would still have to conduct market research and explain why they do not expect additional offers in order to set the contract aside for a WOSB. Program-specific NAICS codes. Participants in 13 of the 14 stakeholder groups we interviewed commented on the requirement that WOSB program set-asides be awarded within certain industries, represented by NAICS codes. For example, two third-party certifiers we interviewed recommended that the NAICS codes be expanded or eliminated to provide greater opportunities for WOSBs to win contracts under the program. Another third-party certifier said that some of its members focus their businesses’ marketing efforts on industries specific to the WOSB program to help them compete for such contracts. Representatives from GSA and DHS made comments about limitations with respect to the WOSB program’s NAICS code requirement. Staff we interviewed from three contracting offices made similar statements, adding that the NAICS codes limit opportunities to award a contract to a WOSB or EDWOSB because they are sufficient in some industry areas but not others. All five of these stakeholder groups suggested that NAICS codes be removed from the program’s requirements to increase opportunities for WOSBs. Conversely, staff from five other contracting offices we interviewed generally expressed positive views about the WOSB program’s NAICS code requirements and stated that eligible codes line up well with the services for which they generally contract. Finally, SBA officials noted that there are no plans to reassess the NAICS codes until about 2020. However, SBA officials also stated that the NAICS code requirements complicate the WOSB program and add confusion for contracting officers who use program, as compared to other socioeconomic programs that do not have such requirements, such as HUBZone or 8(a). Requirement to verify eligibility documentation. Staff from 7 of the 14 stakeholder groups we interviewed discussed the requirement for the contracting officer to review program eligibility documentation and how this requirement affects their decision to use the program. For example, staff from one contracting office said that using the 8(a) or HUBZone programs is easier because 8(a) and HUBZone applicants are already certified by SBA; therefore, the additional step to verify documentation for eligibility is not needed. GSA officials noted that eliminating the need for contracting officers to take additional steps to review eligibility documentation for WOSB-program set-asides—in addition to checking the System for Award Management—could create more opportunities for WOSBs by reducing burden on contracting officers. However, staff from two contracting offices said it is not more difficult to award contracts under the WOSB program versus other socioeconomic programs. WOSB program guidance. Staff from 13 of the14 stakeholder groups we interviewed discussed guidance available to contracting officers under the WOSB program. Most generally said that the program requirements outlined in the FAR are fairly detailed and help contracting officers implement the program. According to SBA officials, SBA provides training on WOSB program requirements to contracting officers in federal agencies by request, through outreach events, and through an annual webinar. SBA officials also said that the training materials include all the regulatory issues that contracting officers must address. However, representatives from two third-party certifiers described feedback received from their members about the need to provide additional training and guidance for contracting officers to better understand and implement the WOSB program. Staff from two contracting offices also expressed the need for SBA to provide additional training and guidance. Staff from one of these contracting offices said that the last time they received training on the WOSB program was in 2011, when the program was first implemented. Staff in the other contracting office added that the most recent version of a WOSB compliance guide they could locate online was at least 6 years old. SBA officials estimated that the WOSB compliance guide was removed from their public website in March 2016 because it was difficult to keep the document current and officials did not want to risk publishing a guide that was out-of-date. SBA officials also said that there are no plans to issue an updated guide as the FAR is sufficient. The stakeholder groups also identified positive aspects of the WOSB program. Specifically, staff from seven stakeholder groups believed that the program provided greater opportunities for women-owned small businesses to obtain contracts in industries in which they are underrepresented. In addition, staff from three stakeholder groups mentioned that SBA-led initiatives, such as the Small Business Procurement Advisory Council and SBA’s co-sponsorship of the ChallengeHER program, help improve collaboration between federal agencies and the small business community and overall government contracting opportunities for women-owned small businesses. The WOSB program aims to enhance federal contracting opportunities for women-owned small businesses. However, weaknesses in SBA’s management of the program continue to hinder its effectiveness. As of February 2019, SBA had not fully implemented comprehensive procedures to monitor the performance of the WOSB program’s third- party certifiers and had not taken steps to provide reasonable assurance that only eligible businesses obtain WOSB set-aside contracts, as recommended in our 2014 report. Without ongoing monitoring and reviews of third-party certifier reports, SBA cannot ensure that the certifiers are fulfilling the requirements of their agreements with SBA, and it is missing opportunities to gain information that could help improve the program’s processes. Further, limitations in SBA’s procedures for conducting, documenting, and analyzing eligibility examinations inhibit its ability to better understand the eligibility of businesses before they apply for and potentially receive contracts, which exposes the program to unnecessary risk of fraud. In addition, given that SBA does not expect to finish implementing the changes in the 2015 NDAA until January 1, 2020, these continued oversight deficiencies increase program risk. As a result, we maintain that our previous recommendations should be addressed. In addition, SBA has not addressed deficiencies that the SBA OIG identified previously—and that we also identified during this review— related to WOSB set-asides being awarded under ineligible industry codes. Although SBA has updated its training and outreach materials for the WOSB program to address NAICS code requirements, it has not developed plans to review FPDS-NG data or provide targeted outreach or training to agencies that may be using ineligible codes. As a result, SBA is not aware of the extent to which individual agencies are following program requirements and which agencies may require targeted outreach or additional training. Reviewing FPDS-NG data would allow SBA to identify those agencies (and contracting offices within them) that could benefit from such training. Without taking these additional steps, SBA cannot provide reasonable assurance that WOSB program requirements are being met. The SBA Administrator or her designee should (1) develop a process for periodically reviewing FPDS-NG data to determine the extent to which agencies are awarding WOSB program set-asides under ineligible NAICS codes and (2) take steps to address any issues identified, such as providing targeted outreach or training to agencies making awards under ineligible codes. (Recommendation 1) We provided a draft of this report to DHS, DOD, GSA, and SBA for review and comment. DHS, DOD, and GSA indicated that they did not have comments. SBA provided a written response, reproduced in appendix II, in which it agreed with our recommendation. SBA stated that it will implement a process to review WOSB program data extracted from FPDS-NG and certified by each agency. Specifically, through the government-wide Small Business Procurement Advisory Council, SBA plans to provide quarterly presentations to contracting agencies’ staff that would include training and an analysis and review of the data. The response also reiterated that SBA has contacted GSA to implement a system change to FedBizOpps.gov that would prevent contracting officers from entering an invalid NAICS code for a WOSB program set-aside. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time we will send copies of this report to appropriate congressional committees and members, the Acting Secretary of DOD, the Secretary of DHS, the Administrator of GSA, the Administrator of SBA, and other interested parties. This report will also be available at no charge on our website at http://www.gao.gov. If you or your staff have any questions concerning this report, please contact me at (202) 512-8678 or shearw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. This report examines (1) the extent to which the Small Business Administration (SBA) has implemented changes to the Women-Owned Small Business Program (WOSB program) made by the 2015 National Defense Authorization Act (2015 NDAA); (2) the extent to which SBA has implemented changes to address previously identified oversight deficiencies; and (3) changes in WOSB program use since 2011 and stakeholder views on its use, including since the 2015 implementation of sole-source authority. To describe the extent to which SBA has implemented changes to the WOSB program made by the 2015 NDAA, we reviewed relevant legislation, including the 2015 NDAA; related proposed regulations; and SBA documentation. We reviewed comment letters on the advance notice of proposed rulemaking for the new WOSB program certification process from three of the four SBA-approved third-party certifiers: the El Paso Hispanic Chamber of Commerce, the U.S. Women’s Chamber of Commerce, and the Women’s Business Enterprise National Council. To ensure the accuracy of our characterization of the comment letters, one staff member independently summarized the third-party certifiers’ comments on the advance notice, and a second staff member then reviewed the results. We also interviewed SBA officials, including officials from SBA’s Office of Government Contracting and Business Development. To respond to the second and third objectives, we conducted interviews on SBA’s implementation and oversight of the WOSB program and its use with SBA officials, three of the WOSB program’s four third-party certifiers, three selected agencies (and three agency components within two of the agencies), and a total of eight selected contracting offices within six selected agencies or components. Using data from the Federal Procurement Data System-Next Generation (FPDS-NG), we judgmentally selected the three federal agencies and three components (for a total of six federal agencies and components) because their WOSB program dollar obligations (including competed and sole-source) were among the largest or because we had interviewed them for our prior work. Specifically, we selected the following six agencies or agency components: the Department of Homeland Security (DHS) and, within DHS, the Coast Guard; the Department of Defense (DOD) and, within DOD, the U.S. Army and U.S. Navy; and the General Services Administration (GSA). Within the components and GSA, we judgmentally selected eight contracting offices (two each from Coast Guard, U.S. Army, U.S. Navy, and GSA) based on whether they had a relatively large amount of obligations and had used multiple types of WOSB program set- asides (competed or sole-source) to WOSBs or economically disadvantaged women-owned small businesses (EDWOSB). To address our second objective, we reviewed the findings and recommendations in our October 2014 report and in audit reports issued by the SBA Office of Inspector General (OIG) in May 2015 and June 2018. We also reviewed SBA documentation on the WOSB program, including SBA’s 2017 Standard Operating Procedures and 2018 WOSB Program Desk Guide, results from 2016 compliance reviews of the four third-party certifiers, and SBA eligibility examinations from fiscal years 2012 through 2018. In addition, we analyzed FPDS-NG data on contract obligations to WOSB program set-asides from the third quarter of fiscal year 2011 through the third quarter of fiscal year 2018 to determine whether set-asides were made using eligible program-specific North American Industry Classification System (NAICS) codes. To conduct this analysis, we compared contract obligations in FPDS-NG with the NAICS codes eligible under the WOSB program at the time of the award for the time frame under review. The WOSB program’s eligible NAICS codes have changed three times since the program was implemented in 2011, but the eligible industries have changed once. SBA commissioned the RAND Corporation to conduct the first study to assist SBA in determining eligible NAICS codes under the WOSB program. Based on the results of the RAND study, SBA identified 45 four-digit WOSB NAICS codes and 38 four-digit EDWOSB NAICS codes, for a total of 83 four-digit NAICS codes. WOSB and EDWOSB NAICS codes are different and do not overlap. In December 2015, the Department of Commerce issued the next study, which increased the total NAICS codes under the program to 113 four-digit codes, with 92 WOSB NAICS codes and 21 EDWOSB NAICS codes (which became effective March 2016). Often, there is a time lag between the effective date of NAICS codes and when they are entered in FPDS-NG. Therefore, we did not classify a contract as having an ineligible NAICS code if the code eventually became eligible under the WOSB program. We also excluded actions in FPDS-NG coded other than as a small business. These actions represented a small amount of contract obligations—approximately $125,000. We compared SBA information on its oversight activities and responses to previously identified deficiencies, federal internal control standards, and GAO’s fraud risk framework. We assessed the reliability of FPDS-NG data by considering their known strengths and weaknesses, based on our past work and through electronic testing for missing data, outliers, and inconsistent coding in the data elements we used for our analysis. We also reviewed FPDS-NG documentation, including the FPDS-NG data dictionary, FPDS-NG data validation rules, FPDS-NG user manual, prior GAO reliability assessments, and relevant SBA OIG audit reports. Based on these steps, we concluded that the data were sufficiently reliable for the purposes of reporting on trends in the WOSB program and the use of sole-source authority under the program. To describe how participation in the WOSB program has changed since 2011, including since the 2015 implementation of sole-source authority, we analyzed FPDS-NG data from the third quarter of fiscal year 2011 through the third quarter of fiscal year 2018. We identified any trends in WOSB program participation using total obligation dollars set aside for competitive and sole-source contracts awarded to WOSBs and EDWOSBs under the program. We also compared data on obligations for set-asides under the WOSB program with federal contract obligations for WOSB-program-eligible goods and services to all women-owned small businesses, including those made under different set-aside programs or with no set-asides, to determine the relative usage of the WOSB program. In our analysis, we excluded from WOSB program set-aside data actions in FPDS-NG coded other than as a small business (representing approximately $125,000) or coded under ineligible NAICS codes that were never eligible under the WOSB program (representing approximately $76.3 million). To describe stakeholder views on WOSB program use, we conducted semistructured interviews to gather responses from 14 stakeholder groups. These groups consisted of staff within three federal agencies (DHS, DOD, and GSA), eight contracting offices within these agencies, and three third-party certifiers (selection criteria described above). One person summarized the results of the interviews, and another person reviewed the summary of the interviews to ensure an accurate depiction of the comments. In addition, a third person then reviewed the summarized results. We conducted this performance audit from October 2017 to March 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. In addition to the contact named above, Allison Abrams (Assistant Director), Tiffani Humble (Analyst-in-Charge), Pamela Davidson, Jonathan Harmatz, Julia Kennon, Jennifer Schwartz, Rebecca Shea, Jena Sinkfield, Tyler Spunaugle, and Tatiana Winger made key contributions to this report.

Summary: In 2000, Congress authorized the WOSB program, allowing contracting officers to set aside procurements to women-owned small businesses in industries in which they are substantially underrepresented. To be eligible to participate in the WOSB program, firms have the option to self-certify or be certified by a third-party certifier. However, the 2015 NDAA changed the WOSB program by (1) authorizing SBA to implement sole-source authority, (2) eliminating the option for firms to self-certify as being eligible for the program and (3) allowing SBA to implement a new certification process. GAO was asked to review the WOSB program. This report discusses (1) the extent to which SBA has addressed the 2015 NDAA changes, (2) SBA's efforts to address previously identified deficiencies, and (3) use of the WOSB program. GAO reviewed relevant laws, regulations, and program documents; analyzed federal contracting data from April 2011 through June 2018; and interviewed SBA officials, officials from contracting agencies selected to obtain a range of experience with the WOSB program, and three of the four private third-party certifiers. The Small Business Administration (SBA) has implemented one of the three changes to the Women-Owned Small Business (WOSB) program authorized in the National Defense Authorization Act of 2015 (2015 NDAA). Specifically, in September 2015 SBA published a final rule to implement sole-source authority, effective October 2015. As of February 2019, SBA had not eliminated the option for program participants to self-certify that they are eligible to participate, as required by 2015 NDAA. SBA officials stated that this requirement would be addressed as part of the new certification process for the WOSB program, which they expect to implement by January 1, 2020. SBA has not addressed WOSB program oversight deficiencies identified in GAO's 2014 review (GAO-15-54). For example, GAO previously recommended that SBA establish procedures to assess the performance of four third-party certifiers—private entities approved by SBA to certify the eligibility of WOSB firms. While SBA conducted a compliance review of the certifiers in 2016, it has no plans to regularly monitor them. By not improving its oversight of the WOSB program, SBA is limiting its ability to ensure third-party certifiers are following program requirements. In addition, the implementation of sole-source authority in light of these continued oversight deficiencies can increase program risk. Consequently, GAO maintains that its prior recommendations should be addressed. In addition, similar to previous findings from SBA's Office of Inspector General, GAO found that about 3.5 percent of contracts using a WOSB set-aside were awarded for ineligible goods or services from April 2011 through June 2018. SBA does not review contracting data that could identify this problem and inform SBA which agencies making awards may need targeted outreach or training. As a result, SBA cannot provide reasonable assurance that WOSB program requirements are being met and that the program is meeting its goals. While federal contract obligations to all women-owned small businesses and WOSB program set-asides have increased since fiscal year 2012, WOSB program set-asides remain a small percentage (see figure). GAO recommends that SBA develop a process for periodically reviewing the extent to which WOSB program set-asides are awarded for ineligible goods or services and use the results to address identified issues, such as through targeted outreach or training on the WOSB program. SBA agreed with the recommendation.