Small Business Contracting: DOD Should Take Actions to Ensure That Its Pilot Mentor-Protégé Program Enhances the Capabilities of Protégé Firms
Released: 2017-04-11T00:00:00

Letter

Background

DOD’s Pilot Mentor- Protégé Program

The purpose of DOD’s Pilot Mentor-Protégé Program, as established in statute, is to provide incentives to major DOD contractors to furnish eligible disadvantaged small business concerns with assistance designed to (1) enhance the capabilities of eligible disadvantaged small business concerns to perform as subcontractors and suppliers under DOD contracts and other contracts and subcontracts; and (2) increase the participation of such business concerns as subcontractors and suppliers under DOD contracts, other federal government contracts, and commercial contracts. DOD’s Office of Small Business Programs (OSBP) manages the program, including developing and overseeing the policies and procedures for the program. In addition, a number of DOD components administer individual mentor-protégé agreements. DOD components and the mentors and protégés that participate in the program are required to follow DOD’s regulations and policies and procedures. For example, DOD’s regulations state that the director of small business programs for each DOD component has the authority to approve firms as mentors and to approve mentor-protégé agreements, among other things. OSBP has developed tools to facilitate program administration, including a template to assist mentors and protégés in developing proposed agreements and a checklist to assist DOD components with ensuring that proposed agreements address required elements, which we discuss later in this report. However, DOD components and mentors and protégés are not required to use the tools.

DOD’s Pilot Mentor-Protégé Program offers two types of agreements. Reimbursable agreements provide payments directly to the mentor for the cost of providing developmental assistance to the protégé. Credit agreements provide mentors with credit toward their subcontracting goals for providing developmental assistance to protégés. Mentors can provide protégés with various types of assistance under mentor-protégé agreements, including general business management, subcontract awards, progress payments, advance payments, and loans. Mentors may also obtain assistance for protégés from Small Business Development Centers, Procurement Technical Assistance Centers, historically black colleges and universities, and minority institutions of higher education (see fig. 1).

DOD policies and procedures also require mentors to report on the progress made under active mentor-protégé agreements in semiannual reports. Mentors must also include in semiannual reports, among other things, (1) any assistance obtained by the mentor firm for the protégé firm from Small Business Development Centers, Procurement Technical Assistance Centers, historically black colleges and universities, and minority institutions of higher education for developmental assistance provided to protégés, (2) dollars credited (if any) toward applicable subcontracting goals as a result of developmental assistance provided to the protégé, and (3) the impact of the agreement in terms of capabilities enhanced, certifications received, or technology transferred. In addition, annual reports are to contain data on the protégé’s employment, revenue, and participation in DOD contracts. DOD procedures also require protégés to report on the progress made in each of the 2 years following the completion of their agreement. Further, DOD procedures require the Defense Contract Management Agency (DCMA), another DOD component that manages agreements for the department, to conduct a performance review annually of each mentor-protégé agreement.

DOD’s process for establishing and approving mentor-protégé agreements generally consists of nine steps (as illustrated in fig. 2): 1. Mentor seeks approval to participate in program. A firm that is interested in participating in the DOD Pilot Mentor-Protégé Program must submit an application for approval as a mentor to the Director of OSBP. To be eligible as a mentor, among other factors, a firm must not be affiliated with the protégé prior to the approval of the agreement and must demonstrate it is qualified to provide assistance that will contribute to the program’s purpose, among other requirements. 2. Mentor establishes a counterpart. The mentor is solely responsible for selecting the protégé. To be eligible as a protégé, a firm must be, among other things, a women-owned small business, Historically Underutilized Business Zone (HUBZone) small business, service- disabled veteran-owned small business, or an eligible entity employing the severely disabled. 3. Mentor conducts a needs assessment. Once the mentor selects the protégé, the mentor conducts a preliminary assessment of the protégé’s developmental needs. The mentor and protégé then mutually agree on the developmental assistance that the mentor is to provide to address those needs. 4. Parties determine the type of agreement. The mentor and protégé may apply for either a reimbursable mentor-protégé agreement or a credit agreement. 5. Parties develop the agreement. The mentor and protégé develop a proposed mentor-protégé agreement that must address a number of required elements, which we discuss in greater detail later in this report. 6. Mentor submits proposed agreement for review. The mentor submits the proposed agreement to the cognizant DOD component for review and approval. 7. DOD component approves or denies agreement. As previously noted, the director of small business programs for each DOD component has the authority to approve mentor-protégé agreements. 8. Parties start the agreement. The mentor and protégé begin the agreement, which cannot generally exceed 3 years. 9. Parties report on progress. The mentor is required to submit a semiannual report on the progress made under an active mentor- protégé agreement throughout the term of an agreement. The protégé is required to submit an annual report on progress made during each fiscal year of the program term and for each of the 2 fiscal years following the expiration of the agreement.

The NDAA for Fiscal Year 2016 made several changes to DOD’s Pilot Mentor-Protégé Program. For example, with respect to the relationship between a mentor firm and protégé firm, the act strengthened affiliation rules by explicitly stating that the presence of certain family relationships between owners and managers in both the mentor and protégé firms constitutes affiliation and thereby is not allowed. In addition, the act amended the size requirement so that DOD’s size standard for protégés is set at less than half of that used by SBA. See appendix III for additional information on selected changes to the program made by the NDAA for Fiscal Year 2016.

SBA’s All Small Mentor- Protégé Program

According to SBA, the All Small Mentor-Protégé Program is designed to apply to all federal small business contracts, including those that small businesses currently participate in under other federal mentor-protégé programs. In addition, according to SBA, the rule establishing the regulations for the program is intended to be consistent with the regulations for SBA’s 8(a) Business Development Mentor-Protégé Program. SBA’s All Small Mentor-Protégé Program includes certain requirements for the mentor-protégé programs of other federal agencies, but, as required by federal law, the program exempts DOD’s Pilot Mentor- Protégé Program from these requirements. For example, the final rule states that the head of a department or agency must submit a plan to SBA for any previously existing mentor-protégé program that the department or agency seeks to continue within 1 year of the effective date of the rule (or by August 24, 2017). In addition, the final rule states that a federal department or agency may not start a new mentor-protégé program unless the head of the department or agency submits a plan to the SBA Administrator for the program and the SBA Administrator approves the plan. The incentives provided to mentors under SBA’s programs differ from those provided under DOD’s Pilot Mentor-Protégé Program, as we discuss in greater detail later in this report.

DOD’s Procedures Do Not Provide Reasonable Assurance That Mentor-Protégé Agreements Contain All Required Elements

DOD components have the authority to approve mentor-protégé agreements, and they are required to follow the regulations, policies, and procedures DOD has established for the program. However, DOD lacks controls needed to provide reasonable assurance that mentor-protégé agreements approved by DOD components contain all elements required by DOD policies and regulations.

DOD’s regulations and OSBP procedures do not prescribe the process DOD components should follow to approve mentor-protégé agreements. Instead, DOD’s OSBP has allowed DOD components to develop their own detailed procedures. As shown in the following examples, these procedures differ in terms of some elements, such as criteria and time frames:

Air Force. The Department of the Air Force (Air Force) issues an announcement each fiscal year that outlines its process for considering proposed mentor-protégé agreements. The fiscal year 2016 announcement describes a two-step solicitation process. For fiscal year 2016, the Air Force was first to solicit white papers from prospective mentors describing, among other things, the proposed developmental assistance the mentor planned to provide to the protégé and the anticipated benefits of the agreement to the Air Force and protégé. Second, the Air Force was to invite mentors with white papers found to be consistent with the program goals to submit a formal mentor-protégé agreement proposal. As part of the proposal, mentors were required to submit a mentor-protégé agreement using the template found on OSBP’s website. The announcement also stated that the Air Force planned to consider four factors when determining whether to recommend proposals for award: (1) relevance of proposed technology to current Air Force requirements, (2) overall technical approach to providing transfer of technology and developmental assistance to the protégé, (3) the mentor’s capacity and capability to achieve the objectives of the mentor-protégé program, and (4) estimated cost of the proposed assistance.

Army. The Department of the Army (Army) guidance states that proposed mentor-protégé agreements are to be approved and funded twice each fiscal year. The guidance states that the Army intends to approve and fund proposed agreements that have a strong technical component or focus on the innovative transfer of state-of-the-art technology that supports Army troops. The Army guidance further states that mentors are required to submit a mentor-protégé agreement using the template found on OSBP’s website. The guidance notes that the Army is to consider the following factors in descending order of importance when evaluating proposed mentor- protégé agreements: (1) subcontracting and prime contractor opportunities, (2) technical approach for developmental assistance, (3) involvement of historically black colleges and universities and minority institutions, (4) relevance to Army programs, (5) corporate capability and commitment, (6) management plan, and (7) mentor and protégé past and present performance.

Navy. The Department of the Navy (Navy) guidance states that the prospective mentor and protégé are required to schedule an informal introductory briefing with the Navy’s director of small business programs to discuss the proposed concept and objective of the mentor-protégé agreement. As part of the briefing, the mentor and protégé are required to discuss, among other things, the mentor’s experience providing developmental assistance under other mentor- protégé agreements; the proposed benefits to the mentor, protégé, and the Navy under the proposed agreement; and the rationale for why the proposed agreement should be sponsored. The Navy is to then provide the mentor and protégé with feedback on whether to submit a proposed mentor-protégé agreement. The Navy’s guidance also states that the Navy accepts mentor-protégé agreement proposals during three time frames each fiscal year (December 1-31, March 1-31, and August 1-31). As part of the proposal, mentors are required to submit a mentor-protégé agreement using the template found on OSBP’s website. The guidance further states that the Navy is to consider the following factors in descending order of importance when evaluating proposed mentor-protégé agreements: (1) merit of the technology transfer to the protégé, (2) perceived benefit of the agreement to the Navy, (3) perceived benefit of the agreement to the protégé, (4) percentage of hours associated with technology transfer, (5) subcontracting opportunities available to the protégé, (6) utilization of historically black colleges and universities and minority institutions, Procurement Technical Assistance Centers, and Small Business Development Centers, and (7) proposed cost.

Although DOD components have flexibility in considering a range of factors when approving mentor-protégé agreements, the agreements themselves are required to include certain elements that are defined in DOD’s procedures, and these elements serve a variety of purposes. For example, an agreement must include the protégé’s primary North American Industry Classification System (NAICS) code, which is used to determine the protégé’s eligibility to participate in the program. In addition, the agreement must describe the developmental assistance planned for the protégé, how the assistance is to address the protégé’s identified needs and enhance its ability to perform successfully under contracts or subcontracts within DOD and other federal agencies, and factors to assess the protégé firm’s developmental progress under the program, including specific milestones for providing assistance. The agreement must also specify a program participation term that does not exceed 3 years, which provides assurance that the mentor provides assistance to the protégé in a timely manner. Further, the agreement also is required to be signed and dated by the mentor and protégé, which provides evidence of meeting the program requirement between the two parties. Figure 3 presents a complete list of the elements that mentors and protégés are required to include in their agreements.

Our review of a randomly selected probability sample of 44 of the 78 total active DOD mentor-protégé agreements in place as of June 2016 found that a number of these agreements were missing required elements. Specifically, based on our review, we estimate that 27 percent of the agreements did not address all required elements. With respect to specific elements, we estimate that 25 percent of the agreements did not include the signature of the mentor and protégé, 9 percent did not include the protégé’s primary NAICS code, and 7 percent did not include an anticipated start and end date for the agreement. These missing elements suggest that the components’ procedures for approving mentor-protégé agreements do not provide reasonable assurance that agreements are completed in accordance with DOD requirements.

According to officials from OSBP, which manages DOD’s Pilot Mentor- Protégé Program, the DOD component that approves a mentor-protégé agreement is responsible for providing assurance that the agreement includes the required elements in accordance with the program’s policies. However, DOD’s procedures do not define how DOD components should review these proposals. OSBP officials told us that OSBP developed a checklist for DOD components to use to document the review of proposed mentor-protégé agreements. The checklist includes whether the mentor and protégé have signed the agreement, whether the mentor and protégé meet the program’s eligibility requirements, and whether the agreement includes a description and milestones for the developmental assistance that is to be provided under the agreement, among other things. A senior OSBP official added that OSBP does not require DOD components to use the checklist, nor does OSBP periodically review the actions DOD components have taken to review mentor-protégé agreements for approval because OSBP does not want to micromanage the components. Instead, the official said that OSBP relies on DOD components to ensure that approved mentor-protégé agreements include all required elements, and in some cases, DOD components have developed their own checklist.

Federal internal control standards state that management should implement control activities through policies, and that this principle may be applied by, among other things, periodically reviewing control activities for continued relevance and effectiveness in achieving the entity’s objective or addressing related risks. As part of its responsibility for managing the Pilot-Mentor Protégé Program, OSBP is ultimately responsible for overseeing the program’s policies and procedures. Because OSBP does not review the procedures established by DOD components for approving proposed mentor-protégé agreements or conduct its own monitoring of the components, it does not have assurance that the agreements approved by the components include all required elements. As a result, OSBP cannot ensure that the program requirements are serving their intended purposes—for example, as previously discussed, missing industry codes are used to determine whether protégés are eligible to participate in the program, and the signatures of the mentor and protégé are required for the agreement to meet the program requirements.

DOD Lacks Performance Goals and Measures to Facilitate Effective Program Assessment

Although DOD has established some performance measures for its Pilot Mentor-Protégé Program, the department lacks performance goals and other measures needed to fully assess the program. DOD prepares an annual report for the program related to the progress of protégés under active mentor-protégé agreements and protégés that completed or otherwise terminated participation in the program during the preceding 2 fiscal years. The report includes specific performance measures that describe changes in protégés’ annual revenue, number of employees, number of DOD prime contract awards, and number of DOD subcontract awards. DOD’s annual reports show that protégés experienced increased revenues, number of employees, and DOD prime contracts and subcontracts while participating in the program from fiscal years 2011 through 2015. In contrast, the reports show that protégés that left the program during this period experienced decreased revenues and employees in the 2 years following program participation (see table 1).

However, we found that DOD lacks performance goals and additional measures needed to effectively assess its Pilot Mentor-Protégé Program. Specifically, DOD has not established any measurable goals for the performance information it reports for the program. Performance measurement is the ongoing monitoring and reporting of program accomplishments, particularly progress toward pre-established goals. We have previously identified performance measurement as a best practice that allows organizations to track progress in achieving their goals and gives managers crucial information to identify gaps in program performance and plan any needed improvements. In addition, according to federal internal control standards, managers need to compare actual performance against planned or expected results and analyze significant differences. While DOD collects and reports information on protégés’ annual revenue, employment levels, and prime contract and subcontract awards, it has not established goals for these measures. As a result, DOD is limited in its ability to use this information to assess program performance.

In addition, DOD has not established measures for one part of the program’s statutory purpose. The information that DOD currently reports relates to the part of the program purpose to increase the participation of small businesses in DOD and other federal government contracts. As previously discussed, part of the program’s purpose is also to enhance the capabilities of eligible small businesses to perform as subcontractors and suppliers under DOD and other contracts and subcontracts, and the current program measures do not address this part of the program’s purpose. Mentors report information to DOD in semiannual reports on how the capabilities of protégés were enhanced, what certifications the protégés obtained, and what technology the mentors transferred to the protégés. However, DOD does not include this information in its annual report for the program, and it has not developed performance measures or goals related to this information. As a result, DOD is not using the information it collects to fully assess how well the program is enhancing the capabilities of eligible small businesses to perform under DOD and other contracts, and, therefore, not fully addressing the purpose of the program.

OSBP officials told us that they have not considered setting targets each year for the program’s current performance measures because these measures—revenues, employee levels, and prime contract and subcontract awards—do not reflect the primary focus of the program, which the officials noted is to assist small businesses in becoming part of the defense supply chain. The officials noted that DOD is working to develop measures that provide a better indication of program effectiveness. For example, the officials described a potential measure that would assess how the assistance mentors provide to protégés under agreements helps resolve operational challenges. However, DOD had yet to establish any such measures as of January 2017. Moreover, the officials estimated that it would take about 2 years for DOD to develop and establish baselines for these measures. Until DOD establishes performance goals and related performance measures consistent with the program’s stated purpose, it may be more difficult for DOD to analyze the effectiveness of the Pilot Mentor-Protégé Program and to identify and prioritize potential improvements. As a result, Congress may not have the information it needs to make informed decisions on whether to reauthorize the pilot program, terminate it, or make it permanent.

SBA and DOD Programs Have Key Differences, but the Agencies Plan to Consider Harmonization in Future Discussions

DOD’s Pilot Mentor-Protégé Program and SBA’s All Small Mentor- Protégé Program have a number of key differences, including their eligibility requirements and participation incentives. These differences are largely based on statutory provisions and program rules that may—but will not necessarily—affect future efforts to harmonize the programs. Implementation of SBA’s All Small Mentor-Protégé Program is ongoing, but DOD and SBA officials said they plan to consider harmonizing their programs after SBA’s program is fully implemented in July 2017.

Program Differences Include Eligibility Requirements for Protégés and Participation Incentives

Based on our review of statutory provisions and program rules, we found that DOD’s Pilot Mentor-Protégé Program and SBA’s All Small Mentor- Protégé Program differ in a number of ways. For example, DOD and SBA apply the SBA size standards differently when determining the eligibility of protégés under their respective programs. SBA’s small business size standards establish the largest that a firm can be and still qualify as a small business for federal government programs. DOD’s Pilot Mentor- Protégé Program limits the size of eligible protégés to less than one-half of SBA’s small business size standard. In contrast, a firm can qualify as a protégé under SBA’s All Small Mentor-Protégé Program if it does not exceed the SBA small business size standard for its primary NAICS code.

Table 2 illustrates how DOD and SBA apply the SBA size standard differently under their respective programs for selected NAICS codes. For example, a business that manufactures printed circuit assemblies would be eligible as a protégé under DOD’s Pilot Mentor-Protégé Program as long as it employed fewer than 375 employees; under SBA’s All Small Mentor-Protégé Program, the same business would be eligible as long as it employed no more than 750 employees.

Further, DOD’s Pilot Mentor-Protégé Program permits mentors to enter into agreements with more than one protégé. Under SBA’s All Small Mentor-Protégé Program, mentors generally are not permitted to have more than one protégé at a time. However, SBA may authorize a firm to mentor more than one protégé at a time if it can demonstrate that the additional mentor-protégé relationship will not adversely affect the development of either protégé firm (e.g., the second firm may not be a competitor of the first firm), but under no circumstances does SBA permit a mentor to have more than three protégés at one time.

In addition, DOD’s Pilot Mentor-Protégé Program receives appropriated funds, while SBA’s All Small Mentor-Protégé Program does not; as previously noted, DOD’s Pilot Mentor-Protégé Program received $28.3 million for fiscal year 2016. DOD uses its appropriated funding to provide direct reimbursement to mentors for providing developmental assistance to protégés. Incentive mechanisms are another key difference between DOD’s Pilot Mentor-Protégé Program and SBA’s All Small Mentor- Protégé Program. Under DOD’s program, mentors may receive direct reimbursement or credit toward subcontracting goals for providing developmental assistance to protégés. Under statute, generally a mentor in DOD’s Pilot Mentor-Protégé Program may not receive reimbursements for costs of assistance that exceed $1 million in a fiscal year. Under SBA’s All Small Mentor-Protégé Program, a mentor and protégé may agree to seek any government prime contract or subcontract jointly as a small business, provided the protégé qualifies as “small” for the procurement, and receive exclusion from the affiliation rules for the duration of the mentor-protégé agreement after both meeting certain regulatory requirements and obtaining SBA approval. In addition, SBA’s program permits federal agencies to provide incentives during the contract evaluation process to a mentor that will provide significant subcontracting work to its SBA-approved protégé firm, where appropriate.

Although the statutory provisions and program rules that determine mentor incentives differ, DOD’s Pilot Mentor-Protégé Program and SBA’s All Small Mentor-Protégé Program face similar inherent risk that mentors may receive benefits but may not necessarily provide developmental assistance to protégés as agreed to in their mentor-protégé agreements. Officials from both agencies said they take certain actions intended to mitigate this risk. For example, DOD requires an annual review of each mentor-protégé agreement to verify the performance information provided in the annual reports. Similarly, SBA reviews the protégé’s report on the mentor-protégé relationship and may decide not to continue the agreement if it finds that the mentor has not provided the agreed upon assistance or the assistance has not resulted in developmental gains for the protégé.

Appendix IV provides more details on the characteristics of DOD’s Pilot Mentor-Protégé Program and SBA’s All Small Mentor-Protégé Program.

DOD and SBA Plan to Consider Opportunities to Harmonize Their Programs Once Implementation of SBA’s New Program Is Complete

As previously noted, SBA’s All Small Mentor-Protégé Program is designed to apply to nearly all federal small business contracts, including those that small businesses currently participate in as part of other federal mentor-protégé programs. Under the new program, federal agencies will be required to be granted approval from the SBA Administrator in order to continue their existing mentor-protégé programs or to start new ones. By statute, federal departments and agencies with mentor-protégé programs in effect before January 2, 2013, are permitted to continue them until late July 2017 without being granted the SBA approval. However, after late summer 2017, approval must be granted to continue the program. Although it is too soon to tell whether the other federal agencies that currently administer mentor-protégé programs will seek to continue their own programs, SBA’s experience in implementing the All Small Mentor- Protégé Program may provide useful insights for harmonizing other federal mentor-protégé programs—that is, identifying and assessing differences and eliminating or reducing inconsistencies, where appropriate. For example, implementing SBA’s program may involve actions to potentially harmonize characteristics of mentor-protégé programs that serve different socioeconomic categories of small businesses and multiple agencies with various procurement needs.

Although the statute creating SBA’s program exempted DOD from the program’s requirements, the legislation implementing DOD’s program and the final rule implementing SBA’s program state that both programs are designed to enhance the capabilities of protégé firms to perform under federal government and commercial contracts. This general similarity in program purpose could facilitate harmonization between the programs. DOD and SBA officials said that opportunities to harmonize the programs may exist; however, they said it is too soon to explore harmonization because SBA is in the early stages of implementing the All Small Mentor- Protégé Program. Officials said that they plan to consider opportunities to harmonize their programs once implementation of SBA’s new program is complete later in 2017.

Conclusions

DOD’s Pilot Mentor-Protégé Program is intended to both (1) enhance the capabilities of eligible disadvantaged small business concerns to perform as subcontractors and suppliers under DOD contracts and other contracts and subcontracts and (2) increase the participation of such business concerns as subcontractors and suppliers under DOD contracts, other federal government contracts, and commercial contracts. DOD procedures require that mentor-protégé agreements contain certain elements to help provide assurance that the agreements support program purposes and that the participants meet program requirements, among other things. However, we found that program participants did not consistently include all of the required elements in these agreements and that DOD’s OSBP does not review the procedures that DOD components use to approve the agreements, as suggested by federal internal control standards. As a result, some agreements have been approved even though they were missing required elements, and DOD cannot ensure that these program requirements are serving their intended purposes.

In addition, DOD’s performance measures for the decades-old program lack performance goals and additional measures needed to effectively assess its Pilot Mentor-Protégé Program. Although DOD officials said they are considering changes to their performance measures, they had yet to make such changes as of January 2017. Until DOD establishes performance measures and related measurable goals consistent with the program’s stated purpose, it may be more difficult for DOD to fully assess the effectiveness of the Pilot Mentor-Protégé Program and to identify and prioritize potential improvements. In addition, Congress may not have the information it needs to make informed decisions on whether to reauthorize the pilot program, terminate it, or make it permanent.

Recommendations for Executive Action

To provide reasonable assurance that DOD’s Pilot Mentor-Protégé Program meets its mission, we recommend that the Director, DOD’s Office of Small Business Programs, take the following two actions: 1. Conduct periodic reviews of the processes DOD components follow to approve agreements and take oversight actions, as appropriate. 2. Complete actions to develop performance goals and related measures that are consistent with the program’s stated purpose.

Agency Comments

We provided a draft of this report to DOD and SBA for comment. In its written comments, which are summarized and reprinted in appendix V, DOD concurred with both recommendations. SBA provided technical comments, which we incorporated as appropriate.

DOD concurred with our recommendation to conduct periodic reviews of the processes DOD components follow to approve agreements and take oversight actions, as appropriate. DOD said it has drafted a standard operating procedure (SOP) that addresses GAO’s recommendations. In addition, DOD said that the SOP requires DOD component program managers to review mentor-protégé proposal packages prior to submitting them for review and concurrence. DOD also said the draft SOP includes a new checklist that identifies required elements, such as the North American Industry Classification System code, mentor approval letter, and fully executed copy of the mentor-protégé agreement.

DOD also concurred with our recommendation to complete actions to develop performance goals and related measures that are consistent with the program’s stated purpose. DOD stated that it is currently evaluating ways to develop measures that provide a better indication of program effectiveness for active mentor-protégé agreements and the 2 years following program participation. DOD noted that establishing baseline performance goals for all protégés to meet would not be advantageous due to the differences of each agreement and the small business needs of each respective DOD component; however, DOD stated that establishing a set of goals by tier group, for example, may better determine program performance as it would even the playing fields between large and small business concerns. Finally, DOD stated that it collects information from mentors on how they have enhanced the capabilities of protégés and plans to begin including these data in the annual report to Congress once performance measures and goals have been established.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, DOD’s Director, Office of Small Business Programs, and the SBA Administrator. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-8678 or shearw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

This report examines (1) the Department of Defense’s (DOD) procedures for approving mentor-protégé agreements; (2) DOD’s performance measures for the program; and (3) differences between DOD’s Pilot Mentor-Protégé Program and the Small Business Administration’s (SBA) All Small Mentor-Protégé Program and the agencies’ efforts to harmonize the two programs.

To address the first objective, we reviewed DOD’s current regulations and policies and procedures for approving mentor-protégé agreements, including changes mandated by the National Defense Authorization Act (NDAA) for Fiscal Year 2016. In addition, we reviewed guidance developed by three military services (the Departments of the Air Force, Army, and Navy) that administer and have the authority to approve mentor-protégé agreements. We also reviewed a randomly selected probability sample of 44 of the 78 active DOD mentor-protégé agreements in place as of June 2016 to assess how well DOD’s procedures provide reasonable assurance that mentor-protégé agreements meet program requirements.

With this probability sample, each member of the study population had a nonzero probability of being included, and that probability could be computed for any member. Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Since each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s results as a 95 percent confidence interval. This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn. All percentage estimates in this report are presented along with their margins of error at the 95 percent confidence level.

We interviewed DOD officials from the Office of Small Business Programs (OSBP) to discuss the procedures for approving mentor-protégé agreements and the controls in place to ensure that DOD components followed the procedures. We assessed DOD’s practices against Standards for Internal Control in the Federal Government. We also reviewed the NDAA for Fiscal Year 2016 to identify changes that the act made to the DOD Pilot Mentor-Protégé Program, such as amending the size requirement so that a protégé is less than half of the SBA size standard for its primary North American Industry Classification System code.

To address the second objective, we reviewed the measures and other information DOD reports on the performance of its Pilot Mentor-Protégé Program, including DOD’s current measures, which provide information related to the participation of protégés as subcontractors and suppliers under DOD and other contracts, and whether these measures address how the capabilities of protégé firms have been enhanced. This review included the annual reports prepared by the Defense Contract Management Agency (DCMA) for fiscal years 2011 through 2015 that provide aggregate data on protégé firms’ annual revenues, number of employees, and awards of prime contracts and subcontracts during their tenure in the program and for 2 years after they leave the program. We also reviewed the most recent semiannual report provided to DOD by the mentor and protégé for the randomly selected probability sample of mentor-protégé agreements we reviewed to examine the types of performance information included in these reports. We also interviewed DOD officials regarding their use of performance information. We compared DOD’s practices to those GAO previously identified as being associated with agencies that were successful in measuring their performance and federal internal control standards.

To address the third objective, we reviewed the DOD Pilot Mentor- Protégé Program’s implementing statute and subsequent legislative changes to the program, including those made by the NDAA for Fiscal Year 2016. We also reviewed SBA’s final rule which implemented provisions of the Small Business Jobs Act of 2010 and the National Defense Authorization Act for Fiscal Year 2013, which established the All Small Mentor-Protégé Program and the policies and procedures for SBA’s 8(a) Business Development Program. We identified selected characteristics of the programs, including the purpose, funding, number of agreements, eligibility requirements of mentors and protégés, authority to approve agreements, types of assistance mentors may provide to protégés, reporting requirements for mentors and protégés, and the mechanism used to encourage mentor participation in each program. We also identified differences between the programs for some of these characteristics and discussed them with DOD and SBA officials, including how differences between the programs could affect possible efforts to harmonize the programs.

In addition, we analyzed a randomly selected probability sample of 44 of the 78 active DOD mentor-protégé agreements in place as of June 2016 to identify characteristics of protégé firms. These characteristics include the number of years protégé firms have been in business, the number of employees of the protégé firms, annual revenues of the protégé firms, and their previous contracting experience. We also identified the types of developmental assistance mentor firms agreed to provide to protégé firms based on information reported for the randomly selected probability sample of agreements. These results can be found in appendix II.

We conducted this performance audit from January 2016 to April 2017, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Selected Characteristics of DOD Protégé Firms

Under the Department of Defense’s (DOD) Pilot Mentor-Protégé Program, mentor firms agree to provide eligible small business concerns (protégé firms) with technical, managerial, and other business development assistance under approved mentor-protégé agreements. Based on our review of a randomly selected probability sample of 44 of the 78 active DOD mentor-protégé agreements in place as of June 2016, we analyzed selected characteristics of protégé firms that enter the program and the types of developmental assistance that mentor firms agree to provide protégé firms.

As shown in figure 4, we estimate that over half (57 percent) of protégé firms have been in business for 10 years or less, and an estimated 21 percent have been in business 5 years or less. We also estimate that more than 33 percent of protégé firms have been in business for more than 15 years.

We also analyzed DOD mentor-protégé agreement documentation to determine the number of employees in protégé firms. We found that an estimated 77 percent of the protégé firms had 50 or fewer employees (see fig. 5).

In addition, we examined DOD mentor-protégé agreement documentation to determine protégé firms’ annual revenue. We found that an estimated 57 percent of protégé firms had annual revenue of $5 million or less. More than three-quarters of protégé firms had annual revenue of $10 million or less, and an estimated 17 percent of protégé firms had annual revenues above $10 million (see fig. 6).

We also analyzed DOD mentor-protégé agreement documentation to determine the protégé firms’ previous contracting experience. We found that an estimated 39 percent of protégé firms indicated having obtained a DOD prime contract in the 2 fiscal years before the proposed mentor- protégé agreement was submitted. We also found that an estimated 74 percent of protégé firms indicated having obtained DOD subcontracts in the 2 fiscal years before the proposed mentor-protégé agreement was submitted and that an estimated 26 percent indicated having received a federal agency (other than DOD) subcontract award from the mentor firm during that time. In addition, an estimated 53 percent of protégé firms indicated having received subcontract awards from their mentor firm in the 2 fiscal years before the proposed agreement was submitted. We also found that an estimated 13 percent of protégé firms indicated they did not have previous contracting experience (see fig. 7).

Based on our analysis of DOD mentor-protégé agreement documentation, we found that mentor firms anticipated providing some types of assistance more frequently than others. For example, we found that all mentor firms anticipated providing general business assistance to their protégé firms. In addition, we found that an estimated 93 percent of mentor firms anticipated awarding subcontracts to their protégé firms, and an estimated 80 percent anticipated obtaining assistance for their protégé firms through historically black colleges and universities, minority institutions of higher education, Procurement Technical Assistance Centers, or Small Business Development Centers. In contrast, an estimated 2 percent of mentor firms anticipated extending loans to protégé firms. We also found that an estimated 20 percent anticipated providing progress payments and an estimated 18 percent anticipated providing advance payments to their protégé firms (see fig. 8).

Appendix III: Selected Changes to DOD’s Pilot Mentor-Protégé Program in the NDAA for Fiscal Year 2016

The National Defense Authorization Act (NDAA) for Fiscal Year 2016 made a number of changes to the Department of Defense’s (DOD) Pilot Mentor-Protégé Program. In addition to the changes previously discussed, Section 861 of the NDAA for Fiscal Year 2016 also made the following changes and additions, among others:

Extended the DOD mentor-protégé program for 3 more years through fiscal year 2018 for the formation of new agreements and for the reimbursement of incurred costs and credit toward attaining subcontracting goals, under existing agreements incurred through fiscal year 2021.

Imposed a limit on when protégé firms can enter into mentor-protégé agreements under the program to only during the 5-year period beginning on the date the protégé firm enters into its first mentor- protégé agreement.

Changed program eligibility requirements to include that a protégé firm may, among other things, be a small business concern that is either a nontraditional defense contractor or currently provides goods or services in the private sector that are critical to enhancing the capabilities of the defense supplier base and fulfilling key DOD needs.

Required mentor firms to report at least once a fiscal year any new awards of subcontracts on a competitive or noncompetitive basis to the protégé firm under DOD contracts or other contracts, including the value of such subcontracts.

Required mentor firms to report at least once a fiscal year any assistance obtained on behalf of the protégé from Small Business Development Centers, certain entities providing procurement technical assistance, or historically black colleges or universities or minority institutions of higher education.

Required the Office of Small Business Programs (OSBP) to review reports submitted by mentor firms on specific statutory topics and if a mentor-protégé agreement is not furthering the purpose of the program, decide not to approve any continuation of the agreement.

Removed “business development” assistance from the permissible mentor firm personnel provided assistance and removed cash in exchange for an ownership interest in the protégé firm from the types of assistance a mentor firm may provide to a protégé firm.

Revised the definitions of the term ‘‘disadvantaged small business concern” limiting it to firms that have less than half the size standard corresponding to their primary North American Industry Classification System code.

Changed affiliation rules by explicitly stating that the presence of certain family relationships between the owners and managers in the mentor and protégé firms constitutes affiliation and thereby is not allowed.

Appendix IV: Selected Characteristics of DOD’s Pilot Mentor-Protégé and SBA’s Small Business Mentor-Protégé Programs

Department of Defense (DOD) Pilot Mentor-Protégé Program

Enhance the capabilities of eligible small business concerns to perform as subcontractors and suppliers under DOD contracts and other contracts and subcontracts, and increase the participation of such business concerns as subcontractors and suppliers under DOD contracts, other federal government contracts, and commercial contracts.

Small Business Administration (SBA) 8(a) Business Development (BD) Mentor-Protégé Program Enhance the capabilities of the protégé (an 8(a) BD Program participant), assist the protégé with meeting the goals established in its SBA-approved business plan, and improve the protégé’s ability to successfully compete for contracts.

SBA All Small Mentor-Protégé Program Enhance the capabilities of protégé firms by requiring approved mentors to provide business development assistance to protégé firms and to improve the protégé firms’ ability to successfully compete for federal contracts.

28.3 million (fiscal year 2016) 78 (as of June 2016)

465 (as of November 2016)

45 (as of November 2016)

Protégé must be a small disadvantaged business, a business entity owned and controlled by an Indian tribe or a Native Hawaiian Organization, a women-owned business, a Historically Underutilized Business Zone (HUBZone) small business, service-disabled veteran-owned small business, a small business that statutorily qualifies as a nontraditional defense contractor, or an eligible entity employing the severely disabled. Protégé must qualify as small by being less than half the size standard corresponding to its primary North American Industry Classification System (NAICS) code. Mentor may rely on written representation that the protégé qualifies as small when entering into a mentor-protégé agreement but must confirm a protégé’s status as a HUBZone small business concern.

Protégé must (1) qualify as small for the size standard corresponding to its primary North American Industry Classification System (NAICS) code or identify that it is seeking business development assistance with respect to a secondary NAICS code and qualify as small for the size standard corresponding to that NAICS code; and (2) demonstrate how the business development assistance to be received through its proposed mentor-protégé relationship would advance the goals and objectives set forth in its business plan.

Protégé must qualify as small for the size standard corresponding to its primary North American Industry Classification System (NAICS) code or identify that it is seeking business development assistance with respect to a secondary NAICS code and qualify as small for the size standard corresponding to that NAICS code. A firm may self-certify that it qualifies as small for its primary or identified secondary NAICS code. When the firm is not small for the primary NAICS code and seeks to qualify for the secondary NAICS code, the firm must demonstrate how the mentor-protégé relationship is a logical business progression for the firm and will further develop or expand current capabilities. SBA will not approve a relationship in a secondary NAICS code in which the firm has no prior experience.

Department of Defense (DOD) Pilot Mentor-Protégé Program

A mentor, among other requirements, must be: (1) not affiliated with the protégé firm prior to the approval of the agreement; (2) demonstrate that it is qualified to provide assistance that will contribute to the purpose of the program and is in good financial health and character and does not appear on a federal list of debarred or suspended contractors; (3) an entity other than small business that is a DOD prime contractor with an active subcontracting plan (unless a waiver to the small business exception has been obtained from the Director, Small Business Programs, Office of the Undersecretary of Defense for Acquisition, Technology, and Logistics); or (4) a graduated 8(a) firm that provides documentation of its ability to serve as a mentor; and (5) approved to participate as a mentor.

Small Business Administration (SBA) 8(a) Business Development (BD) Mentor-Protégé Program Mentor must (1) be capable of carrying out its responsibilities to assist the protégé firm under the proposed mentor-protégé agreement; (2) possesses good character; (3) not appear on the federal list of debarred or suspended contractors; and (4) be able to impart value to a protégé due to lessons learned and practical experience gained because of the 8(a) BD program, or through its knowledge of general business operations and government contracting.

Mentor must demonstrate that it (1) is capable of carrying out its responsibilities to assist the protégé under the proposed mentor-protégé agreement; (2) possesses good character; (3) does not appear on the federal list of debarred or suspended contractors; and (4) can impart value to a protégé firm due to lessons learned and practical experience gained or through its knowledge of general business operations and government contracting.

Director, Small Business Programs, for each military department or defense agency generally approves mentor- protégé agreements.

The Associate Administrator for Business Development or his/her designee must approve the mentor-protégé agreement. The agreement will not be approved if SBA determines that the assistance to be provided is not sufficient to promote any real developmental gains to the protégé, or if SBA determines that the agreement is merely a vehicle to enable the mentor to receive 8(a) contracts.

The Associate Administrator for Business Development or his/her designee must approve the mentor-protégé agreement. The agreement will not be approved if SBA determines that the assistance to be provided is not sufficient to promote any real developmental gains to the protégé, or if SBA determines that the agreement is merely a vehicle to enable the mentor to receive small business contracts.

Department of Defense (DOD) Pilot Mentor-Protégé Program

A mentor may enter into agreements with more than one protégé.

Small Business Administration (SBA) 8(a) Business Development (BD) Mentor-Protégé Program Generally, a mentor will have no more than one protégé at a time. However, SBA may authorize a concern to mentor more than one protégé at a time where it can demonstrate that the additional mentor-protégé relationship will not adversely affect the development of either protégé firm (e.g., the second firm may not be a competitor of the first firm). Under no circumstances will a mentor be permitted to have more than three protégés at one time between the SBA 8(a) Business Development Mentor-Protégé Program and the SBA All Small Mentor-Protégé Program. SBA may authorize a participant to be both a mentor and a protégé at the same time where the participant can demonstrate that the second relationship will not compete or otherwise conflict with the first mentor- protégé relationship.

SBA All Small Mentor-Protégé Program Generally, a mentor will have no more than one protégé at a time. However, SBA may authorize a concern to mentor more than one protégé at a time where it can demonstrate that the additional mentor-protégé relationship will not adversely affect the development of either protégé firm (e.g., the second firm may not be a competitor of the first firm). Under no circumstances will a mentor be permitted to have more than three protégés at one time between the SBA 8(a) Business Development Mentor Protégé Program and the SBA All Small Mentor-Protégé Program. SBA may authorize a participant to be both a mentor and a protégé at the same time where the participant can demonstrate that the second relationship will not compete or otherwise conflict with the first mentor- protégé relationship.

Protégé may only have one active DOD mentor-protégé agreement.

Protégé firm may generally have only one mentor at a time. SBA may approve a second mentor for a particular protégé firm where additional relationship will not compete/conflict with the existing mentor-protégé relationship and either (1) the added relationship pertains to an unrelated secondary NAICS code or (2) the protégé is seeking expertise the existing mentor does not possess.

Protégé firm may generally have only one mentor at a time. SBA may approve a second mentor for a particular protégé firm where additional relationship will not compete/conflict with the existing mentor-protégé relationship and either (1) the added relationship pertains to an unrelated secondary NAICS code or (2) the protégé is seeking expertise the existing mentor does not possess.

Department of Defense (DOD) Pilot Mentor-Protégé Program

Mentor may, among other actions, provide management and/or technical assistance, award subcontracts, make loans, make advance and/or progress payments under subcontracts, and obtain assistance from Small Business Developmental Centers, Procurement Technical Assistance Centers, historically black colleges and universities, or minority institutions of higher education.

Small Business Administration (SBA) 8(a) Business Development (BD) Mentor-Protégé Program Mentor may provide management and/or technical assistance, award subcontracts, trade education, make loans and/or equity investments, and cooperate on joint venture projects.

Mentor may provide management and/or technical assistance, award subcontracts, trade education, make loans and/or equity investments, and cooperate on joint venture projects.

Mentors must report on the progress made under active mentor-protégé agreements semiannually, including (1) dollars obligated (for reimbursable agreements); (2) expenditures; (3) dollars credited toward applicable subcontracting goals, if any; (4) number and dollar value of subcontracts awarded to the protégé firm; (5) description of developmental assistance provided, including milestones achieved; (6) impact of agreement in terms of capabilities enhanced, certifications received, and/or technology enhanced. Protégés must report data annually on the progress made in employment, revenues, and participation in DOD contracts. The protégé report may be provided as part of the mentor report.

In its annual business plan update, the protégé must report to SBA for the protégé’s preceding program year (1) all technical and/or management assistance provided by the mentor to the protégé; (2) all loans to and/or equity investments made by the mentor in the protégé; (3) all subcontracts awarded to the protégé by the mentor, and the value of each subcontract; (4) all federal contracts awarded to the mentor/protégé relationship as a joint venture (designating each as an 8(a), small business set aside, or unrestricted procurement), the value of each contract, and the percentage of the contract performed and the percentage of revenue accruing to each party to the joint venture; and (5) a narrative describing the success such assistance has had in addressing the developmental needs of the protégé and addressing any problems encountered.

Protégés must report to SBA annually, within 30 day of the agreement approval anniversary, (1) all technical and/or management assistance provided by the mentor to protégé; (2) all loans to and/or equity investments made by the mentor in the protégé; (3) all subcontracts awarded to the protégé by the mentor and all subcontracts awarded to the mentor by the protégé, and the value of each subcontract; (4) all federal contracts awarded to the mentor-protégé relationship as a joint venture, the value of each contract, and the percentage of the contract performed and the percentage of revenue accruing to each party to the joint venture; and (5) a narrative describing the success such assistance has had in addressing the developmental needs of the protégé and addressing any problems encountered. Additionally, the protégé must report the mentoring services it receives by category and hours.

Department of Defense (DOD) Pilot Mentor-Protégé Program

DOD may provide mentors with either cost reimbursement or credit against applicable subcontracting goals established under contracts with DOD or other federal agencies.

Small Business Administration (SBA) 8(a) Business Development (BD) Mentor-Protégé Program A mentor and protégé may participate in a joint venture as a small business for any government prime contract or subcontract, including procurements with a dollar value less than half the size standard corresponding to the assigned NAICS code and 8(a) sole source contracts, provided the protégé qualifies as small for the procurement and additional provisions for 8(a) sole source contracts.

A mentor and protégé may participate in a joint venture as a small business for any government prime contract or subcontract, provided the protégé qualifies as small for the procurement. Such a joint venture may seek any type of small business contract (i.e., small business set-aside, 8(a), HUBZone, service-disabled veteran-owned small business, or women-owned small business) for which the protégé firm qualifies and within certain requirements, receives exclusion from the affiliation rules for the duration of the joint venture agreement. Procuring activities may provide incentives in the contract evaluation process to a mentor that will provide significant subcontracting work to its SBA- approved protégé firm, where appropriate.

Appendix V: Comments from the Department of Defense

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Marshall Hamlett (Assistant Director), Abiud Amaro Diaz, Isidro Gomez, Farrah Graham, Marc Molino, Jessica Sandler, and Jennifer Schwartz made key contributions to this report.