Small Business Contracting: DOD Should Take Actions to Ensure That Its Pilot Mentor-Protégé Program Enhances the Capabilities of Protégé Firms
Released: 2017-04-11T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: DOD's Pilot Mentor-Protégé Program, was first authorized as a pilot program in 1990, and has been repeatedly renewed as a pilot program, most recently through September 30, 2018. For fiscal year 2016, total funding for this program was $28.3 million.

The joint explanatory statement to accompany the National Defense Authorization Act for Fiscal Year 2016 includes a provision for GAO to report on DOD's pilot program. This report examines, among other things, (1) DOD's procedures for approving mentor-protégé agreements and (2) DOD's performance measures for the program. GAO analyzed DOD guidance, reviewed a randomly selected probability sample of active DOD mentor-protégé agreements and estimated their completeness at a 95 percent confidence interval, reviewed DOD's annual program reports for fiscal years 2011 through 2015, and interviewed agency officials.

section_title: What GAO Found
paragraphs: The Department of Defense (DOD) relies on military services and agencies (DOD components) to approve the agreements that establish relationships between participants in its Pilot Mentor-Protégé Program. This program provides incentives for major defense contractors (mentors) to provide assistance to small disadvantaged firms (protégés) in an effort to enhance their capability to compete for federal and commercial contracts. However, DOD does not have reasonable assurance that approved agreements include all elements required by the program's regulations and policies. These elements include, among others, the protégé's industry code and the signature and date of the mentor and protégé. These elements serve a variety of purposes—for example, the industry code is used to determine the protégé's eligibility to participate in the program, and the signature and date of mentor and protégé are required in order for the agreement to be legally bindingto meet program requirements. Based on a review of a randomly selected probability sample of 44 of the 78 active mentor-protégé agreements in place as of June 2016, GAO estimates that 27 percent of agreements were missing required elements. For example, GAO estimates that 25 percent of agreements were not signed by both the mentor and protégé. Federal internal control standards state that management should implement control activities through policies and practices, including periodically reviewing control activities for continued effectiveness. DOD's Office of Small Business Programs (OSBP) manages the program and oversees program policies and procedures. However, OSBP does not review the DOD components' processes for approving mentor-protégé agreements and therefore has not taken appropriate oversight actions to provide reasonable assurance that agreements meet all requirements. As a result, the components have approved agreements that do not include required elements, and OSBP cannot ensure that the requirements are serving their various purposes.

DOD's fiscal year 2011 through 2015 annual reports on its Pilot Mentor-Protégé Program include performance measures for several areas, but DOD lacks performance goals and other measures needed to effectively assess the program. Some of these measures show that during this period, protégés' revenue, number of employees, and DOD prime and subcontract awards increased while protégés participated in the program, but revenues and employment levels decreased in the 2 years after their participation ended. GAO found that DOD has not established any measurable goals for these measures. In addition, DOD collects information from mentors on how they have enhanced the capabilities of protégés, but DOD does not include this information in the program's annual report and has not developed performance measures or goals related to this information. GAO has previously identified performance measurement as a best practice that allows organizations to track progress and gives managers information to plan needed improvements. DOD officials told GAO they are working to develop measures that better indicate program outcomeseffectiveness, but as of January 2017 they had not established such measures. Without performance goals and related measures, DOD may be limited in its ability to analyze the effectiveness of the program, and Congress may not have information needed to inform future decisions regarding the program.

section_title: What GAO Recommends
paragraphs: GAO recommends that DOD (1) conduct periodic reviews of the components' processes for approving agreements and address identified deficiencies, as appropriate, and (2) develop performance goals and related measures that are consistent with the program's stated purpose. DOD concurred with GAO's recommendations.

section_title: Letter
paragraphs: The Department of Defense (DOD) Pilot Mentor-Protégé Program  provides incentives for major defense contractors (mentors) that agree to  provide technical, managerial, and other business development  assistance to small business firms (protégés) under approved mentor- protégé agreements. The assistance provided under these agreements is  intended to strengthen protégés’ capabilities to perform as subcontractors  and suppliers under defense and other federal contracts, as well as  commercial contracts. Congress first authorized the program as a pilot  program in 1990, and at that time, the program was scheduled to expire  in 1994.  However, the program has been renewed as a pilot program  since 1990; the most recent renewal authorizes the program through  September 30, 2018. In addition, DOD’s program is the only federal  Mentor-Protégé Program that receives appropriated funding.  According  to DOD, the total funding amount for the program was $28.3 million for  fiscal year 2016. As of June 2016, the program had 78 active mentor- protégé agreements in place.

The Small Business Administration (SBA) administers two separate  mentor-protégé programs. SBA’s 8(a) Business Development Mentor- Protégé Program, which the agency established through regulation in  1998, is designed to enable successful firms to provide various forms of  business development assistance to 8(a) Business Development  Program participants.  In addition, SBA’s new All Small Mentor-Protégé  Program was established in summer 2016 in accordance with provisions  in the Small Business Jobs Act of 2010 and the National Defense  Authorization Act (NDAA) for Fiscal Year 2013. Where the 8(a) Business  Development Mentor-Protégé Program is open only to participants in  SBA’s 8(a) Business Development Program, the All Small Mentor- Protégé Program is open to all small businesses. SBA began accepting  applications for the new All Small Mentor-Protégé Program on October 1,  2016.

The joint explanatory statement to accompany the NDAA for Fiscal Year  2016 contains a provision for GAO to report on DOD’s Pilot Mentor- Protégé Program. This report examines (1) DOD’s procedures for  approving mentor-protégé agreements; (2) DOD’s performance measures  for the program; and (3) differences between DOD’s Pilot Mentor-Protégé  Program and SBA’s newly established All Small Mentor-Protégé Program  and the agencies’ efforts to harmonize the two programs.

To address the first objective, we reviewed DOD’s current regulations,  policies, and procedures for approving mentor-protégé agreements,  including changes mandated by the NDAA for Fiscal Year 2016 related to  DOD’s process for approving such agreements. In addition, we reviewed  guidance developed by selected DOD components that administer and  have the authority to approve mentor-protégé agreements.  We then  assessed whether DOD components approved mentor-protégé  agreements in compliance with DOD’s current regulations, policies, and  procedures. We also interviewed DOD officials from the Office of Small  Business Programs (OSBP) regarding DOD’s procedures for approving  proposed mentor-protégé agreements and controls to ensure that the  procedures are being followed. We assessed DOD’s practices against  Standards for Internal Control in the Federal Government.  Further, we  reviewed a randomly selected probability sample of 44 of the 78 active  DOD mentor-protégé agreements in place as of June 2016. With this  probability sample, each member of the study population had a nonzero  probability of being included, and that probability could be computed for  any member.

To address the second objective, we reviewed DOD’s performance  measures and information DOD collects and reports on the performance  of its Pilot Mentor-Protégé Program. In addition, we reviewed semiannual  reports provided to DOD by mentor and protégé firms to examine the  information DOD collects on performance under approved mentor- protégé agreements. We compared DOD’s practices to those GAO has  previously identified as being associated with agencies that were  successful in measuring their performance and federal internal control  standards.  We also interviewed DOD officials from OSBP regarding the  performance information DOD collects and reports for the Pilot Mentor- Protégé Program.

To address the third objective, we reviewed the DOD Pilot Mentor- Protégé Program’s implementing statute and subsequent legislative  changes to the program. We also reviewed SBA’s final rule to implement  the All Small Mentor-Protégé Program. We identified differences between  the DOD and SBA programs and interviewed DOD and SBA officials to  discuss the similarities and differences we identified, including how  differences between the programs could affect possible efforts to  harmonize the programs. See appendix I for a more detailed description  of our scope and methodology.

In addition, we identified certain characteristics of protégé firms from the  randomly selected probability sample of 44 of the 78 active DOD mentor- protégé agreements in place as of June 2016. We also identified the  types of developmental assistance mentor firms agreed to provide to  protégé firms based on information reported for the randomly selected  probability sample of agreements. The results of this analysis can be  found in appendix II.

We conducted this performance audit from January 2016 to April 2017, in  accordance with generally accepted government auditing standards.

Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: DOD’s Pilot Mentor- Protégé Program
paragraphs: The purpose of DOD’s Pilot Mentor-Protégé Program, as established in  statute, is to provide incentives to major DOD contractors to furnish  eligible disadvantaged small business concerns with assistance designed  to (1) enhance the capabilities of eligible disadvantaged small business  concerns to perform as subcontractors and suppliers under DOD  contracts and other contracts and subcontracts; and (2) increase the  participation of such business concerns as subcontractors and suppliers  under DOD contracts, other federal government contracts, and  commercial contracts.  DOD’s Office of Small Business Programs  (OSBP) manages the program, including developing and overseeing the  policies and procedures for the program.  In addition, a number of DOD  components administer individual mentor-protégé agreements.  DOD  components and the mentors and protégés that participate in the program  are required to follow DOD’s regulations and policies and procedures. For  example, DOD’s regulations state that the director of small business  programs for each DOD component has the authority to approve firms as  mentors and to approve mentor-protégé agreements, among other things.  OSBP has developed tools to facilitate program administration, including  a template to assist mentors and protégés in developing proposed  agreements and a checklist to assist DOD components with ensuring that  proposed agreements address required elements, which we discuss later  in this report. However, DOD components and mentors and protégés are  not required to use the tools.

DOD’s Pilot Mentor-Protégé Program offers two types of agreements.  Reimbursable agreements provide payments directly to the mentor for the  cost of providing developmental assistance to the protégé. Credit  agreements provide mentors with credit toward their subcontracting goals  for providing developmental assistance to protégés.  Mentors can  provide protégés with various types of assistance under mentor-protégé  agreements, including general business management, subcontract  awards, progress payments, advance payments, and loans.  Mentors  may also obtain assistance for protégés from Small Business  Development Centers, Procurement Technical Assistance Centers,  historically black colleges and universities, and minority institutions of  higher education (see fig. 1).

DOD policies and procedures also require mentors to report on the  progress made under active mentor-protégé agreements in semiannual  reports. Mentors must also include in semiannual reports, among other  things, (1) any assistance obtained by the mentor firm for the protégé firm  from Small Business Development Centers, Procurement Technical  Assistance Centers, historically black colleges and universities, and  minority institutions of higher education for developmental assistance  provided to protégés, (2) dollars credited (if any) toward applicable  subcontracting goals as a result of developmental assistance provided to  the protégé, and (3) the impact of the agreement in terms of capabilities  enhanced, certifications received, or technology transferred. In addition,  annual reports are to contain data on the protégé’s employment, revenue,  and participation in DOD contracts. DOD procedures also require  protégés to report on the progress made in each of the 2 years following  the completion of their agreement. Further, DOD procedures require the  Defense Contract Management Agency (DCMA), another DOD  component that manages agreements for the department, to conduct a  performance review annually of each mentor-protégé agreement.

DOD’s process for establishing and approving mentor-protégé  agreements generally consists of nine steps (as illustrated in fig. 2):  1.  Mentor seeks approval to participate in program. A firm that is  interested in participating in the DOD Pilot Mentor-Protégé Program  must submit an application for approval as a mentor to the Director of  OSBP. To be eligible as a mentor, among other factors, a firm must  not be affiliated with the protégé prior to the approval of the  agreement and must demonstrate it is qualified to provide assistance  that will contribute to the program’s purpose, among other  requirements.  2.  Mentor establishes a counterpart. The mentor is solely responsible for  selecting the protégé. To be eligible as a protégé, a firm must be,  among other things, a women-owned small business, Historically  Underutilized Business Zone (HUBZone) small business, service- disabled veteran-owned small business, or an eligible entity  employing the severely disabled.  3.  Mentor conducts a needs assessment. Once the mentor selects the  protégé, the mentor conducts a preliminary assessment of the  protégé’s developmental needs. The mentor and protégé then  mutually agree on the developmental assistance that the mentor is to  provide to address those needs.  4.  Parties determine the type of agreement. The mentor and protégé  may apply for either a reimbursable mentor-protégé agreement or a  credit agreement.  5.  Parties develop the agreement. The mentor and protégé develop a  proposed mentor-protégé agreement that must address a number of  required elements, which we discuss in greater detail later in this  report.  6.  Mentor submits proposed agreement for review. The mentor submits  the proposed agreement to the cognizant DOD component for review  and approval.  7.  DOD component approves or denies agreement. As previously noted,  the director of small business programs for each DOD component has  the authority to approve mentor-protégé agreements.  8.  Parties start the agreement. The mentor and protégé begin the  agreement, which cannot generally exceed 3 years.  9.  Parties report on progress. The mentor is required to submit a  semiannual report on the progress made under an active mentor- protégé agreement throughout the term of an agreement. The protégé  is required to submit an annual report on progress made during each  fiscal year of the program term and for each of the 2 fiscal years  following the expiration of the agreement.

The NDAA for Fiscal Year 2016 made several changes to DOD’s Pilot  Mentor-Protégé Program.  For example, with respect to the relationship  between a mentor firm and protégé firm, the act strengthened affiliation  rules by explicitly stating that the presence of certain family relationships  between owners and managers in both the mentor and protégé firms  constitutes affiliation and thereby is not allowed. In addition, the act  amended the size requirement so that DOD’s size standard for protégés  is set at less than half of that used by SBA. See appendix III for additional  information on selected changes to the program made by the NDAA for  Fiscal Year 2016.

section_title: SBA’s All Small Mentor- Protégé Program
paragraphs: According to SBA, the All Small Mentor-Protégé Program is designed to  apply to all federal small business contracts, including those that small  businesses currently participate in under other federal mentor-protégé  programs. In addition, according to SBA, the rule establishing the  regulations for the program is intended to be consistent with the  regulations for SBA’s 8(a) Business Development Mentor-Protégé  Program.  SBA’s All Small Mentor-Protégé Program includes certain  requirements for the mentor-protégé programs of other federal agencies,  but, as required by federal law, the program exempts DOD’s Pilot Mentor- Protégé Program from these requirements. For example, the final rule  states that the head of a department or agency must submit a plan to  SBA for any previously existing mentor-protégé program that the  department or agency seeks to continue within 1 year of the effective date  of the rule (or by August 24, 2017). In addition, the final rule states that  a federal department or agency may not start a new mentor-protégé  program unless the head of the department or agency submits a plan to  the SBA Administrator for the program and the SBA Administrator  approves the plan. The incentives provided to mentors under SBA’s  programs differ from those provided under DOD’s Pilot Mentor-Protégé  Program, as we discuss in greater detail later in this report.

section_title: DOD’s Procedures Do Not Provide Reasonable Assurance That Mentor-Protégé Agreements Contain All Required Elements
paragraphs: DOD components have the authority to approve mentor-protégé  agreements, and they are required to follow the regulations, policies, and  procedures DOD has established for the program. However, DOD lacks  controls needed to provide reasonable assurance that mentor-protégé  agreements approved by DOD components contain all elements required  by DOD policies and regulations.

DOD’s regulations and OSBP procedures do not prescribe the process  DOD components should follow to approve mentor-protégé agreements.  Instead, DOD’s OSBP has allowed DOD components to develop their  own detailed procedures. As shown in the following examples, these  procedures differ in terms of some elements, such as criteria and time  frames:

Air Force. The Department of the Air Force (Air Force) issues an  announcement each fiscal year that outlines its process for  considering proposed mentor-protégé agreements. The fiscal year  2016 announcement describes a two-step solicitation process. For  fiscal year 2016, the Air Force was first to solicit white papers from  prospective mentors describing, among other things, the proposed  developmental assistance the mentor planned to provide to the  protégé and the anticipated benefits of the agreement to the Air Force  and protégé. Second, the Air Force was to invite mentors with white  papers found to be consistent with the program goals to submit a  formal mentor-protégé agreement proposal. As part of the proposal,  mentors were required to submit a mentor-protégé agreement using  the template found on OSBP’s website. The announcement also  stated that the Air Force planned to consider four factors when  determining whether to recommend proposals for award: (1)  relevance of proposed technology to current Air Force requirements,  (2) overall technical approach to providing transfer of technology and  developmental assistance to the protégé, (3) the mentor’s capacity  and capability to achieve the objectives of the mentor-protégé  program, and (4) estimated cost of the proposed assistance.

Army. The Department of the Army (Army) guidance states that  proposed mentor-protégé agreements are to be approved and funded  twice each fiscal year. The guidance states that the Army intends to  approve and fund proposed agreements that have a strong technical  component or focus on the innovative transfer of state-of-the-art  technology that supports Army troops. The Army guidance further  states that mentors are required to submit a mentor-protégé  agreement using the template found on OSBP’s website. The  guidance notes that the Army is to consider the following factors in  descending order of importance when evaluating proposed mentor- protégé agreements: (1) subcontracting and prime contractor  opportunities, (2) technical approach for developmental assistance,  (3) involvement of historically black colleges and universities and  minority institutions, (4) relevance to Army programs, (5) corporate  capability and commitment, (6) management plan, and (7) mentor and  protégé past and present performance.

Navy. The Department of the Navy (Navy) guidance states that the  prospective mentor and protégé are required to schedule an informal  introductory briefing with the Navy’s director of small business  programs to discuss the proposed concept and objective of the  mentor-protégé agreement. As part of the briefing, the mentor and  protégé are required to discuss, among other things, the mentor’s  experience providing developmental assistance under other mentor- protégé agreements; the proposed benefits to the mentor, protégé,  and the Navy under the proposed agreement; and the rationale for  why the proposed agreement should be sponsored. The Navy is to  then provide the mentor and protégé with feedback on whether to  submit a proposed mentor-protégé agreement. The Navy’s guidance  also states that the Navy accepts mentor-protégé agreement  proposals during three time frames each fiscal year (December 1-31,  March 1-31, and August 1-31). As part of the proposal, mentors are  required to submit a mentor-protégé agreement using the template  found on OSBP’s website. The guidance further states that the Navy  is to consider the following factors in descending order of importance  when evaluating proposed mentor-protégé agreements: (1) merit of  the technology transfer to the protégé, (2) perceived benefit of the  agreement to the Navy, (3) perceived benefit of the agreement to the  protégé, (4) percentage of hours associated with technology transfer,  (5) subcontracting opportunities available to the protégé, (6) utilization  of historically black colleges and universities and minority institutions,  Procurement Technical Assistance Centers, and Small Business  Development Centers, and (7) proposed cost.

Although DOD components have flexibility in considering a range of  factors when approving mentor-protégé agreements, the agreements  themselves are required to include certain elements that are defined in  DOD’s procedures, and these elements serve a variety of purposes. For  example, an agreement must include the protégé’s primary North  American Industry Classification System (NAICS) code, which is used to  determine the protégé’s eligibility to participate in the program. In  addition, the agreement must describe the developmental assistance  planned for the protégé, how the assistance is to address the protégé’s  identified needs and enhance its ability to perform successfully under  contracts or subcontracts within DOD and other federal agencies, and  factors to assess the protégé firm’s developmental progress under the  program, including specific milestones for providing assistance. The  agreement must also specify a program participation term that does not  exceed 3 years, which provides assurance that the mentor provides  assistance to the protégé in a timely manner.  Further, the agreement  also is required to be signed and dated by the mentor and protégé, which  provides evidence of meeting the program requirement between the two  parties. Figure 3 presents a complete list of the elements that mentors  and protégés are required to include in their agreements.

Our review of a randomly selected probability sample of 44 of the 78 total  active DOD mentor-protégé agreements in place as of June 2016 found  that a number of these agreements were missing required elements.  Specifically, based on our review, we estimate that 27 percent of the  agreements did not address all required elements.  With respect to  specific elements, we estimate that 25 percent of the agreements did not  include the signature of the mentor and protégé, 9 percent did not include  the protégé’s primary NAICS code, and 7 percent did not include an  anticipated start and end date for the agreement. These missing elements  suggest that the components’ procedures for approving mentor-protégé  agreements do not provide reasonable assurance that agreements are  completed in accordance with DOD requirements.

According to officials from OSBP, which manages DOD’s Pilot Mentor- Protégé Program, the DOD component that approves a mentor-protégé  agreement is responsible for providing assurance that the agreement  includes the required elements in accordance with the program’s policies.  However, DOD’s procedures do not define how DOD components should  review these proposals. OSBP officials told us that OSBP developed a  checklist for DOD components to use to document the review of proposed  mentor-protégé agreements. The checklist includes whether the mentor  and protégé have signed the agreement, whether the mentor and protégé  meet the program’s eligibility requirements, and whether the agreement  includes a description and milestones for the developmental assistance  that is to be provided under the agreement, among other things. A senior  OSBP official added that OSBP does not require DOD components to use  the checklist, nor does OSBP periodically review the actions DOD  components have taken to review mentor-protégé agreements for  approval because OSBP does not want to micromanage the components.  Instead, the official said that OSBP relies on DOD components to ensure  that approved mentor-protégé agreements include all required elements,  and in some cases, DOD components have developed their own  checklist.

Federal internal control standards state that management should  implement control activities through policies, and that this principle may  be applied by, among other things, periodically reviewing control activities  for continued relevance and effectiveness in achieving the entity’s  objective or addressing related risks.  As part of its responsibility for  managing the Pilot-Mentor Protégé Program, OSBP is ultimately  responsible for overseeing the program’s policies and procedures.  Because OSBP does not review the procedures established by DOD  components for approving proposed mentor-protégé agreements or  conduct its own monitoring of the components, it does not have  assurance that the agreements approved by the components include all  required elements. As a result, OSBP cannot ensure that the program  requirements are serving their intended purposes—for example, as  previously discussed, missing industry codes are used to determine  whether protégés are eligible to participate in the program, and the  signatures of the mentor and protégé are required for the agreement to  meet the program requirements.

section_title: DOD Lacks Performance Goals and Measures to Facilitate Effective Program Assessment
paragraphs: Although DOD has established some performance measures for its Pilot  Mentor-Protégé Program, the department lacks performance goals and  other measures needed to fully assess the program. DOD prepares an  annual report for the program related to the progress of protégés under  active mentor-protégé agreements and protégés that completed or  otherwise terminated participation in the program during the preceding 2  fiscal years. The report includes specific performance measures that  describe changes in protégés’ annual revenue, number of employees,  number of DOD prime contract awards, and number of DOD subcontract  awards.  DOD’s annual reports show that protégés experienced  increased revenues, number of employees, and DOD prime contracts and  subcontracts while participating in the program from fiscal years 2011  through 2015. In contrast, the reports show that protégés that left the  program during this period experienced decreased revenues and  employees in the 2 years following program participation (see table 1).

However, we found that DOD lacks performance goals and additional  measures needed to effectively assess its Pilot Mentor-Protégé Program.  Specifically, DOD has not established any measurable goals for the  performance information it reports for the program. Performance  measurement is the ongoing monitoring and reporting of program  accomplishments, particularly progress toward pre-established goals. We  have previously identified performance measurement as a best practice  that allows organizations to track progress in achieving their goals and  gives managers crucial information to identify gaps in program  performance and plan any needed improvements. In addition, according  to federal internal control standards, managers need to compare actual  performance against planned or expected results and analyze significant  differences. While DOD collects and reports information on protégés’  annual revenue, employment levels, and prime contract and subcontract  awards, it has not established goals for these measures. As a result,  DOD is limited in its ability to use this information to assess program  performance.

In addition, DOD has not established measures for one part of the  program’s statutory purpose. The information that DOD currently reports  relates to the part of the program purpose to increase the participation of  small businesses in DOD and other federal government contracts. As  previously discussed, part of the program’s purpose is also to enhance  the capabilities of eligible small businesses to perform as subcontractors  and suppliers under DOD and other contracts and subcontracts, and the  current program measures do not address this part of the program’s  purpose. Mentors report information to DOD in semiannual reports on  how the capabilities of protégés were enhanced, what certifications the  protégés obtained, and what technology the mentors transferred to the  protégés. However, DOD does not include this information in its annual  report for the program, and it has not developed performance measures  or goals related to this information. As a result, DOD is not using the  information it collects to fully assess how well the program is enhancing  the capabilities of eligible small businesses to perform under DOD and  other contracts, and, therefore, not fully addressing the purpose of the  program.

OSBP officials told us that they have not considered setting targets each  year for the program’s current performance measures because these  measures—revenues, employee levels, and prime contract and  subcontract awards—do not reflect the primary focus of the program,  which the officials noted is to assist small businesses in becoming part of  the defense supply chain. The officials noted that DOD is working to  develop measures that provide a better indication of program  effectiveness. For example, the officials described a potential measure  that would assess how the assistance mentors provide to protégés under  agreements helps resolve operational challenges. However, DOD had yet  to establish any such measures as of January 2017. Moreover, the  officials estimated that it would take about 2 years for DOD to develop  and establish baselines for these measures. Until DOD establishes  performance goals and related performance measures consistent with the  program’s stated purpose, it may be more difficult for DOD to analyze the  effectiveness of the Pilot Mentor-Protégé Program and to identify and  prioritize potential improvements. As a result, Congress may not have the  information it needs to make informed decisions on whether to  reauthorize the pilot program, terminate it, or make it permanent.

section_title: SBA and DOD Programs Have Key Differences, but the Agencies Plan to Consider Harmonization in Future Discussions
paragraphs: DOD’s Pilot Mentor-Protégé Program and SBA’s All Small Mentor- Protégé Program have a number of key differences, including their  eligibility requirements and participation incentives. These differences are  largely based on statutory provisions and program rules that may—but  will not necessarily—affect future efforts to harmonize the programs.  Implementation of SBA’s All Small Mentor-Protégé Program is ongoing,  but DOD and SBA officials said they plan to consider harmonizing their  programs after SBA’s program is fully implemented in July 2017.
subsections: section_title: Program Differences Include Eligibility Requirements for Protégés and Participation Incentives
paragraphs: Based on our review of statutory provisions and program rules, we found  that DOD’s Pilot Mentor-Protégé Program and SBA’s All Small Mentor- Protégé Program differ in a number of ways. For example, DOD and SBA  apply the SBA size standards differently when determining the eligibility of  protégés under their respective programs. SBA’s small business size  standards establish the largest that a firm can be and still qualify as a  small business for federal government programs.  DOD’s Pilot Mentor- Protégé Program limits the size of eligible protégés to less than one-half  of SBA’s small business size standard. In contrast, a firm can qualify as a  protégé under SBA’s All Small Mentor-Protégé Program if it does not  exceed the SBA small business size standard for its primary NAICS code.

Table 2 illustrates how DOD and SBA apply the SBA size standard  differently under their respective programs for selected NAICS codes. For  example, a business that manufactures printed circuit assemblies would  be eligible as a protégé under DOD’s Pilot Mentor-Protégé Program as  long as it employed fewer than 375 employees; under SBA’s All Small  Mentor-Protégé Program, the same business would be eligible as long as  it employed no more than 750 employees.

Further, DOD’s Pilot Mentor-Protégé Program permits mentors to enter  into agreements with more than one protégé. Under SBA’s All Small  Mentor-Protégé Program, mentors generally are not permitted to have  more than one protégé at a time. However, SBA may authorize a firm to  mentor more than one protégé at a time if it can demonstrate that the  additional mentor-protégé relationship will not adversely affect the  development of either protégé firm (e.g., the second firm may not be a  competitor of the first firm), but under no circumstances does SBA permit  a mentor to have more than three protégés at one time.

In addition, DOD’s Pilot Mentor-Protégé Program receives appropriated  funds, while SBA’s All Small Mentor-Protégé Program does not; as  previously noted, DOD’s Pilot Mentor-Protégé Program received $28.3  million for fiscal year 2016. DOD uses its appropriated funding to provide  direct reimbursement to mentors for providing developmental assistance  to protégés. Incentive mechanisms are another key difference between  DOD’s Pilot Mentor-Protégé Program and SBA’s All Small Mentor- Protégé Program. Under DOD’s program, mentors may receive direct  reimbursement or credit toward subcontracting goals for providing  developmental assistance to protégés. Under statute, generally a mentor  in DOD’s Pilot Mentor-Protégé Program may not receive reimbursements  for costs of assistance that exceed $1 million in a fiscal year.  Under  SBA’s All Small Mentor-Protégé Program, a mentor and protégé may  agree to seek any government prime contract or subcontract jointly as a  small business, provided the protégé qualifies as “small” for the  procurement, and receive exclusion from the affiliation rules for the  duration of the mentor-protégé agreement after both meeting certain  regulatory requirements and obtaining SBA approval. In addition, SBA’s  program permits federal agencies to provide incentives during the  contract evaluation process to a mentor that will provide significant  subcontracting work to its SBA-approved protégé firm, where appropriate.

Although the statutory provisions and program rules that determine  mentor incentives differ, DOD’s Pilot Mentor-Protégé Program and SBA’s  All Small Mentor-Protégé Program face similar inherent risk that mentors  may receive benefits but may not necessarily provide developmental  assistance to protégés as agreed to in their mentor-protégé agreements.  Officials from both agencies said they take certain actions intended to  mitigate this risk. For example, DOD requires an annual review of each  mentor-protégé agreement to verify the performance information provided  in the annual reports. Similarly, SBA reviews the protégé’s report on the  mentor-protégé relationship and may decide not to continue the  agreement if it finds that the mentor has not provided the agreed upon  assistance or the assistance has not resulted in developmental gains for  the protégé.

Appendix IV provides more details on the characteristics of DOD’s Pilot  Mentor-Protégé Program and SBA’s All Small Mentor-Protégé Program.

section_title: DOD and SBA Plan to Consider Opportunities to Harmonize Their Programs Once Implementation of SBA’s New Program Is Complete
paragraphs: As previously noted, SBA’s All Small Mentor-Protégé Program is  designed to apply to nearly all federal small business contracts, including  those that small businesses currently participate in as part of other federal  mentor-protégé programs. Under the new program, federal agencies will  be required to be granted approval from the SBA Administrator in order to  continue their existing mentor-protégé programs or to start new ones. By  statute, federal departments and agencies with mentor-protégé programs  in effect before January 2, 2013, are permitted to continue them until late  July 2017 without being granted the SBA approval. However, after late  summer 2017, approval must be granted to continue the program.  Although it is too soon to tell whether the other federal agencies that  currently administer mentor-protégé programs will seek to continue their  own programs, SBA’s experience in implementing the All Small Mentor- Protégé Program may provide useful insights for harmonizing other  federal mentor-protégé programs—that is, identifying and assessing  differences and eliminating or reducing inconsistencies, where  appropriate. For example, implementing SBA’s program may involve  actions to potentially harmonize characteristics of mentor-protégé  programs that serve different socioeconomic categories of small  businesses and multiple agencies with various procurement needs.

Although the statute creating SBA’s program exempted DOD from the  program’s requirements, the legislation implementing DOD’s program and  the final rule implementing SBA’s program state that both programs are  designed to enhance the capabilities of protégé firms to perform under  federal government and commercial contracts. This general similarity in  program purpose could facilitate harmonization between the programs.  DOD and SBA officials said that opportunities to harmonize the programs  may exist; however, they said it is too soon to explore harmonization  because SBA is in the early stages of implementing the All Small Mentor- Protégé Program. Officials said that they plan to consider opportunities to  harmonize their programs once implementation of SBA’s new program is  complete later in 2017.

section_title: Conclusions
paragraphs: DOD’s Pilot Mentor-Protégé Program is intended to both (1) enhance the  capabilities of eligible disadvantaged small business concerns to perform  as subcontractors and suppliers under DOD contracts and other contracts  and subcontracts and (2) increase the participation of such business  concerns as subcontractors and suppliers under DOD contracts, other  federal government contracts, and commercial contracts. DOD  procedures require that mentor-protégé agreements contain certain  elements to help provide assurance that the agreements support program  purposes and that the participants meet program requirements, among  other things. However, we found that program participants did not  consistently include all of the required elements in these agreements and  that DOD’s OSBP does not review the procedures that DOD components  use to approve the agreements, as suggested by federal internal control  standards. As a result, some agreements have been approved even  though they were missing required elements, and DOD cannot ensure  that these program requirements are serving their intended purposes.

In addition, DOD’s performance measures for the decades-old program  lack performance goals and additional measures needed to effectively  assess its Pilot Mentor-Protégé Program. Although DOD officials said  they are considering changes to their performance measures, they had  yet to make such changes as of January 2017. Until DOD establishes  performance measures and related measurable goals consistent with the  program’s stated purpose, it may be more difficult for DOD to fully assess  the effectiveness of the Pilot Mentor-Protégé Program and to identify and  prioritize potential improvements. In addition, Congress may not have the  information it needs to make informed decisions on whether to  reauthorize the pilot program, terminate it, or make it permanent.

section_title: Recommendations for Executive Action
paragraphs: To provide reasonable assurance that DOD’s Pilot Mentor-Protégé  Program meets its mission, we recommend that the Director, DOD’s  Office of Small Business Programs, take the following two actions:  1.  Conduct periodic reviews of the processes DOD components follow to  approve agreements and take oversight actions, as appropriate.  2.  Complete actions to develop performance goals and related  measures that are consistent with the program’s stated purpose.

section_title: Agency Comments
paragraphs: We provided a draft of this report to DOD and SBA for comment. In its  written comments, which are summarized and reprinted in appendix V,  DOD concurred with both recommendations. SBA provided technical  comments, which we incorporated as appropriate.

DOD concurred with our recommendation to conduct periodic reviews of  the processes DOD components follow to approve agreements and take  oversight actions, as appropriate. DOD said it has drafted a standard  operating procedure (SOP) that addresses GAO’s recommendations. In  addition, DOD said that the SOP requires DOD component program  managers to review mentor-protégé proposal packages prior to  submitting them for review and concurrence. DOD also said the draft  SOP includes a new checklist that identifies required elements, such as  the North American Industry Classification System code, mentor approval  letter, and fully executed copy of the mentor-protégé agreement.

DOD also concurred with our recommendation to complete actions to  develop performance goals and related measures that are consistent with  the program’s stated purpose. DOD stated that it is currently evaluating  ways to develop measures that provide a better indication of program  effectiveness for active mentor-protégé agreements and the 2 years  following program participation. DOD noted that establishing baseline  performance goals for all protégés to meet would not be advantageous  due to the differences of each agreement and the small business needs  of each respective DOD component; however, DOD stated that  establishing a set of goals by tier group, for example, may better  determine program performance as it would even the playing fields  between large and small business concerns. Finally, DOD stated that it  collects information from mentors on how they have enhanced the  capabilities of protégés and plans to begin including these data in the  annual report to Congress once performance measures and goals have  been established.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of Defense, DOD’s Director, Office of Small  Business Programs, and the SBA Administrator. In addition, the report is  available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact  me at (202) 512-8678 or shearw@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix VI.

section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report examines (1) the Department of Defense’s (DOD) procedures  for approving mentor-protégé agreements; (2) DOD’s performance  measures for the program; and (3) differences between DOD’s Pilot  Mentor-Protégé Program and the Small Business Administration’s (SBA)  All Small Mentor-Protégé Program and the agencies’ efforts to harmonize  the two programs.

To address the first objective, we reviewed DOD’s current regulations and  policies and procedures for approving mentor-protégé agreements,  including changes mandated by the National Defense Authorization Act  (NDAA) for Fiscal Year 2016. In addition, we reviewed guidance  developed by three military services (the Departments of the Air Force,  Army, and Navy) that administer and have the authority to approve  mentor-protégé agreements. We also reviewed a randomly selected  probability sample of 44 of the 78 active DOD mentor-protégé  agreements in place as of June 2016 to assess how well DOD’s  procedures provide reasonable assurance that mentor-protégé  agreements meet program requirements.

With this probability sample, each member of the study population had a  nonzero probability of being included, and that probability could be  computed for any member. Because we followed a probability procedure  based on random selections, our sample is only one of a large number of  samples that we might have drawn. Since each sample could have  provided different estimates, we express our confidence in the precision  of our particular sample’s results as a 95 percent confidence interval. This  is the interval that would contain the actual population value for 95  percent of the samples we could have drawn. All percentage estimates in  this report are presented along with their margins of error at the 95  percent confidence level.

We interviewed DOD officials from the Office of Small Business Programs  (OSBP) to discuss the procedures for approving mentor-protégé  agreements and the controls in place to ensure that DOD components  followed the procedures. We assessed DOD’s practices against  Standards for Internal Control in the Federal Government. We also  reviewed the NDAA for Fiscal Year 2016 to identify changes that the act  made to the DOD Pilot Mentor-Protégé Program, such as amending the  size requirement so that a protégé is less than half of the SBA size  standard for its primary North American Industry Classification System  code.

To address the second objective, we reviewed the measures and other  information DOD reports on the performance of its Pilot Mentor-Protégé  Program, including DOD’s current measures, which provide information  related to the participation of protégés as subcontractors and suppliers  under DOD and other contracts, and whether these measures address  how the capabilities of protégé firms have been enhanced. This review  included the annual reports prepared by the Defense Contract  Management Agency (DCMA) for fiscal years 2011 through 2015 that  provide aggregate data on protégé firms’ annual revenues, number of  employees, and awards of prime contracts and subcontracts during their  tenure in the program and for 2 years after they leave the program. We  also reviewed the most recent semiannual report provided to DOD by the  mentor and protégé for the randomly selected probability sample of  mentor-protégé agreements we reviewed to examine the types of  performance information included in these reports. We also interviewed  DOD officials regarding their use of performance information. We  compared DOD’s practices to those GAO previously identified as being  associated with agencies that were successful in measuring their  performance and federal internal control standards.

To address the third objective, we reviewed the DOD Pilot Mentor- Protégé Program’s implementing statute and subsequent legislative  changes to the program, including those made by the NDAA for Fiscal  Year 2016. We also reviewed SBA’s final rule which implemented  provisions of the Small Business Jobs Act of 2010 and the National  Defense Authorization Act for Fiscal Year 2013, which established the All  Small Mentor-Protégé Program and the policies and procedures for  SBA’s 8(a) Business Development Program.  We identified selected  characteristics of the programs, including the purpose, funding, number of  agreements, eligibility requirements of mentors and protégés, authority to  approve agreements, types of assistance mentors may provide to  protégés, reporting requirements for mentors and protégés, and the  mechanism used to encourage mentor participation in each program. We  also identified differences between the programs for some of these  characteristics and discussed them with DOD and SBA officials, including  how differences between the programs could affect possible efforts to  harmonize the programs.

In addition, we analyzed a randomly selected probability sample of 44 of  the 78 active DOD mentor-protégé agreements in place as of June 2016  to identify characteristics of protégé firms. These characteristics include  the number of years protégé firms have been in business, the number of  employees of the protégé firms, annual revenues of the protégé firms,  and their previous contracting experience. We also identified the types of  developmental assistance mentor firms agreed to provide to protégé firms  based on information reported for the randomly selected probability  sample of agreements. These results can be found in appendix II.

We conducted this performance audit from January 2016 to April 2017, in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

section_title: Appendix II: Selected Characteristics of DOD Protégé Firms
paragraphs: Under the Department of Defense’s (DOD) Pilot Mentor-Protégé  Program, mentor firms agree to provide eligible small business concerns  (protégé firms) with technical, managerial, and other business  development assistance under approved mentor-protégé agreements.  Based on our review of a randomly selected probability sample of 44 of  the 78 active DOD mentor-protégé agreements in place as of June 2016,  we analyzed selected characteristics of protégé firms that enter the  program and the types of developmental assistance that mentor firms  agree to provide protégé firms.

As shown in figure 4, we estimate that over half (57 percent) of protégé  firms have been in business for 10 years or less, and an estimated 21  percent have been in business 5 years or less. We also estimate that  more than 33 percent of protégé firms have been in business for more  than 15 years.

We also analyzed DOD mentor-protégé agreement documentation to  determine the number of employees in protégé firms. We found that an  estimated 77 percent of the protégé firms had 50 or fewer employees  (see fig. 5).

In addition, we examined DOD mentor-protégé agreement documentation  to determine protégé firms’ annual revenue. We found that an estimated  57 percent of protégé firms had annual revenue of $5 million or less.  More than three-quarters of protégé firms had annual revenue of $10  million or less, and an estimated 17 percent of protégé firms had annual  revenues above $10 million (see fig. 6).

We also analyzed DOD mentor-protégé agreement documentation to  determine the protégé firms’ previous contracting experience. We found  that an estimated 39 percent of protégé firms indicated having obtained a  DOD prime contract in the 2 fiscal years before the proposed mentor- protégé agreement was submitted. We also found that an estimated 74  percent of protégé firms indicated having obtained DOD subcontracts in  the 2 fiscal years before the proposed mentor-protégé agreement was  submitted and that an estimated 26 percent indicated having received a  federal agency (other than DOD) subcontract award from the mentor firm  during that time. In addition, an estimated 53 percent of protégé firms  indicated having received subcontract awards from their mentor firm in  the 2 fiscal years before the proposed agreement was submitted. We also  found that an estimated 13 percent of protégé firms indicated they did not  have previous contracting experience (see fig. 7).

Based on our analysis of DOD mentor-protégé agreement  documentation, we found that mentor firms anticipated providing some  types of assistance more frequently than others. For example, we found  that all mentor firms anticipated providing general business assistance to  their protégé firms.  In addition, we found that an estimated 93 percent of  mentor firms anticipated awarding subcontracts to their protégé firms, and  an estimated 80 percent anticipated obtaining assistance for their protégé  firms through historically black colleges and universities, minority  institutions of higher education, Procurement Technical Assistance  Centers, or Small Business Development Centers. In contrast, an  estimated 2 percent of mentor firms anticipated extending loans to  protégé firms. We also found that an estimated 20 percent anticipated  providing progress payments and an estimated 18 percent anticipated  providing advance payments to their protégé firms (see fig. 8).

section_title: Appendix III: Selected Changes to DOD’s Pilot Mentor-Protégé Program in the NDAA for Fiscal Year 2016
paragraphs: The National Defense Authorization Act (NDAA) for Fiscal Year 2016  made a number of changes to the Department of Defense’s (DOD) Pilot  Mentor-Protégé Program.  In addition to the changes previously  discussed, Section 861 of the NDAA for Fiscal Year 2016 also made the  following changes and additions, among others:

Extended the DOD mentor-protégé program for 3 more years through  fiscal year 2018 for the formation of new agreements and for the  reimbursement of incurred costs and credit toward attaining  subcontracting goals, under existing agreements incurred through  fiscal year 2021.

Imposed a limit on when protégé firms can enter into mentor-protégé  agreements under the program to only during the 5-year period  beginning on the date the protégé firm enters into its first mentor- protégé agreement.

Changed program eligibility requirements to include that a protégé  firm may, among other things, be a small business concern that is  either a nontraditional defense contractor or currently provides goods  or services in the private sector that are critical to enhancing the  capabilities of the defense supplier base and fulfilling key DOD needs.

Required mentor firms to report at least once a fiscal year any new  awards of subcontracts on a competitive or noncompetitive basis to  the protégé firm under DOD contracts or other contracts, including the  value of such subcontracts.

Required mentor firms to report at least once a fiscal year any  assistance obtained on behalf of the protégé from Small Business  Development Centers, certain entities providing procurement  technical assistance, or historically black colleges or universities or  minority institutions of higher education.

Required the Office of Small Business Programs (OSBP) to review  reports submitted by mentor firms on specific statutory topics and if a  mentor-protégé agreement is not furthering the purpose of the  program, decide not to approve any continuation of the agreement.

Removed “business development” assistance from the permissible  mentor firm personnel provided assistance and removed cash in  exchange for an ownership interest in the protégé firm from the types  of assistance a mentor firm may provide to a protégé firm.

Revised the definitions of the term ‘‘disadvantaged small business  concern” limiting it to firms that have less than half the size standard  corresponding to their primary North American Industry Classification  System code.

Changed affiliation rules by explicitly stating that the presence of  certain family relationships between the owners and managers in the  mentor and protégé firms constitutes affiliation and thereby is not  allowed.

section_title: Appendix IV: Selected Characteristics of DOD’s Pilot Mentor-Protégé and SBA’s Small Business Mentor-Protégé Programs
subsections: section_title: Department of Defense (DOD) Pilot Mentor-Protégé Program
paragraphs: Enhance the capabilities of  eligible small business concerns  to perform as subcontractors  and suppliers under DOD  contracts and other contracts  and subcontracts, and increase  the participation of such  business concerns as  subcontractors and suppliers  under DOD contracts, other  federal government contracts,  and commercial contracts.

Small Business  Administration (SBA) 8(a)  Business Development (BD)  Mentor-Protégé Program  Enhance the capabilities of the  protégé (an 8(a) BD Program  participant), assist the protégé  with meeting the goals  established in its SBA-approved  business plan, and improve the  protégé’s ability to successfully  compete for contracts.

SBA All Small Mentor-Protégé  Program Enhance the capabilities of  protégé firms by requiring  approved mentors to provide  business development  assistance to protégé firms and  to improve the protégé firms’  ability to successfully compete  for federal contracts.

28.3 million (fiscal year 2016) 78 (as of June 2016)

465 (as of November 2016)

45 (as of November 2016)

Protégé must be a small  disadvantaged business, a  business entity owned and  controlled by an Indian tribe or a  Native Hawaiian Organization, a  women-owned business, a  Historically Underutilized  Business Zone (HUBZone)  small business, service-disabled  veteran-owned small business,  a small business that statutorily  qualifies as a nontraditional  defense contractor, or an  eligible entity employing the  severely disabled.  Protégé must qualify as small  by being less than half the size  standard corresponding to its  primary North American  Industry Classification System  (NAICS) code.  Mentor may rely on written  representation that the protégé  qualifies as small when entering  into a mentor-protégé  agreement but must confirm a  protégé’s status as a HUBZone  small business concern.

Protégé must (1) qualify as  small for the size standard  corresponding to its primary  North American Industry  Classification System (NAICS)  code or identify that it is seeking  business development  assistance with respect to a  secondary NAICS code and  qualify as small for the size  standard corresponding to that  NAICS code; and (2)  demonstrate how the business  development assistance to be  received through its proposed  mentor-protégé relationship  would advance the goals and  objectives set forth in its  business plan.

Protégé must qualify as small  for the size standard  corresponding to its primary  North American Industry  Classification System (NAICS)  code or identify that it is seeking  business development  assistance with respect to a  secondary NAICS code and  qualify as small for the size  standard corresponding to that  NAICS code.  A firm may self-certify that it  qualifies as small for its primary  or identified secondary NAICS  code. When the firm is not small  for the primary NAICS code and  seeks to qualify for the  secondary NAICS code, the firm  must demonstrate how the  mentor-protégé relationship is a  logical business progression for  the firm and will further develop  or expand current capabilities.  SBA will not approve a  relationship in a secondary  NAICS code in which the firm  has no prior experience.

section_title: Department of Defense (DOD) Pilot Mentor-Protégé Program
paragraphs: A mentor, among other  requirements, must be: (1) not  affiliated with the protégé firm  prior to the approval of the  agreement; (2) demonstrate that  it is qualified to provide  assistance that will contribute to  the purpose of the program and  is in good financial health and  character and does not appear  on a federal list of debarred or  suspended contractors; (3) an  entity other than small business  that is a DOD prime contractor  with an active subcontracting  plan (unless a waiver to the  small business exception has  been obtained from the Director,  Small Business Programs,  Office of the Undersecretary of  Defense for Acquisition,  Technology, and Logistics); or  (4) a graduated 8(a) firm that  provides documentation of its  ability to serve as a mentor; and  (5) approved to participate as a  mentor.

Small Business  Administration (SBA) 8(a)  Business Development (BD)  Mentor-Protégé Program  Mentor must (1) be capable of  carrying out its responsibilities  to assist the protégé firm under  the proposed mentor-protégé  agreement; (2) possesses good  character; (3) not appear on the  federal list of debarred or  suspended contractors; and (4)  be able to impart value to a  protégé due to lessons learned  and practical experience gained  because of the 8(a) BD  program, or through its  knowledge of general business  operations and government  contracting.

Mentor must demonstrate that it  (1) is capable of carrying out its  responsibilities to assist the  protégé under the proposed  mentor-protégé agreement; (2)  possesses good character; (3)  does not appear on the federal  list of debarred or suspended  contractors; and (4) can impart  value to a protégé firm due to  lessons learned and practical  experience gained or through its  knowledge of general business  operations and government  contracting.

Director, Small Business  Programs, for each military  department or defense agency  generally approves mentor- protégé agreements.

The Associate Administrator for  Business Development or  his/her designee must approve  the mentor-protégé agreement.  The agreement will not be  approved if SBA determines  that the assistance to be  provided is not sufficient to  promote any real developmental  gains to the protégé, or if SBA  determines that the agreement  is merely a vehicle to enable the  mentor to receive 8(a)  contracts.

The Associate Administrator for  Business Development or  his/her designee must approve  the mentor-protégé agreement.  The agreement will not be  approved if SBA determines  that the assistance to be  provided is not sufficient to  promote any real developmental  gains to the protégé, or if SBA  determines that the agreement  is merely a vehicle to enable the  mentor to receive small  business contracts.

section_title: Department of Defense (DOD) Pilot Mentor-Protégé Program
paragraphs: A mentor may enter into  agreements with more than one  protégé.

Small Business  Administration (SBA) 8(a)  Business Development (BD)  Mentor-Protégé Program  Generally, a mentor will have no  more than one protégé at a  time. However, SBA may  authorize a concern to mentor  more than one protégé at a time  where it can demonstrate that  the additional mentor-protégé  relationship will not adversely  affect the development of either  protégé firm (e.g., the second  firm may not be a competitor of  the first firm). Under no  circumstances will a mentor be  permitted to have more than  three protégés at one time  between the SBA 8(a) Business  Development Mentor-Protégé  Program and the SBA All Small  Mentor-Protégé Program.  SBA may authorize a participant  to be both a mentor and a  protégé at the same time where  the participant can demonstrate  that the second relationship will  not compete or otherwise  conflict with the first mentor- protégé relationship.

SBA All Small Mentor-Protégé  Program Generally, a mentor will have no  more than one protégé at a  time. However, SBA may  authorize a concern to mentor  more than one protégé at a time  where it can demonstrate that  the additional mentor-protégé  relationship will not adversely  affect the development of either  protégé firm (e.g., the second  firm may not be a competitor of  the first firm). Under no  circumstances will a mentor be  permitted to have more than  three protégés at one time  between the SBA 8(a) Business  Development Mentor Protégé  Program and the SBA All Small  Mentor-Protégé Program.  SBA may authorize a participant  to be both a mentor and a  protégé at the same time where  the participant can demonstrate  that the second relationship will  not compete or otherwise  conflict with the first mentor- protégé relationship.

Protégé may only have one  active DOD mentor-protégé  agreement.

Protégé firm may generally  have only one mentor at a time.  SBA may approve a second  mentor for a particular protégé  firm where additional  relationship will not  compete/conflict with the  existing mentor-protégé  relationship and either (1) the  added relationship pertains to  an unrelated secondary NAICS  code or (2) the protégé is  seeking expertise the existing  mentor does not possess.

Protégé firm may generally  have only one mentor at a time.  SBA may approve a second  mentor for a particular protégé  firm where additional  relationship will not  compete/conflict with the  existing mentor-protégé  relationship and either (1) the  added relationship pertains to  an unrelated secondary NAICS  code or (2) the protégé is  seeking expertise the existing  mentor does not possess.

section_title: Department of Defense (DOD) Pilot Mentor-Protégé Program
paragraphs: Mentor may, among other  actions, provide management  and/or technical assistance,  award subcontracts, make  loans, make advance and/or  progress payments under  subcontracts, and obtain  assistance from Small Business  Developmental Centers,  Procurement Technical  Assistance Centers, historically  black colleges and universities,  or minority institutions of higher  education.

Small Business  Administration (SBA) 8(a)  Business Development (BD)  Mentor-Protégé Program  Mentor may provide  management and/or technical  assistance, award subcontracts,  trade education, make loans  and/or equity investments, and  cooperate on joint venture  projects.

Mentor may provide  management and/or technical  assistance, award subcontracts,  trade education, make loans  and/or equity investments, and  cooperate on joint venture  projects.

Mentors must report on the  progress made under active  mentor-protégé agreements  semiannually, including (1)  dollars obligated (for  reimbursable agreements); (2)  expenditures; (3) dollars  credited toward applicable  subcontracting goals, if any; (4)  number and dollar value of  subcontracts awarded to the  protégé firm; (5) description of  developmental assistance  provided, including milestones  achieved; (6) impact of  agreement in terms of  capabilities enhanced,  certifications received, and/or  technology enhanced.  Protégés must report data  annually on the progress made  in employment, revenues, and  participation in DOD contracts.  The protégé report may be  provided as part of the mentor  report.

In its annual business plan  update, the protégé must report  to SBA for the protégé’s  preceding program year (1) all  technical and/or management  assistance provided by the  mentor to the protégé; (2) all  loans to and/or equity  investments made by the  mentor in the protégé; (3) all  subcontracts awarded to the  protégé by the mentor, and the  value of each subcontract; (4)  all federal contracts awarded to  the mentor/protégé relationship  as a joint venture (designating  each as an 8(a), small business  set aside, or unrestricted  procurement), the value of each  contract, and the percentage of  the contract performed and the  percentage of revenue accruing  to each party to the joint  venture; and (5) a narrative  describing the success such  assistance has had in  addressing the developmental  needs of the protégé and  addressing any problems  encountered.

Protégés must report to SBA  annually, within 30 day of the  agreement approval  anniversary, (1) all technical  and/or management assistance  provided by the mentor to  protégé; (2) all loans to and/or  equity investments made by the  mentor in the protégé; (3) all  subcontracts awarded to the  protégé by the mentor and all  subcontracts awarded to the  mentor by the protégé, and the  value of each subcontract; (4)  all federal contracts awarded to  the mentor-protégé relationship  as a joint venture, the value of  each contract, and the  percentage of the contract  performed and the percentage  of revenue accruing to each  party to the joint venture; and  (5) a narrative describing the  success such assistance has  had in addressing the  developmental needs of the  protégé and addressing any  problems encountered.  Additionally, the protégé must  report the mentoring services it  receives by category and hours.

section_title: Department of Defense (DOD) Pilot Mentor-Protégé Program
paragraphs: DOD may provide mentors with  either cost reimbursement or  credit against applicable  subcontracting goals  established under contracts with  DOD or other federal agencies.

Small Business  Administration (SBA) 8(a)  Business Development (BD)  Mentor-Protégé Program  A mentor and protégé may  participate in a joint venture as  a small business for any  government prime contract or  subcontract, including  procurements with a dollar  value less than half the size  standard corresponding to the  assigned NAICS code and 8(a)  sole source contracts, provided  the protégé qualifies as small  for the procurement and  additional provisions for 8(a)  sole source contracts.

A mentor and protégé may  participate in a joint venture as  a small business for any  government prime contract or  subcontract, provided the  protégé qualifies as small for  the procurement. Such a joint  venture may seek any type of  small business contract (i.e.,  small business set-aside, 8(a),  HUBZone, service-disabled  veteran-owned small business,  or women-owned small  business) for which the protégé  firm qualifies and within certain  requirements, receives  exclusion from the affiliation  rules for the duration of the joint  venture agreement.  Procuring activities may provide  incentives in the contract  evaluation process to a mentor  that will provide significant  subcontracting work to its SBA- approved protégé firm, where  appropriate.

section_title: Appendix V: Comments from the Department of Defense

section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Marshall Hamlett (Assistant  Director), Abiud Amaro Diaz, Isidro Gomez, Farrah Graham, Marc Molino,  Jessica Sandler, and Jennifer Schwartz made key contributions to this  report.


Question-summary annotations: questions: question: How does DOD approve Pilot Mentor-Protégé Program agreements?
answer: The Department of Defense (DOD) relies on military services and agencies (DOD components) to approve the agreements that establish relationships between participants in its Pilot Mentor-Protégé Program.

question: What is the goal of the program?
answer: This program provides incentives for major defense contractors (mentors) to provide assistance to small disadvantaged firms (protégés) in an effort to enhance their capability to compete for federal and commercial contracts.

question: To what extent can DOD assure that agreements include all the required elements?
answer: However, DOD does not have reasonable assurance that approved agreements include all elements required by the program's regulations and policies.
child_questions: question: What elements are required?
answer: These elements include, among others, the protégé's industry code and the signature and date of the mentor and protégé.

question: What are the purposes of these elements?
answer: These elements serve a variety of purposes—for example, the industry code is used to determine the protégé's eligibility to participate in the program, and the signature and date of mentor and protégé are required in order for the agreement to be legally bindingto meet program requirements.

question: To what extent are agreements missing required elements?
answer: Based on a review of a randomly selected probability sample of 44 of the 78 active mentor-protégé agreements in place as of June 2016, GAO estimates that 27 percent of agreements were missing required elements. For example, GAO estimates that 25 percent of agreements were not signed by both the mentor and protégé.

question: What do federal internal control standards state regarding control activities?
answer: Federal internal control standards state that management should implement control activities through policies and practices, including periodically reviewing control activities for continued effectiveness.

question: What does DOD’s OSBP manage?
answer: DOD's Office of Small Business Programs (OSBP) manages the program and oversees program policies and procedures.
child_questions: question: What does OSBP not review?
answer: However, OSBP does not review the DOD components' processes for approving mentor-protégé agreements and therefore has not taken appropriate oversight actions to provide reasonable assurance that agreements meet all requirements.
child_questions: question: As a result, what can OSBP not ensure?
answer: As a result, the components have approved agreements that do not include required elements, and OSBP cannot ensure that the requirements are serving their various purposes.

questions: question: What information do DOD's annual Pilot Mentor-Protégé Program reports include?
answer: DOD's fiscal year 2011 through 2015 annual reports on its Pilot Mentor-Protégé Program include performance measures for several areas, but DOD lacks performance goals and other measures needed to effectively assess the program.
child_questions: question: What have these reports shown?
answer: Some of these measures show that during this period, protégés' revenue, number of employees, and DOD prime and subcontract awards increased while protégés participated in the program, but revenues and employment levels decreased in the 2 years after their participation ended.

question: What goals does DOD have for these measures?
answer: GAO found that DOD has not established any measurable goals for these measures.
child_questions: question: What has GAO previously stated regarding measurable goals?
answer: GAO has previously identified performance measurement as a best practice that allows organizations to track progress and gives managers information to plan needed improvements.

question: What is lacking in the information DOD collects from mentors?
answer: In addition, DOD collects information from mentors on how they have enhanced the capabilities of protégés, but DOD does not include this information in the program's annual report and has not developed performance measures or goals related to this information.

question: What is the goal of the measures DOD is developing?
answer: DOD officials told GAO they are working to develop measures that better indicate program outcomeseffectiveness, but as of January 2017 they had not established such measures.

question: Why are performance goals essential for DOD?
answer: Without performance goals and related measures, DOD may be limited in its ability to analyze the effectiveness of the program, and Congress may not have information needed to inform future decisions regarding the program.

questions: question: How did the National Defense Authorization Act for FY2016 affect GAO?
answer: The joint explanatory statement to accompany the National Defense Authorization Act for Fiscal Year 2016 includes a provision for GAO to report on DOD's pilot program.

question: What does this report examine?
answer: This report examines, among other things, (1) DOD's procedures for approving mentor-protégé agreements and (2) DOD's performance measures for the program.
child_questions: question: How did GAO source data for this report?
answer: GAO analyzed DOD guidance, reviewed a randomly selected probability sample of active DOD mentor-protégé agreements and estimated their completeness at a 95 percent confidence interval, reviewed DOD's annual program reports for fiscal years 2011 through 2015, and interviewed agency officials.