To meet the legislative requirements regarding independent management reviews, DOD issued guidance and instructions providing for a peer review process for services acquisitions. DOD’s guidance generally addresses requirements prescribed in the Act to develop a process to evaluate the specified contracting issues, but according to DOD officials, DOD has not yet determined how the department plans to disseminate lessons learned or track recommendations that result from the newly instituted reviews. DOD officials expect to further refine their processes, including developing a more formal means for disseminating lessons learned and tracking recommendations as DOD assesses its initial experiences with peer reviews. Through the first year of implementation, DPAP, which is responsible for conducting reviews of acquisitions over $1 billion, had conducted 29 peer reviews on 18 services acquisitions. Similarly, the military departments, which are responsible for conducting reviews of their acquisitions under $1 billion, issued guidance that provides for peer reviews at various levels within the departments based on dollar values. The military departments could not, however, determine the exact number of peer reviews conducted because of the absence of comprehensive reporting processes. Further, as peer review processes evolve, the military departments are considering ways to disseminate lessons learned and track recommendations. DPAP issued a memorandum in September 2008 establishing a peer review process to fulfill the requirement for an independent management review of contracts for services. The requirement for a peer review process was subsequently incorporated into DOD Instruction 5000.02, Operation of the Defense Acquisition System, in December 2008. The guidance states that these reviews are intended to ensure consistent and appropriate implementation of policy and regulations, improve the quality of contracting processes, and facilitate sharing best practices and lessons learned. According to DOD officials, peer reviews by design are a means of improving individual acquisitions and not necessarily a tool for strategically managing DOD’s services portfolio. Under DPAP’s guidance, peer reviews supplement its existing process to review and approve services acquisitions. Pursuant to congressional direction, DOD had previously established a management review process that was intended to ensure that DOD services acquisitions are based on clear, performance-based requirements with measurable outcomes and that acquisitions are planned and administered to achieve intended results. In these management reviews, DPAP assesses and approves the acquisition strategies submitted by the military departments or defense agencies for obtaining contractor-provided services estimated to be valued at $1 billion or more. Once the acquisition strategies are approved, DOD contracting offices may continue the acquisition process, including soliciting bids for proposed work and subsequently awarding contracts. DOD may award different contract types to acquire products and services, or issue task orders under existing contracts. In November 2009, we reported that the number of contracts and task orders issued after the acquisition strategies were approved was significant. For example, we reported that nearly 1,900 task orders were issued under the seven professional and management support services acquisitions we reviewed. DOD generally conducts peer reviews at three key points in the acquisition process prior to contract award—prior to issuance of the solicitation (phase 1), prior to request for final proposal revisions (phase 2), and prior to contract award (phase 3)—and is to conduct periodic post-award reviews (phase 4) (see fig. 1). In February 2009, DOD issued guidance that clarified the relationship between the management reviews and the peer reviews. For example, the guidance identifies specific issues to assess and the criteria for the reviewers to use during the management reviews or pre-award peer reviews. According to the guidance, some contracting issues identified in the Act, such as contract type and competition, are to be assessed during the management reviews. Conversely, other contracting issues identified in the Act, including requirements definition and the extent of the agency’s reliance on contractors to perform functions closely associated with inherently governmental functions, are to be assessed during pre-award peer reviews. The pre-award peer reviews also are to evaluate several elements of the source selection process that are not specified in the Act, such as the clarity and consistency of the documentation. Further, the guidance established review criteria for post-award reviews that address each of the contracting issues identified in the Act. For example, during post-award reviews, reviewers are to assess the extent to which the contracting office was able to achieve competition for orders and whether it was using appropriate contract types, well-defined requirements, and appropriate cost/pricing methods. According to DOD officials, in conducting these reviews, DPAP convenes a peer review team consisting of three to five members. Officials said that the teams are generally chaired by a deputy director within DPAP and include participation from senior contracting officials from the military departments and defense agencies as well as legal advisors from the Office of the Secretary of Defense’s General Counsel. The teams review acquisition documents prior to an on-site review and hold discussions with contracting officers over multiple days. Upon completion of the on-site review, peer review teams develop summary memoranda that include observations and recommendations. The February 2009 guidance indicated that DPAP is to review services acquisitions with an expected value of over $1 billion. In addition, DPAP may review acquisitions under that threshold that it has designated as special interest because of the nature or sensitivity of the services to be acquired. According to DOD officials, DPAP does not have a capability to independently identify acquisitions that will require its review, but rather relies on the military departments and defense agencies to notify DPAP of acquisitions that will exceed the threshold. DPAP officials noted that some reviews were not conducted because the military departments did not notify DPAP that a peer review was necessary. DPAP officials stated that they are currently focusing on the pre-award peer reviews and are phasing in post-award peer reviews. As of September 30, 2009, DPAP had conducted 29 peer reviews for 18 services acquisitions. Because the peer review process was only implemented in September 2008, no single acquisition has been subject to all phases of the peer review process and no acquisition has been peer reviewed in both the pre- and post-award phases. While most of the reviews have focused on proposed acquisitions for which the initial contract had not yet been awarded, DPAP has also conducted two phase 3 peer reviews for proposed task orders valued at over $1 billion that were to be issued under an existing contract that had previously been reviewed. DPAP has not yet determined if it will establish a policy for conducting peer reviews for all individual task orders over this amount in the future. For the 29 peer reviews of services acquisitions that DPAP conducted, figure 2 shows when each review occurred and the corresponding milestone. For example, DPAP conducted a phase 1 peer review prior to the issuance of the solicitation for 12 of the 18 services acquisitions. Our review of the summary memoranda of the pre-award peer reviews that DPAP conducted as of September 30, 2009, found that review teams generally documented the evaluation of the use of contracting mechanisms and, to a lesser extent, the use, management, and oversight of subcontractors. DPAP officials noted that other contracting issues may have been discussed during pre-award site visits and not included in the summary memorandum because the peer review team did not identify any concerns that warranted inclusion. Further, we found that review teams made several related recommendations, as illustrated in the following examples: One pre-award peer review team recommended that the contracting office reconsider the number of contracts that it had proposed be awarded under an acquisition. In this case, the contracting office had proposed limiting the number of contracts to three prior to knowing what proposals and business arrangements would be submitted by industry. The peer review team noted that this may unduly restrict flexibility of the military department. Further, the team was unsure if documentation to support the limitation on contract number would be sufficient to withstand a bid protest from an unsuccessful offeror. Another pre-award peer review recommended that the contracting office increase its use of subcontractors and encourage the prime contractors to establish mentor-protégé relationships with their subcontractors to bring more qualified contractors into an industry. Our review of the summary memoranda for the three post-award peer reviews conducted by DPAP found that consistent with guidance, the review teams evaluated all the contracting issues identified in the Act. All three summary memoranda listed the required contracting issues and then reported the peer review teams’ observations and recommendations for the contracting offices to consider for the acquisition, as illustrated by the following examples: One post-award peer review team recommended that the contracting officer modify the contract to include provisions requiring the contractor to provide information on pass-through charges for all future task orders issued. At the time of the peer review, the contract did not contain a clause requiring the contractor to provide such information, and therefore the government was unable to determine the extent of pass-through charges and whether they were excessive. Another post-award team recommended that the contracting office reduce the use of time-and-materials task orders. In this case, the acquisition strategy envisioned that most of the work would be performed through fixed-priced task orders; however, time-and- materials task orders accounted for 62 percent of the value of orders issued under the contract in the first 2 years of performance. While DPAP’s guidance noted that the recommendations made during peer reviews are advisory in nature, it also states that contracting offices are to document in the contract file the disposition of all pre-award peer review recommendations prior to contract award. The guidance does not address recommendations made during post-award reviews. According to DOD officials, contracting offices generally accept recommendations provided by the peer review teams. DPAP officials said that if the contracting office decides not to accept a peer review team’s recommendation, the contracting officer is expected to document the reason in the contract file and provide a copy to DPAP. In addition to providing recommendations to address potential issues in proposed acquisitions, the peer review teams have also identified some best practices.