Defense Acquisitions: Status of DOD's Implementation of Independent Management Reviews for Services Acquisitions
Published: 2010-01-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense (DOD) is the federal government's largest purchaser of contractor-provided services, obligating more than $207 billion on services contracts in fiscal year 2009. DOD contract management has been on GAO's high-risk list since 1992, in part because of continued weaknesses in DOD's management and oversight of contracts for services. The National Defense Authorization Act for Fiscal Year 2008 directed DOD to issue guidance providing for independent management reviews for services acquisitions. The Act required that the guidance provide a means to evaluate specific contracting issues and to address other issues, including identifying procedures for tracking recommendations and disseminating lessons learned. The Act also directed GAO to report on DOD's implementation of its guidance. GAO (1) assessed the extent to which DOD's guidance addressed the Act's requirements and how the guidance was implemented and (2) determined the status of actions taken by the military departments pursuant to DOD's guidance. GAO compared DOD's guidance with the Act's requirements; obtained data on the number of reviews conducted as of September 2009; and analyzed memoranda of 29 acquisitions valued at over $1 billion. In its written comments, DOD noted it planned to refine its processes to better share the lessons learned and best practices identified during peer reviews.
section_title: What GAO Found
paragraphs: To meet the legislative requirement regarding independent management reviews, DOD issued guidance in September 2008 and February 2009 providing for a peer review process for services acquisitions. DOD's guidance generally addresses requirements in the Act to issue guidance designed to evaluate specified contracting issues, but according to officials, DOD has not yet determined how it plans to disseminate lessons learned or track recommendations that result from the newly instituted reviews. Under this guidance, the Office of Defense Procurement and Acquisition Policy (DPAP) is responsible for conducting pre- and post-award peer reviews for services acquisitions with an estimated value of over $1 billion. Peer review teams include senior contracting officials from the military departments and defense agencies as well as legal advisors. As of September 30, 2009, DPAP had conducted 29 reviews of 18 services acquisitions, including 3 post-award reviews. DOD has also conducted peer reviews on two task orders but has not yet determined if it will do so on individual task orders in the future. The peer review teams made a number of recommendations and identified some best practices. DOD officials expect to refine their processes, including developing a more formal means for disseminating lessons learned and tracking recommendations, as DOD assesses its initial experiences with peer reviews. Each of the military departments has issued guidance establishing peer review processes for services acquisitions valued at less than $1 billion although the guidance is still evolving. The departments' guidance identifies the offices or commands tasked with conducting peer reviews based on various dollar thresholds. The military departments reported conducting hundreds of peer reviews for services acquisitions as of September 30, 2009, but could not provide exact numbers because of the lack of comprehensive reporting processes. Further, as peer review processes evolve, the military departments are considering ways to disseminate lessons learned and track recommendations.

section_title: Letter
paragraphs: The Department of Defense (DOD) is the federal government’s largest  purchaser of contractor-provided services, including professional and  management support, information technology support, weapon system  and base operations support, and intelligence support. DOD obligated  more than $207 billion on services contracts in fiscal year 2009, more than  double the amount it obligated in fiscal year 2001, when measured in real  terms. DOD’s reliance on contracted services makes effective  management and oversight of these contracts essential; however, DOD  contract management has been on our high-risk list since 1992.
Our recent work continues to identify weaknesses in DOD’s management  and oversight of services contracts. In 2006, we found that DOD’s  approach to managing services acquisitions did not allow the department  to determine whether its investments in services achieved the desired  outcomes. For example, we found that the department often focused its  efforts on awarding contracts without determining the areas of risk that  needed greater attention or capturing the knowledge to enable more  informed strategic decisions when contracting for such services. In  November 2009, we reported that DOD needed to take further actions to  improve its management of professional and management support  services, particularly in assessing the risks associated with contractors  performing tasks that closely support inherently governmental functions  and in providing guidance for enhancing oversight when contracting for  such services.
To improve DOD’s services acquisition process, Section 808 of the  National Defense Authorization Act for Fiscal Year 2008 (the Act) directed  the Secretary of Defense to issue guidance and implementing instructions  providing for periodic independent management reviews of contracts for  services. The Act specified that the guidance and instructions were to be  designed to evaluate certain issues (hereafter referred to as contracting  issues), including  contract performance in terms of cost, schedule, and requirements;  the use of contracting mechanisms, including the use of competition,  the contract structure and type, the definition of contract  requirements, cost or pricing methods, the award and negotiation of  task orders, and management and oversight mechanisms;  the contractor’s use, management, and oversight of subcontractors;  the staffing of contract management and oversight functions; and  the extent of any pass-through or excessive pass-through charges by  the contractor.
Further, when one contractor provides oversight for services performed  by other contractors, the Act required the DOD guidance to provide  procedures for the periodic review of such contracts to include the  evaluation of  the extent of DOD’s reliance on the contractor performing acquisition  functions closely associated with inherently governmental functions  and  the financial interest of any prime contractor performing acquisition  functions closely associated with inherently governmental functions in  any contract or subcontract in which the prime contractor provided  advice or recommendations to the agency.
In addition to the contracting issues above, the Act required the DOD  guidance to address several elements related to the review process. These  elements include the contracts subject to review, the frequency of reviews,  procedures for tracking the implementation of recommendations made  during reviews, and procedures for identifying and disseminating lessons  learned from reviews.
The Act also directed that we report on DOD’s implementation of the  guidance and instructions for independent management reviews.  Specifically, we (1) assessed the extent to which DOD’s guidance  addressed the Act’s requirements at the department level and how the  guidance was implemented and (2) determined the status of actions taken  by the military departments pursuant to DOD’s guidance. To do so, we  reviewed memoranda issued by DOD in September 2008 and February  2009 and compared these documents to the requirements of the Act. We  obtained information on the number of reviews that the Office of the  Under Secretary of Defense for Acquisition, Technology and Logistics’  Office of Defense Procurement and Acquisition Policy (DPAP) and the  military departments conducted as of September 30, 2009. We reviewed  summary memoranda for DPAP-led reviews of services contracts and  compared the topics discussed in the summary memoranda to DOD policy  and guidance, focusing on the contracting issues specified in the Act. We  also interviewed officials from DPAP and the military departments to  discuss how the guidance and instructions were developed and  implemented for the review process. A more detailed description of our  scope and methodology is included in appendix I.
We conducted this performance audit from October 2009 through January  2010 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: DOD Has Implemented Guidance That Generally Addresses Legislative Requirements
paragraphs: To meet the legislative requirements regarding independent management  reviews, DOD issued guidance and instructions providing for a peer review  process for services acquisitions. DOD’s guidance generally addresses  requirements prescribed in the Act to develop a process to evaluate the  specified contracting issues, but according to DOD officials, DOD has not  yet determined how the department plans to disseminate lessons learned  or track recommendations that result from the newly instituted reviews.  DOD officials expect to further refine their processes, including  developing a more formal means for disseminating lessons learned and  tracking recommendations as DOD assesses its initial experiences with  peer reviews. Through the first year of implementation, DPAP, which is  responsible for conducting reviews of acquisitions over $1 billion, had  conducted 29 peer reviews on 18 services acquisitions. Similarly, the  military departments, which are responsible for conducting reviews of  their acquisitions under $1 billion, issued guidance that provides for peer  reviews at various levels within the departments based on dollar values.  The military departments could not, however, determine the exact number  of peer reviews conducted because of the absence of comprehensive  reporting processes. Further, as peer review processes evolve, the military  departments are considering ways to disseminate lessons learned and  track recommendations.
subsections: section_title: DPAP Has Established a Peer Review Process for Services Acquisitions
paragraphs: DPAP issued a memorandum in September 2008 establishing a peer review  process to fulfill the requirement for an independent management review  of contracts for services. The requirement for a peer review process was  subsequently incorporated into DOD Instruction 5000.02, Operation of the  Defense Acquisition System, in December 2008. The guidance states that  these reviews are intended to ensure consistent and appropriate  implementation of policy and regulations, improve the quality of  contracting processes, and facilitate sharing best practices and lessons  learned. According to DOD officials, peer reviews by design are a means of  improving individual acquisitions and not necessarily a tool for  strategically managing DOD’s services portfolio.
Under DPAP’s guidance, peer reviews supplement its existing process to  review and approve services acquisitions. Pursuant to congressional  direction, DOD had previously established a management review process  that was intended to ensure that DOD services acquisitions are based on  clear, performance-based requirements with measurable outcomes and  that acquisitions are planned and administered to achieve intended  results. In these management reviews, DPAP assesses and approves the  acquisition strategies submitted by the military departments or defense  agencies for obtaining contractor-provided services estimated to be valued  at $1 billion or more. Once the acquisition strategies are approved, DOD  contracting offices may continue the acquisition process, including  soliciting bids for proposed work and subsequently awarding contracts.  DOD may award different contract types to acquire products and services,  or issue task orders under existing contracts. In November 2009, we  reported that the number of contracts and task orders issued after the  acquisition strategies were approved was significant. For example, we  reported that nearly 1,900 task orders were issued under the seven  professional and management support services acquisitions we reviewed.
DOD generally conducts peer reviews at three key points in the acquisition  process prior to contract award—prior to issuance of the solicitation  (phase 1), prior to request for final proposal revisions (phase 2), and prior  to contract award (phase 3)—and is to conduct periodic post-award  reviews (phase 4) (see fig. 1).
In February 2009, DOD issued guidance that clarified the relationship  between the management reviews and the peer reviews. For example, the  guidance identifies specific issues to assess and the criteria for the  reviewers to use during the management reviews or pre-award peer  reviews. According to the guidance, some contracting issues identified in  the Act, such as contract type and competition, are to be assessed during  the management reviews. Conversely, other contracting issues identified  in the Act, including requirements definition and the extent of the agency’s  reliance on contractors to perform functions closely associated with  inherently governmental functions, are to be assessed during pre-award  peer reviews. The pre-award peer reviews also are to evaluate several  elements of the source selection process that are not specified in the Act,  such as the clarity and consistency of the documentation. Further, the  guidance established review criteria for post-award reviews that address  each of the contracting issues identified in the Act. For example, during  post-award reviews, reviewers are to assess the extent to which the  contracting office was able to achieve competition for orders and whether  it was using appropriate contract types, well-defined requirements, and  appropriate cost/pricing methods.
According to DOD officials, in conducting these reviews, DPAP convenes a  peer review team consisting of three to five members. Officials said that  the teams are generally chaired by a deputy director within DPAP and  include participation from senior contracting officials from the military  departments and defense agencies as well as legal advisors from the Office  of the Secretary of Defense’s General Counsel. The teams review  acquisition documents prior to an on-site review and hold discussions with  contracting officers over multiple days. Upon completion of the on-site  review, peer review teams develop summary memoranda that include  observations and recommendations.
The February 2009 guidance indicated that DPAP is to review services  acquisitions with an expected value of over $1 billion. In addition, DPAP  may review acquisitions under that threshold that it has designated as  special interest because of the nature or sensitivity of the services to be  acquired. According to DOD officials, DPAP does not have a capability to  independently identify acquisitions that will require its review, but rather  relies on the military departments and defense agencies to notify DPAP of  acquisitions that will exceed the threshold. DPAP officials noted that some  reviews were not conducted because the military departments did not  notify DPAP that a peer review was necessary.
DPAP officials stated that they are currently focusing on the pre-award  peer reviews and are phasing in post-award peer reviews. As of   September 30, 2009, DPAP had conducted 29 peer reviews for 18 services  acquisitions. Because the peer review process was only implemented in  September 2008, no single acquisition has been subject to all phases of the  peer review process and no acquisition has been peer reviewed in both the  pre- and post-award phases. While most of the reviews have focused on  proposed acquisitions for which the initial contract had not yet been  awarded, DPAP has also conducted two phase 3 peer reviews for  proposed task orders valued at over $1 billion that were to be issued under  an existing contract that had previously been reviewed. DPAP has not yet  determined if it will establish a policy for conducting peer reviews for all  individual task orders over this amount in the future. For the 29 peer  reviews of services acquisitions that DPAP conducted, figure 2 shows  when each review occurred and the corresponding milestone. For  example, DPAP conducted a phase 1 peer review prior to the issuance of  the solicitation for 12 of the 18 services acquisitions.
Our review of the summary memoranda of the pre-award peer reviews that  DPAP conducted as of September 30, 2009, found that review teams  generally documented the evaluation of the use of contracting  mechanisms and, to a lesser extent, the use, management, and oversight of  subcontractors. DPAP officials noted that other contracting issues may  have been discussed during pre-award site visits and not included in the  summary memorandum because the peer review team did not identify any  concerns that warranted inclusion. Further, we found that review teams  made several related recommendations, as illustrated in the following  examples:    One pre-award peer review team recommended that the contracting  office reconsider the number of contracts that it had proposed be  awarded under an acquisition. In this case, the contracting office had  proposed limiting the number of contracts to three prior to knowing  what proposals and business arrangements would be submitted by  industry. The peer review team noted that this may unduly restrict  flexibility of the military department. Further, the team was unsure if  documentation to support the limitation on contract number would be  sufficient to withstand a bid protest from an unsuccessful offeror.
Another pre-award peer review recommended that the contracting  office increase its use of subcontractors and encourage the prime  contractors to establish mentor-protégé relationships with their  subcontractors to bring more qualified contractors into an industry.
Our review of the summary memoranda for the three post-award peer  reviews conducted by DPAP found that consistent with guidance, the  review teams evaluated all the contracting issues identified in the Act. All  three summary memoranda listed the required contracting issues and then  reported the peer review teams’ observations and recommendations for  the contracting offices to consider for the acquisition, as illustrated by the  following examples:    One post-award peer review team recommended that the contracting  officer modify the contract to include provisions requiring the  contractor to provide information on pass-through charges for all  future task orders issued. At the time of the peer review, the contract  did not contain a clause requiring the contractor to provide such  information, and therefore the government was unable to determine  the extent of pass-through charges and whether they were excessive.
Another post-award team recommended that the contracting office  reduce the use of time-and-materials task orders. In this case, the  acquisition strategy envisioned that most of the work would be  performed through fixed-priced task orders; however, time-and- materials task orders accounted for 62 percent of the value of orders  issued under the contract in the first 2 years of performance.
While DPAP’s guidance noted that the recommendations made during peer  reviews are advisory in nature, it also states that contracting offices are to  document in the contract file the disposition of all pre-award peer review  recommendations prior to contract award. The guidance does not address  recommendations made during post-award reviews. According to DOD  officials, contracting offices generally accept recommendations provided  by the peer review teams. DPAP officials said that if the contracting office  decides not to accept a peer review team’s recommendation, the  contracting officer is expected to document the reason in the contract file  and provide a copy to DPAP.
In addition to providing recommendations to address potential issues in  proposed acquisitions, the peer review teams have also identified some  best practices. For example, in one summary memorandum the team  called attention to the contracting office’s post-award performance plan  for the acquisition, which specified how the office intended to evaluate  and assess contract performance to maintain effective contract  surveillance procedures. The team noted that the plan allowed real-time  access to detailed cost performance data when combined with regular  surveillance. According to officials, DOD, however, has not yet issued  guidance establishing procedures to systematically track the  recommendations made by peer review teams or disseminate best  practices as required by the Act. DOD officials noted that to date, sharing  lessons learned from peer reviews has largely occurred through word of  mouth or through conferences. For example, at a December 2009  conference for senior DOD contracting officials, DPAP presented an  update on its peer review process that included a discussion of lessons  learned. To identify methods to better disseminate trends, lessons learned,  and best practices identified during peer reviews, in August 2009 DPAP  established a subcommittee within the Panel on Contracting Integrity.  DPAP officials expect that the subcommittee will report on its findings in  2010. Further, an official stated that DPAP plans to consider ways to track  the implementation of recommendations made during peer reviews.
section_title: Military Departments’ Peer Review Processes Are Evolving
paragraphs: The September 2008 DPAP guidance required the military departments to  establish their own procedures for conducting pre- and post-award peer  reviews on acquisitions under $1 billion, but provided the flexibility to the  services to tailor the process to best meet their needs. In response, the Air  Force issued its guidance in January 2009, the Navy in March 2009, and  the Army in April 2009. The military departments’ policies varied in such  areas as the frequency and timing of the reviews and the organizational  levels delegated responsibility for conducting the reviews. For example,  the Air Force conducts up to five pre-award pre-award peer reviews  whereas the Army conducts two (see fig. 3). The military departments plan  to refine their policies as they gain experience with the peer review  process.
According to officials, both the Air Force and Army modified existing pre- award reviews to incorporate the peer review requirements. The existing  reviews were mandatory steps in each department’s contract award  process and, as such, focused on the proposed acquisition’s contracting  approach, source selection process, and readiness to issue a contract  solicitation. Air Force officials stated that the department previously had a  post-award review process that focused on cost, schedule, and  performance metrics, which was revised to incorporate peer review  requirements. Army officials noted that the Army has focused its attention  on implementing pre-award peer reviews, but has not yet established a  post-award peer review process. These officials noted that the Army plans  to issue guidance on conducting post-award reviews in 2010.
In contrast, the Navy developed a new process, modeled on DPAP’s  process, to review proposed services acquisitions. Navy officials are  considering making some refinements to this process. For example, at the  time of our review the Navy had not yet determined the optimal timing of  its post-award peer reviews. The department was trying to determine a  point at which there had been enough contract performance to evaluate  the contractor while still allowing the contracting officers sufficient time  to implement any peer review team recommendations prior to exercising  an option year.
While DPAP was not required to approve the military departments’  guidance, DPAP officials reported that the guidance issued by the military  departments was consistent with the intent of the September 2008  guidance. There are differences, however, in how the military departments  addressed certain issues. For example, each of the military departments  delegated responsibility for conducting peer reviews to commands and  organizational units within their departments based on expected  acquisition value. In that regard:    The Air Force delegated responsibility for conducting peer reviews to  its major commands for proposed services acquisitions valued from  $50 million to $1 billion.
The Army delegated responsibility to the head of the contracting  activity within each of its commands for conducting peer reviews for  services acquisitions valued from $250 million to $1 billion. Similarly, it  identified the principal assistant responsible for contracting as being  responsible for conducting peer reviews for acquisitions from  $50 million to $250 million.
The Navy delegated responsibility to the Deputy Assistant Secretary of  the Navy – Acquisition and Logistics Management (DASN-A&LM) for  conducting peer reviews for acquisitions valued from $250 million to  $1 billion, while individual commands are responsible for conducting  reviews of acquisitions valued from $50 million to $250 million.
Further, the Air Force does not require peer reviews on noncompetitive  acquisitions—in other words, on contracts awarded using other than full- and-open competition. Air Force officials explained that such contracts  are already reviewed under a separate process and therefore believed that  an additional peer review would be unnecessary. Similarly, both the Air  Force and Army allow the offices responsible for conducting reviews to  waive peer reviews under certain circumstances, whereas the Navy does  not provide for a waiver process. Air Force guidance allows peer reviews  to be waived based on acquisition/source selection history, such as for  recurring acquisitions and where there is no history of bid protests. The  Army also allows peer reviews to be waived but did not specify in its  guidance which acquisitions could be waived.
As of September 2009, the military departments reported conducting  hundreds of peer reviews for services acquisitions, but the departments do  not have comprehensive processes for determining the exact number of  reviews conducted. Specifically:    The Navy reported that it had conducted 257 peer reviews for services  acquisitions, including 5 post-award reviews. The Navy could not  identify how many of the reviews conducted by the commands  occurred by September 30, 2009. DASN-A&LM conducted its first 4  peer reviews on September 22, 2009.
Though the Air Force did not know the specific number of peer  reviews conducted, officials noted that it had conducted up to five pre- award reviews on approximately 85 services acquisitions as of  September 30, 2009.
Army officials stated that though commands had conducted pre-award  peer reviews, an exact number of reviews could not be identified  because the Army does not have a reporting process. The Army also  acknowledged that it did not conduct any post-award reviews because  it has not yet established a post-award peer review process.
As peer review processes evolve, the military departments are considering  ways to disseminate lessons learned and track recommendations. For  example, Navy officials said the department is waiting to see the results of  initial reviews and will then develop additional guidance to address  lessons learned made during peer reviews. Army officials stated that the  department plans to address recommendations and lessons learned in  2010 when it issues guidance on post-award reviews. Finally, Air Force  policy requires commands to submit annual reports to the Secretary of the  Air Force – Acquisition and Contracting Policy that are to include major  issues identified during pre-award peer reviews and the resolutions taken.
section_title: Concluding Observations
paragraphs: DOD’s guidance implementing a peer review process for major services  acquisitions at the departmental level generally addresses the  requirements prescribed by the Act. While DOD has derived benefits from  these initial reviews, it has also recognized that there are issues that still  need to be addressed, such as how to track recommendations and  disseminate lessons learned. Further, at this stage, DOD’s focus has been  on evaluating acquisition strategies and proposed contracts at the pre- award stage. DOD has conducted relatively few post-award reviews, in  which DOD assesses how well it is managing the contractor’s actual  performance. A key issue is whether and how to apply the peer review  process to task orders through which DOD obtains much of its contractor- provided services. Few of these are large enough to reach the $1 billion  DOD review threshold, but below the threshold they could be so  numerous as to overtax the departments’ peer review processes.  Addressing these issues, as well as those at the military department level,  is important if DOD is to achieve its stated objectives for peer reviews— ensuring consistent and appropriate implementation of policy and  regulations, improving the quality of contracting processes, and facilitating  sharing best practices and lessons learned—on a more strategic or  enterprisewide basis rather than limiting the peer reviews’ benefits to the  individual acquisitions being reviewed. Although we are not making any  recommendations because DOD plans to address these issues, resolving  these concerns in a timely manner is essential if DOD is to maximize the  benefits of the peer review process.
section_title: Agency Comments
paragraphs: DOD provided written comments on a draft of this report. In its comments,  DOD stated that peer reviews had improved the quality of its significant  business arrangements. DOD indicated that it will continue to refine its  peer review process to better disseminate trends, lessons learned, and best  practices that are identified during peer reviews. DOD provided a  technical comment, which was incorporated into the report. DOD’s  comments are reprinted in appendix II.
We are sending copies of this report to the Secretary of Defense; the  Secretaries of the Air Force, Army, and Navy; and interested congressional  committees. The report also is available at no charge on GAO’s Web site at  http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: Section 808 of the National Defense Authorization Act for Fiscal Year 2008  (the Act) directs GAO to report on the Department of Defense’s (DOD)  implementation of its guidance and implementing instructions providing  for periodic management reviews of contracts for services. In response to  this mandate, we (1) assessed the extent to which DOD’s guidance  addressed the Act’s requirements at the department level and how the  guidance was implemented and (2) determined the status of actions taken  by the military departments pursuant to DOD’s guidance.
To do so, we reviewed DOD’s September 2008 and February 2009 guidance  issued by the Under Secretary of Defense for Acquisition, Technology and  Logistics’ Office of Defense Procurement and Acquisition Policy (DPAP).  We compared the guidance and instructions to the requirements stipulated  in Section 808 of the Act. The September 2008 guidance indicated that peer  reviews were to be conducted for both supplies and services. As the Act’s  requirements were specific to services acquisitions, we limited our  analysis to services. We also obtained guidance and implementing  instructions issued by the Departments of the Air Force, Army, and Navy.  We interviewed officials from DPAP and the Departments of the Army,  Navy, and Air Force to gain further insight into how each organization  developed its guidance and instructions. DOD’s September 2008  memorandum also indicated that defense agencies were required to  develop their own guidance. While these were outside the scope of our  review, DPAP officials indicated that 13 of 17 defense agencies that DPAP  believed would be required to develop guidance had done so at the time of  this review.
We obtained information on the number of peer reviews on services  acquisitions that DPAP and the military departments reported they had  conducted as of September 30, 2009. DPAP was able to identify the  number of reviews that it had conducted. We determined this information  to be sufficiently reliable for the purposes of our review. The Air Force  provided an approximate number of acquisitions that had been reviewed  but could not identify the number of individual peer reviews conducted.  The Army did not provide any information on the specific number of  reviews conducted. The Navy provided information on the number of  reviews it had conducted but could not specify how many had been  conducted as of September 30, 2009. We could not independently verify  the information provided by the military departments because of the lack  of available documentation.
To determine the nature of the discussions and the issues addressed  during peer reviews, we obtained the summary memoranda from each of  the 29 peer reviews conducted by DPAP as of September 30, 2009. These  29 memoranda represented 18 unique acquisitions, as DPAP had reviewed  some acquisitions more than once. Twenty-six of the memoranda were for  pre-award peer reviews and 3 were for post-award reviews. We analyzed  summary memoranda from each of the 29 peer reviews to determine the  topics discussed in the memoranda, focusing specifically on the  contracting issues identified in the Act. We also interviewed DPAP  officials who chaired or participated in these reviews to obtain their views  on the peer review process.
We conducted this performance audit from October 2009 through January  2010 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Timothy DiNapoli, Assistant  Director; E. Brandon Booth; Morgan Delaney Ramaker; Christopher  Mulkins; Thomas Twambly; and Alyssa Weir made key contributions to  this report.