ARMY CORPS OF 

ENGINEERS

Peer Review Process 

for Civil Works 

Project Studies Can 

Be Improved

Report to the Ranking Member, 

Subcommittee on Water Resources and 

Environment, Committee on 

Transportation and Infrastructure, 

House of Representatives

March 2012

GAO-12-352

United States Government Accountability Office

GAO
United States Government Accountability Office

Highlights of GAO-12-352, a report to the 

Ranking Member, Subcommittee on Water 

Resources and Environment, Committee on 

Transportation and Infrastructure, House of 

Representatives

March 2012

ARMY CORPS OF ENGINEERS

Peer Review Process for Civil Works Project Studies 

Can Be Improved

Why GAO Did This Study

Section 2034 of the Water Resources 

Development Act of 2007 requires that 

certain U.S. Army Corps of Engineers 

(Corps) civil works project studies 

undergo independent external peer 

review to assess the adequacy and 

acceptability of the methods, models, 

and analyses used. In the act, 

Congress established a 7-year trial 

period for this requirement and also 

required the Corps to submit two 

reports on its experiences with the 

peer review process.

GAO was asked to examine (1) the 

number of Corps project studies that 

have undergone independent peer 

review in response to section 2034, 

(2) the cost of these peer reviews, 

(3) the extent to which the Corps’ 

process for determining if a project 

study is subject to peer review is 

consistent with section 2034, (4) the 

process the Corps uses to ensure that 

the contractors it hires and the experts 

the contractors select to review project 

studies are independent and free from 

conflicts of interest, and (5) the extent 

to which peer review recommendations 

have been incorporated into project 

studies. GAO reviewed relevant laws, 

agency guidance, and documents and 

interviewed Corps officials and 

contractors.

What GAO Recommends

GAO recommends that the Department 

of Defense direct the Corps to, among 

other actions, better track peer review 

studies, revise the criteria for 

determining which studies undergo 

peer review and the timing of these 

reviews, and improve its process for 

ensuring contractor independence. The 

department generally concurred with 

these recommendations. 

What GAO Found

Since enactment of the Water Resources Development Act of 2007, 49 project 

studies have undergone peer review but it is unclear how many were performed 

in response to section 2034 requirements because the Army Corps of Engineers 

(Corps) does not make specific determinations or track if a peer review is being 

conducted under section 2034. In February 2011, in response to section 2034, 

the Corps submitted its initial report to Congress summarizing its implementation 

of the peer review process. In its report, however, the Corps did not distinguish 

which studies had been selected for peer review in accordance with section 2034 

and therefore, did not provide Congress information that would help decision 

makers evaluate the requirements of section 2034 at the end of the trial period.

The 49 peer reviews resulted in both direct and indirect costs. Specifically, these 

peer reviews resulted in direct costs of over $9 million in contract costs and fees. 

In addition, Corps staff resources were used to manage the reviews, although 

these costs are not fully quantifiable. Furthermore, the addition of peer review to 

the Corps study process has resulted in indirect costs by altering project study 

schedules to allow for time needed to complete peer reviews. In some cases 

where a peer review was not planned during the early stages of the study 

process, significant delays to project studies occurred while funds were sought to 

pay for the peer review. In contrast, according to some Corps officials, when 

project managers have built in time and identified funding for peer reviews early, 

the process has had less of an impact on project study schedules. 

The Corps’ process for determining whether a project study is subject to peer 

review is more expansive than section 2034 requirements because it uses 

broader criteria, resulting in peer reviews of studies outside the scope of section 

2034. In addition, the process the Corps uses does not include the flexibility 

provided in section 2034, which allows for the exclusion of certain project studies 

from peer review. Moreover, some studies are undergoing peer reviews that do 

not warrant it, according to some Corps officials GAO spoke with. 

The Corps has a process to review general information on contractors’ conflicts 

of interest and independence when selecting them to establish peer review 

panels, but it does not have a process for reviewing project-level information on 

conflicts of interest and independence. As a result, it cannot be assured that 

contractors do not have conflicts at the project-level. In contrast, the Corps’ 

contractors do have a process for reviewing information related to conflicts of 

interest and the independence of experts selected for each peer review panel.

The Corps has adopted and incorporated into its project study reports most of the 

peer review recommendations it has received. Doing so has resulted in some 

technical improvements to study reports but generally has not changed the 

Corps’ decisions about project alternatives, in part because the peer review 

process occurs too late in the project study process to affect decision making, 

according to some Corps officials GAO spoke with. As a result, some 

recommendations about alternatives may not have been implemented because 

the decision on the preferred design had already been made.

View GAO-12-352. For more information, 

contact Anu K. Mittal at (202) 512-3841 or 

mittala@gao.gov. 
Page i GAO-12-352 Army Corps of Engineers

Letter 1 

Background 4

It Is Unclear How Many Peer Reviews Have Been Completed in 

Response to Section 2034 Requirements 15

Completed Peer Reviews Have Cost Millions of Dollars in Direct 

and Indirect Costs 16

The Corps’ Process for Initiating Peer Review Is More Expansive 

and Less Flexible Than Section 2034 Requirements 20

Gaps Exist in the Corps’ Process for Screening Its Contractors Who 

Are Responsible for Selecting Experts for Peer Review Panels 25

The Corps Has Adopted Most Peer Review Recommendations, 

Resulting in Technical Improvements but Generally No Changes 

in Project Decisions 28

Conclusions 32

Recommendations for Executive Action 33

Agency Comments and Our Evaluation 33

Appendix I Objectives, Scope, and Methodology 36

Appendix II Characteristics of Studies That Underwent Peer Review 39

Appendix III Comments from the Department of Defense 48

Appendix IV GAO Contact and Staff Acknowledgments 52

Tables

Table 1: Corps EC 209 Guidance on Conducting Peer Review 20

Table 2: Characteristics of Studies That Underwent Peer Review 39

Figures

Figure 1: National Academy of Sciences Criteria for Selecting an 

Appropriate Level of Review 8

Figure 2: Key Steps in the Feasibility Study Process 14

Contents
Page ii GAO-12-352 Army Corps of Engineers

Abbreviations

EIS environmental impact statement

NAS National Academy of Sciences

NEPA National Environmental Policy Act

OMB Office of Management and Budget

WRDA Water Resources Development Act

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-12-352 Army Corps of Engineers

United States Government Accountability Office

Washington, DC 20548

March 8, 2012

The Honorable Timothy Bishop

Ranking Member

Subcommittee on Water Resources and Environment

Committee on Transportation and Infrastructure

House of Representatives

Dear Mr. Bishop:

Through its civil works program, the U.S. Army Corps of Engineers 

(Corps) constructs, operates, and maintains thousands of civil works 

projects related to water resources across the United States.1 These 

projects aim to provide safe and reliable waterways; reduce risk to 

people, homes, and communities from flooding and coastal storms; 

restore and protect the environment; and address water resources 

challenges. A Corps civil works project generally starts with a study of a 

water resources issue and the development of various alternatives to 

address it. Such studies can span the full range of Corps civil works 

projects, and can include those that are small and low impact and others 

that are large and complex, with potentially significant economic and 

environmental impacts. Through its civil works program, the Corps 

operates 50 centers of expertise and seven research laboratories that 

assist its eight divisions and 38 district offices in the planning, design, and 

technical review of civil works projects.2

Through its civil works projects, the Corps provides vital public 

engineering services in peace and war to strengthen the nation’s security, 

Six of these centers are focused 

on the quality and effectiveness of water resources planning and are 

referred to as “planning centers of expertise.” 

 

1

The Corps has both a military and a civil works program. The military program provides, 

among other things, engineering and construction services to other federal agencies and 

foreign governments, and the civil works program is responsible for investigating, 

developing, and maintaining water resources projects. This report discusses only the civil 

works program. 

2

The centers of expertise are designated individuals or organizations—located either in 

district offices, division offices, or research laboratories—with capability or expertise in a 

specialized area. The Corps designates employees at various levels within the Corps to 

oversee, manage, and coordinate the centers.

 
 

Page 2 GAO-12-352 Army Corps of Engineers

energize the economy, and reduce risks from disasters. These projects 

involve navigation and flood control activities, environmental restoration, 

and emergency response—most recently including emergency response 

to Missouri River flooding and rebuilding after Hurricane Katrina. 

Technical errors in past studies of Corps projects, however, had raised 

concerns about the effectiveness of the Corps’ internal review processes 

and the quality of the studies that the Corps used as a basis for its civil 

works projects.3 For example, in March 2006, we reported that certain 

studies completed by the Corps from 1992 through 2002 were fraught 

with errors, mistakes, and miscalculations and used invalid assumptions 

and outdated data.4

In the wake of these reports, Congress passed section 2034 of the Water 

Resources Development Act (WRDA) of 2007, which requires that studies 

for certain Corps projects undergo independent peer review.

We also reported that these Corps studies 

understated costs, overstated benefits, and did not provide a reasonable 

basis for decision making. Similar findings have been documented in 

reviews by the National Academy of Sciences (NAS) and other 

organizations, which concluded that the Corps’ review processes needed 

to be strengthened.

5 To conduct 

such peer review, the Corps hires a contractor to select a panel of 

independent experts, who assess the adequacy and acceptability of the 

economic, engineering, and environmental methods, models, and 

analyses used in a Corps’ project study. Upon completion of the peer 

review, the Corps is to consider recommendations from the review before 

making a final decision on the project.6

 

3

National Research Council, Inland Navigation System Planning: The Upper Mississippi 

River-Illinois Waterway (Washington, D.C.: National Academies Press, 2001), and GAO, 

Corps of Engineers: Observations on Planning and Project Management Processes for 

the Civil Works Program, GAO-06-529T (Washington, D.C.: Mar. 15, 2006).

4

GAO-06-529T. GAO’s review focused on four civil works studies.

5

Water Resources Development Act of 2007, Pub. L. No. 110-114, § 2034 (Nov. 8, 2007), 

codified at 33 U.S.C. § 2343. 

6

The final Corps approval of a project recommendation is signed by the Chief of 

Engineers—the Corps’ commanding officer—in what is known as the signed Chief’s 

report. This report summarizes the study’s results and includes a final project 

recommendation; the report is generally submitted to Congress for authorization of 

construction.
 

Page 3 GAO-12-352 Army Corps of Engineers

You asked us to review the efforts the Corps has made to implement and 

comply with the independent peer review requirements in section 2034 of 

WRDA 2007. This report examines (1) the number of Corps project 

studies that have undergone independent external peer review in 

response to section 2034, (2) the cost of these peer reviews, (3) the 

extent to which the Corps’ process for determining if a project study is 

subject to peer review is consistent with section 2034, (4) the process the 

Corps uses to ensure that the contractors it hires and the experts the 

contractors select to review project studies are independent and free from 

conflicts of interest, and (5) the extent to which peer review 

recommendations have been incorporated into project studies.

To conduct this work, we reviewed relevant legal requirements, policy 

guidance, review plans, and peer review reports for project studies that 

were subject to a peer review and for which a report had been completed 

since WRDA 2007 was passed. In addition, we selected a 

nongeneralizable sample of six peer reviews to examine in greater depth, 

to better understand the costs associated with conducting these reviews, 

as well as the overall impact of the process on the timeline of the project 

study and the study’s outcome. Because this sample was a nonprobability 

sample, the information derived from these reviews is not generalizable to 

all peer reviews, but the reviews serve as illustrative examples that 

provide valuable insights into the Corps’ peer review process; we 

selected a review from each planning center of expertise and at least one 

for each of the three contractors selected to conduct peer reviews. We 

conducted semistructured interviews with officials from Corps 

headquarters, the planning centers of expertise involved in managing the 

peer reviews, all of the Corps’ eight divisions, and from 10 geographically 

dispersed districts that had conducted project studies that underwent peer 

review. We also conducted semistructured interviews with the three 

contractors, as well as with selected peer review panel members and 

local sponsors of Corps civil works projects.7

To determine the number of studies that underwent peer review and the 

cost of these reviews, we reviewed all completed peer review reports and 

 

7

The Corps also typically receives funds from each project’s local sponsor, which may be 

a state, tribal, county, or local agency or government. WRDA of 1986 stipulated that 

nonfederal sponsors share the cost of planning and implementing most Corps civil works 

projects. The division of federal and nonfederal cost sharing required varies by project 

purpose. 
 

Page 4 GAO-12-352 Army Corps of Engineers

contract award documentation. In addition, for the six peer reviews we 

examined in depth, we analyzed information on costs associated with 

managing the review process, including those associated with district and 

other staff time. To determine the extent to which the Corps’ process for 

determining if a study is subject to peer review is consistent with section 

2034, we analyzed the legal requirements and relevant Corps policy 

guidance for determining when to conduct peer reviews, and we reviewed 

documentation on Corps decisions. To determine the process the Corps 

uses to ensure that the contractors it hires and the experts the contractors 

select are independent and free from conflicts of interest, we analyzed 

relevant Corps policy guidance and reviewed documentation provided by 

the contractors to demonstrate that they and the selected experts meet 

the requirements. To determine the extent to which recommendations are 

incorporated into project studies, we analyzed relevant policy guidance 

and summarized peer review recommendations and Corps responses, 

but we did not assess the quality of the peer review recommendations or 

the technical sufficiency of the Corps responses to these 

recommendations. In addition, for the six reviews examined in depth, we 

analyzed project studies, recommendations, and Corps responses to 

describe the overall timeline and impact of the peer review process. 

Appendix I describes our scope and methodology in greater detail.

We conducted this performance audit from April 2011 to March 2012 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

The purpose of the Corps’ civil works project study process is to inform 

federal decision makers whether a water resources project warrants 

further federal investment. The study process is conducted in two phases: 

reconnaissance and feasibility. In the reconnaissance phase, the Corps 

conducts an initial evaluation of potential solutions to a water resources 

problem. If the Corps determines that a project potentially warrants 

federal investment, it proceeds to a more detailed feasibility study. The 

feasibility phase generally begins with the signing of a feasibility costshare agreement between the Corps and the local project sponsor. 

Background 

Corps Process for Civil 

Works Project Studies
 

Page 5 GAO-12-352 Army Corps of Engineers

Feasibility studies are generally prepared by the Corps’ 38 district offices, 

with review and oversight provided by the cognizant Corps division office 

and by headquarters. During the feasibility phase, the Corps formulates 

and evaluates alternative plans for achieving the project’s objectives and 

reviews the proposed project to assess whether the benefits of 

constructing it outweigh its costs. At the beginning of this phase, a 

feasibility scoping meeting is held to bring the Corps, the local sponsor, 

and other agencies together to reach agreement on the problems and 

solutions to be investigated during the feasibility study and the scope of 

the analysis required. The next step includes an alternative formulation 

briefing to identify and resolve any legal or policy concerns and to obtain 

headquarters approval of the tentatively selected plan and to release the 

draft report to the public. Finally, the draft feasibility report—which 

presents the study results and findings, including those developed in the 

reconnaissance phase—is released to the public. At the conclusion of the 

feasibility phase, the Corps selects a recommended plan for proceeding 

with the project.

The feasibility report also includes analysis and documentation to meet 

the requirements of the National Environmental Policy Act (NEPA).8

Under NEPA, federal agencies are to assess the effects of major federal 

actions, such as Corps construction projects, that significantly affect the 

environment and prepare a detailed statement on the environmental 

impacts of those actions. NEPA has two principal purposes: (1) to ensure 

that an agency carefully considers detailed information concerning 

significant environmental impacts and (2) to ensure that this information 

will be made available to the public. NEPA requires an agency to prepare 

a detailed statement on the environmental impacts of any “major federal 

action” significantly affecting the environment. NEPA implementing 

regulations generally require an agency to prepare either an 

environmental assessment9

 

8

Pub. L. No. 91-190, 83 Stat. 852 (1970), codified as amended at 42 U.S.C. §§ 4321-4347 

(2011). 

or an environmental impact statement 

9

An environmental assessment is a concise public document that provides sufficient 

evidence and analysis for determining whether to prepare an environmental impact 

statement or a finding of no significant impact and is to include brief discussions of the 

need for the proposal, alternatives, the environmental impacts of the proposed action and 

alternatives, and a listing of agencies and persons consulted. 40 C.F.R. § 1508.9 (2011).
 

Page 6 GAO-12-352 Army Corps of Engineers

(EIS).10

Corps project studies have historically been subject to various levels of 

internal and external review under a number of authorities as well as the 

Chief of Engineers’ responsibility to ensure the quality of Corps studies. 

For example, in 1902, Congress created the Board of Engineers for 

Rivers and Harbors, which was the result of efforts to address 

inconsistent treatment of proposed Corps projects. The board was made 

up of Corps staff. Until 1992, when Congress terminated the board, it 

reviewed thousands of Corps studies for civil works projects and made 

unfavorable recommendations on more than half. At the time the board 

was abolished, there was concern that too much duplicative review was 

occurring between the board and other internal Corps review processes. 

Subsequently, in the Flood Control Act of 1944, Congress established a 

mechanism for external review of Corps projects by giving the head of the 

Department of the Interior and the governors of affected states an 

opportunity to comment on proposed Corps projects before authorization. 

Furthermore, starting in 1970, under NEPA, environmental impact 

statements for Corps projects were required to be sent to the heads of 

other federal agencies and governors of affected states for comment. As 

a result of the Flood Control Act of 1970, Congress created the position of 

Assistant Secretary of the Army for Civil Works, who coordinates the 

review of Corps studies with the Office of Management and Budget 

(OMB) before they are submitted to Congress.

NEPA implementing regulations also specify requirements and 

procedures—such as providing the public with an opportunity to comment 

on the draft EIS for at least 45 days.

Recent congressional interest in establishing an independent external 

peer review process for Corps project studies began in the late 1990s, 

following a series of damaging reports and events, including allegations 

that the Corps had manipulated information to justify projects. 

Investigations conducted by NAS and the Army’s Inspector General 

identified various problems with the Corps internal review process, 

 

10An EIS is a more detailed statement than an environmental assessment. An EIS must, 

among other things, (1) describe the environment that will be affected, (2) identify 

alternatives to the proposed action and identify the agency’s preferred alternative, (3) 

present the environmental impacts of the proposed action and alternatives, and (4) identify 

any adverse environmental impacts that cannot be avoided should the proposed action be 

implemented. 42 U.S.C. § 4332(c) (2011), 40 C.F.R. §§ 1501.4, 1508.11 (2011).

Increasing Interest in Peer 

Review of Corps Project 

Studies 
 

Page 7 GAO-12-352 Army Corps of Engineers

including the manipulation of economic analysis and potential institutional 

bias toward large construction projects.11 Around this time, WRDA 2000 

required the Corps to contract with NAS to study and make 

recommendations concerning the use of peer review for feasibility 

reports, including recommending potential criteria to determine how to 

apply peer review.12 In 2002, NAS released its study concluding that the 

Corps’ more complex water resources project planning studies should be 

subject to external, independent review.13

 

11National Research Council, Review of the U.S. Army Corps of Engineers Restructured 

Upper Mississippi River-Illinois Waterway Feasibility Study (Washington, D.C.: National 

Academies Press, 2004), and National Research Council, New Directions in Water 

Resources Planning for the U.S. Army Corps of Engineers (Washington, D.C.: National 

Academies Press, 1999). 

The study also found that not 

all Corps project studies necessarily require such review, recommending 

instead that external peer review be reserved for studies that are 

expensive, will affect a large area, are highly controversial, or involve high 

levels of risk. The study estimated that about five Corps projects per year 

would likely be subject to this level of review. According to the NAS study, 

criteria for selecting the appropriate level of review should balance the 

risks and consequences of inadequate review against the resources 

required for more complex and stringent levels of review. In addition, the 

study identified several criteria that should be considered in determining 

the appropriate level of review for Corps studies, primarily that as project 

magnitude and risks increase, an increasing degree of independence and 

scope of review are warranted (see fig. 1).

12Water Resources Development Act of 2000, Pub. L. No. 106-541, § 216(b) (2000).

13National Research Council, Review Procedures for Water Resources Project Planning

(Washington, D.C.: National Academies Press, 2002). 
 

Page 8 GAO-12-352 Army Corps of Engineers

Figure 1: National Academy of Sciences Criteria for Selecting an Appropriate Level of Review

In addition to recommendations related to the appropriate level of review 

for project studies, the 2002 NAS study made several other 

recommendations about the Corps’ peer review process. It recommended 

that peer review results be presented to the Chief of Engineers before a 

final decision on a project study is made, that the Chief of Engineers 

respond in writing to each key point of the peer review report, and that 

peer review be initiated early enough in the Corps’ study process so that 

review results can be meaningfully incorporated into project design.

After NAS published its 2002 study, OMB in December 2004 issued its 

Final Information Quality Bulletin for Peer Review citing the Information 

Quality Act,14

 

14Pub. L. No. 106-554, App. C, Title V, § 515(a) (2000). 

as well as its general authorities to oversee the quality of 

agency information, analyses, and regulatory actions. This OMB bulletin 
 

Page 9 GAO-12-352 Army Corps of Engineers

established governmentwide guidance on enhancing peer review 

practices and covers what information is subject to peer review, the 

selection of appropriate peer reviewers, opportunities for public 

participation, and related issues.15

The Corps faced further criticism after the failure of Corps levees and 

floodwalls in New Orleans in the wake of Hurricane Katrina in August 

2005. In 2006, the Corps announced “Twelve Actions for Change,” which 

included a set of actions intended to transform the Corps’ priorities, 

processes, and planning and apply lessons learned from Hurricanes 

Katrina and Rita. Among these actions was to employ independent review 

for projects with significant consequences, especially the potential for loss 

of life if the project were to fail.

The Corps’ Engineering Circular 1105-

2-408 (EC 408) was issued in May 2005 and established procedures for 

ensuring the credibility and quality of Corps documents by supplementing 

its previous review process, including to add external peer review to its 

review process in special cases where risk and magnitude warrant this 

level of review.

In November 2007, Congress passed WRDA 2007 and included section 

2034, which establishes a 7-year trial period for peer reviews of certain 

studies of civil works projects; this trial generally applies to project studies 

initiated by the Corps from November 2005 through November 2014.16

 

15Office of Management and Budget, Final Information Quality Bulletin for Peer Review, 

M-05-03 (Washington, D.C.: 2004). The OMB guidance for peer review applies to 

important scientific assessments, referred to as “influential scientific information,” which 

includes “highly influential scientific assessments,” disseminated by the federal 

government. The guidance calls for peer review of information that is based on novel 

methods or presents complex challenges for interpretation, contains precedent-setting 

methods or models, presents conclusions that are likely to change prevailing practices, or 

is likely to affect policy decisions that have a significant impact.

The Corps was to provide an initial report to Congress on its 

implementation of the peer review trial under section 2034 by November 

2010 and is to provide a final report by November 2013. In February 

2011, the Corps submitted its initial report to Congress summarizing its 

16Section 2034 applies to (1) project studies initiated from November 2005 through 

November 2007 and for which the array of alternatives to address the water resources 

problem had not been identified and to (2) project studies initiated from November 2007 

through November 2014.

Section 2034 of WRDA of 

2007
 

Page 10 GAO-12-352 Army Corps of Engineers

experiences implementing the peer review process in response to the 

requirement in section 2034 of WRDA 2007.

Section 2034 defines a project study as a feasibility or reevaluation 

study—including the EIS for that study—or any other study associated 

with a modification of a water resources project that includes an EIS. 

Under section 2034, project studies that meet at least one of the following 

criteria are required to undergo peer review:

• The project has an estimated total cost of more than $45 million.

• The governor of an affected state requests an independent peer 

review.

• The Chief of Engineers determines that the project study is 

controversial (i.e., significant public dispute exists as to the project’s 

size, nature, or effects or its economic or environmental costs or 

benefits).

In addition, if the head of a federal or state agency charged with reviewing 

a project study determines the project is likely to have a significant 

adverse impact on environmental, cultural, or other resources, he or she 

may request that the Corps consider a peer review by an independent 

panel of experts.

WRDA 2007 also provides some instances where exceptions may be 

made to peer review. For example, the Corps may exclude from peer 

review certain projects having a total estimated cost of more than 

$45 million but do not include an EIS, have not been determined by the 

Corps to be controversial, and come below specified thresholds of 

adverse impacts. The Corps may also exclude other project studies 

meeting certain exclusion criteria. For example, the Corps may exclude 

studies that involve

• only the rehabilitation or replacement of existing hydropower turbines, 

lock structures, or flood control gates within the same footprint and for 

the same purpose as an existing water resources project;

• an activity for which the Corps and industry have ample experience, 

so the activity may be considered routine; and

• minimal life safety risk.
 

Page 11 GAO-12-352 Army Corps of Engineers

Section 2034 also has requirements for the Corps concerning the peer 

review panel and its independence, as well as for timing the peer review 

and publishing peer review reports, as described in more detail below.

Under section 2034, the Corps is required to contract with NAS, a similar 

independent scientific and technical advisory organization, or an “eligible 

organization” to establish a panel of experts that will review a project 

study. Section 2034 defines an eligible organization as having the 

following five characteristics:

• is a 501(c)(3) tax-exempt organization,

• is independent,

• is free from conflicts of interest,

• does not carry out or advocate for or against federal water resources 

projects, and

• has experience in establishing and administering panels.

Section 2034 states that when establishing peer review panels, 

contractors must apply NAS’s policy for selecting committee members to 

ensure that they also have no conflicts of interest. The NAS Policy on 

Committee Composition and Balance and Conflicts of Interest outlines 

several criteria for selecting peer review panel members, including the 

following:

• All panel members must be highly qualified in terms of knowledge, 

training, and experience.

• The knowledge, experience, and perspectives of the panel members 

must be thoughtfully and carefully assessed and balanced in terms of 

the subtleties and complexities of the particular scientific, technical, 

and other issues to be addressed.

• Potential sources of bias must be assessed to determine that the 

panel’s report will not be compromised by issues of bias or lack of 

objectivity.

Contractor Selection, 

Conflicts of Interest, and 

Independence 
 

Page 12 GAO-12-352 Army Corps of Engineers

• Panel members must not have financial interests that could 

significantly impair their objectivity or create an unfair competitive 

advantage for any person or organization.

• Panel members must not obtain and use, or intend to use, confidential 

information not reasonably available to the public for their own or 

direct and substantial economic benefit.

• Panel members must not serve as a member on a peer review panel 

that is to review the panel member’s own work.

• Panel members must not have become committed to a fixed position 

related to the review for which they have a significant directly related 

interest or duty.

• Persons currently employed by the agency sponsoring the study 

cannot be panel members, except in extremely limited special 

circumstances.17

Additionally, section 2034 requires that both the experts selected for the 

peer review panels and the organizations managing the peer review 

selections be independent. Section 2034 does not define the term 

independent, but both the 2002 NAS peer review study and OMB’s Final 

Information Quality Bulletin for Peer Review regard independent to mean 

external to the Corps.18

 

17In special circumstances and to the extent not prohibited by federal or state laws or 

regulations, such an individual may serve as a member of such a committee where the 

following requirements are met: (1) the service of the individual on the committee must be 

based upon the unique scientific or technical expertise the individual brings to the 

committee; (2) the individual must not be involved in any way within the agency in any 

deliberative or decision-making process or any policymaking or similar process relating to 

the study or other activity or the expected or intended results of the study or other activity; 

and (3) it must be specifically determined during the committee appointment process that 

service by the individual will not compromise, or appear to compromise, the independence 

or objectivity of the particular study or other activity in which the committee is engaged.

Specifically, the NAS study states that a fully 

independent review can be accomplished only by reviewers who are free 

of conflicts of interest and are appointed by a group external to the Corps. 

Similarly, the OMB bulletin states that independent reviewers are 

generally not employed by the agency or office producing the document.

18National Research Council, Review Procedures (2002), and Office of Management and 

Budget, Final Information Quality Bulletin for Peer Review (2004). 
 

Page 13 GAO-12-352 Army Corps of Engineers

Section 2034 requires that the peer review be conducted during the 

period from the signing of the feasibility cost-share agreement between 

the Corps and the local sponsor and 60 days after the last day of the 

public comment period for the draft project study. Additionally, section 

2034 lists three points during the feasibility study process at each of 

which the Chief of Engineers must consider whether to initiate peer 

review:

• when the without-project conditions—current and forecasted 

conditions if the project were not constructed—are identified,

• when the array of alternatives to be considered are identified, and

• when the preferred alternative is identified.

Figure 2 shows the key steps in the feasibility study process, including 

those specified in section 2034. The Corps can conduct peer review at 

any time during the steps shown highlighted in gray in the figure, but 

according to Corps officials it generally conducts peer review after the 

draft feasibility report has been completed. The Washington-level review 

shown as the final step in the figure concludes with a signed Chief’s 

report for project studies that will be submitted to Congress for 

authorization.

Timing of Peer Review and 

Publishing of Peer Review 

Reports 
 

Page 14 GAO-12-352 Army Corps of Engineers

Figure 2: Key Steps in the Feasibility Study Process

Note: The Corps’ independent peer review process can take place at any time during the steps 

enclosed by the shaded gray area.

a

The feasibility scoping meeting brings the Corps, local sponsor and relevant government agencies 

together to reach agreement on the problems and solutions to be investigated during the feasibility 

study. The without-project conditions are identified during the feasibility scoping meeting.

b

The alternative formulation briefing confirms that the plan formulated for the project study and the 

definition of federal and nonfederal responsibilities are consistent with applicable laws, statutes, 

executive orders, regulations, and current policy guidance. The array of alternatives to be considered 

is identified at the alternative formulation briefing.

c

The preferred alternative is identified when the draft feasibility report is released.
 

Page 15 GAO-12-352 Army Corps of Engineers

Section 2034 also requires the Corps to prepare and make publicly 

available a written response to all completed peer review reports before it 

finalizes project studies. The Corps must provide Congress with a copy of 

both the completed peer review report and the Corps’ written response 

when the signed Chief’s report or other final decision document for the 

project study is transmitted to Congress.

The Corps’ Engineering Circular 1165-2-209 (EC 209) was issued in 

January 2010 and establishes its civil works review policy, which outlines 

the processes for implementing product review requirements for Corps 

civil works projects. EC 209 was developed to include the specific 

requirements for independent peer review contained in section 2034, 

OMB’s 2004 peer review guidance, as well as other Corps policy 

considerations. EC 209 requires that districts, in coordination with the 

relevant Corps planning center of expertise, prepare review plans for 

project studies. These review plans are to describe the appropriate levels 

of potential review that the specific project study will be subject to, such 

as the district’s quality control procedures, agency technical review, peer 

review, and policy and legal review. If a project study review plan 

indicates that a peer review will not be conducted, then the district is 

required to develop a risk-based recommendation for why the peer review 

is not required. This recommendation should document, among other 

things, that the project study is of such limited scope or impact that it 

would not benefit from a peer review.19

Since enactment of WRDA 2007, 49 Corps civil works project studies 

have undergone peer review as of January 2012, but it is unclear how 

many of these reviews were performed in response to the requirements in 

section 2034. This is because the Corps does not make specific 

determinations or track whether a peer review is being conducted in 

response to the requirements of section 2034. Of the 49 project studies 

that underwent peer review, the majority were for ecosystem restoration 

projects, flood risk management projects, or deep draft navigation 

projects. (App. II lists the 49 project studies that underwent peer review 

since WRDA 2007 was passed, including information on project and 

 

19Earlier Corps guidance related to peer review was issued in August 2008 in Engineering 

Circular 1105-2-410 (EC 410), Review of Decision Documents, and in May 2005 in 

Engineering Circular 1105-2-408 (EC 408), Peer Review of Decision Documents. 

Corps Guidance for 

Implementing Peer Review

It Is Unclear How 

Many Peer Reviews 

Have Been Completed 

in Response to 

Section 2034 

Requirements 
 

Page 16 GAO-12-352 Army Corps of Engineers

study type, as well as the district, division, and planning center of 

expertise associated with each study.)

Moreover, it is not possible to determine how many project studies were 

required to undergo peer review in response to WRDA’s section 2034 

requirements because the Corps does not centrally track project studies 

and could not provide us a list of all project studies that fell within the 

scope of section 2034. Our review of relevant Corps documents for the 49 

project studies that underwent peer review, such as review plans and 

completed review reports, found that none of these documents specifies 

the authority under which peer reviews were conducted. Corps 

headquarters officials told us that the Corps does not make specific 

determinations as to whether a peer review is being conducted under 

section 2034 but instead focuses on ensuring that the peer review is 

being carried out in compliance with EC 209, which, in their view, 

complies with section 2034. These officials also told us that, to ensure the 

quality of Corps project studies, the agency may choose to conduct peer 

reviews under its other authorities, even if those peer reviews are not 

required by section 2034.

In February 2011, the Corps submitted its initial report to Congress in 

response to the requirement in section 2034 of WRDA 2007 summarizing 

its experiences implementing the peer review process. In the report, the 

Corps noted that the 29 peer reviews that had been completed as of 

February 2011 followed the procedures described in agency and OMB 

guidance. The Corps report stated that, in its view, section 2034 

provisions reinforce and add further definition to the Corps’ process. 

Nevertheless, because the Corps did not distinguish which studies had 

been selected for peer review in accordance with section 2034, we 

believe that it did not provide Congress with the type of information 

required by section 2034 that would help congressional decision makers 

evaluate the trial program.

The 49 peer reviews conducted by the Corps since November 2007 

resulted in direct costs of about $9 million in contract costs and contract 

administration fees. In addition, Corps staff resources were also used to 

manage peer reviews, but these costs are not fully quantifiable. 

Furthermore, the addition of peer review to the Corps study process has 

resulted in indirect costs by altering project study schedules because of 

the additional time required to complete the peer review. In some cases 

where a peer review was not planned for during the early stages of the 

Completed Peer 

Reviews Have Cost 

Millions of Dollars in 

Direct and Indirect 

Costs 
 

Page 17 GAO-12-352 Army Corps of Engineers

study process, significant delays in the project studies have resulted from 

the addition of the peer review.

The 49 project studies for which the Corps completed peer reviews since 

November 2007 cost about $9 million in contract costs and contract 

administration fees to establish and manage the expert panels for these 

reviews. In addition, Corps staff resources were also used to manage the 

peer reviews, but these costs are not fully quantifiable. The Corps used 

the services of three contractors to manage the peer review process: the 

nonprofit Battelle Memorial Institute, which managed 46 of the reviews; 

the nonprofit Noblis, which managed two; and NAS, which managed one. 

The cost per panel varied considerably. For example, the contracts 

managed by Battelle cost from about $76,000 to $484,000 for studies that 

underwent peer review,20 but the panel managed by NAS cost over 

$500,000. (See app. II for information on the contract costs for each of 

the 49 peer reviews.) In addition to the $9 million in contractor costs, the 

Corps incurred about $109,000 for the administration of the contracts for 

the 49 peer reviews. Specifically, the Corps used two different entities—

the Institute for Water Resources and the Army Research Office—to 

administer these contracts, and both of these entities charged 

administration fees.21

Corps staff resources were also used to manage the peer review 

process—including developing the scope of work for reviews, 

coordinating establishment of contracts, reviewing contract proposals, 

and responding to panel comments that the Corps received during a peer 

review process. Corps district, division, headquarters, and planningcenter-of-expertise staff also spent time managing the peer review 

process. The total of these costs, however, is not fully quantifiable across 

These fees ranged from no fee to 3 percent of the 

contract cost.

 

20In some cases, individual project studies underwent multiple phases of peer review 

because of revisions made to the study after the initial peer review report, and peer review 

was completed under multiple reports and contracts. This dollar range does not include 

one contract that cost about $677,000 because the cost covered peer review for five 

studies.

21The Institute for Water Resources is the Corps center of expertise for integrated water 

resources management. The U.S. Army Research Laboratory's Army Research Office is 

the Army's research agency in the engineering, physical, information, and life sciences. 

These entities’ contracting offices administer the Corps’ contracts for establishing and 

managing peer review panels. 

Direct Costs Related to 

Peer Reviews Have Totaled 

Over $9 Million
 

Page 18 GAO-12-352 Army Corps of Engineers

all Corps districts because not all districts track district staff time spent on 

peer reviews. For those districts that did track or could estimate district 

staff time spent on peer review-related activities, we found the following 

examples of the staff resources that may have been dedicated to 

managing and responding to peer review activities:

• The Green Bay Dredged Material Management Plan peer review cost 

about $101,000 in staff time, according to data provided by district 

officials. But district officials involved in this peer review said that the 

cost in terms of staff time may have been higher than typical because 

this peer review was the first conducted for a study in that district.

• The Chatfield Water Reallocation Study peer review cost about 

$20,000 in staff time as of December 2011, but the Corps response 

has not been completed for this peer review, and additional staff time 

could be involved.

• For the Boston Harbor study, district officials estimated that costs 

totaled about $77,000 for district staff time, agency technical review 

team labor, and contractor fees for assisting the district with 

responding to peer review panel comments.

• For the American River study, district officials estimated that costs 

came to about $40,000 for district staff time.

Similar to district staff time, other staff time involved in managing the peer 

review process, including headquarters, division, and some planning 

centers of expertise time, is also not always tracked and therefore not 

fully quantifiable: some of these positions are funded with general funds 

and not project-specific funds, according to Corps officials we spoke with. 

We did find two examples, however, where planning centers of expertise 

staff time devoted to peer review related activities was tracked. In these 

two instances, the cost of planning centers of expertise staff time devoted 

to peer review activities amounted to about $12,000 for the peer review of 

the Boston Harbor study and about $32,000 for the American River study.

The addition of peer review to the Corps study process has also had an 

indirect cost because it has affected project study schedules. Planning 

centers of expertise and district officials estimate that obtaining the 

contract and executing the peer review generally take about a year. The 

breakdown for the peer review process, according to some of these 

officials, is about 3 months to initiate the contract; 3 to 6 months for the 

Addition of Peer Review 

Has Affected Project Study 

Schedules 
 

Page 19 GAO-12-352 Army Corps of Engineers

review to be completed; and an additional 3 months to close out the 

review, which involves responding to and receiving clarification on panel 

comments. Some of these processes occur concurrently with other 

aspects of the project study, but some parts of the peer review process, 

such as responding to panel comments, may add time to the study 

schedule. According to Corps officials, the addition of peer review to the 

project schedule adds steps to the review process and takes time away 

from other projects. In addition, according to several Corps officials, some 

project studies have been delayed because the district did not allocate 

funding for the peer review and therefore had to wait until additional 

funding was available. In some cases, this delay added significant time to 

the schedule. In contrast, according to some Corps division and planningcenter-of-expertise officials, when the project manager had built in time 

for the peer review and had identified funding for it early, the peer review 

process had much less of an impact on the overall project study 

schedule.

Local sponsors are also concerned about the impact that this additional 

time is adding to project studies, according to Corps officials and local 

sponsors we spoke with. Their concern arises largely because local 

sponsors share the cost of the Corps study and depend on its timely 

completion. District officials told us that because of the cost-sharing 

requirements and the current economic environment demand is greater 

from local sponsors for the Corps to finish studies quickly and keep costs 

down. Two local sponsors told us that delays negatively affect local 

sponsors because they can lose business if a project is not completed in 

a timely manner. Similarly, sponsors are accountable to their own local 

governments or state legislatures, and additional delays or time required 

for peer review can create challenges in getting continued support for a 

project. For example, in the case of the Green Bay Dredged Material 

Management Plan, Corps officials told us that peer review increased the 

cost of the project and caused a 5- to 6-month delay at a time when the 

local sponsor was attempting to acquire grant money contingent on 

completion of the dredged material management plan.
 

Page 20 GAO-12-352 Army Corps of Engineers

The Corps’ process for determining whether a project study is subject to 

peer review is more expansive than section 2034 requirements because it 

uses broader criteria; this has resulted in peer reviews of studies that are 

outside the scope of section 2034. In addition, the process the Corps 

uses does not include the flexibility provided in section 2034 to exclude 

certain project studies from peer review. Moreover, some studies are 

undergoing peer review that do not warrant it, according to some Corps 

officials we spoke with.

The Corps’ process for determining whether a project study is subject to 

peer review uses criteria that are broader than the requirements of 

section 2034. As table 1 shows, the Corps relies on its guidance outlined 

in EC 209 when selecting project studies for peer review, and this 

guidance extends beyond section 2034 requirements.

Table 1: Corps EC 209 Guidance on Conducting Peer Review

Basis

Criteria for conducting peer review

Section 

2034

OMB 

bulletin Othera

Significant threat to human lifeb X 

The total project cost is greater than $45 million X 

Request by the governor of an affected state X 

Request by the head of a federal or state agency X 

Significant public dispute as to size, nature, or 

effects of the project

Xc 

Significant public dispute as to the economic or 

environmental cost or benefit of the project Xc

Information is based on novel methods X 

Presents complex challenges for interpretation X 

Contains precedent-setting methods or models X 

Presents conclusions that are likely to change 

prevailing practices

X 

Any other circumstance where the Chief of 

Engineers determines peer review is warranted

X 

Source: GAO analysis of Corps guidance.

a

For example, Corps officials noted that the criteria for peer review reflect the agency’s general 

authority to conduct peer review in association with its responsibility to produce quality projects.

b

Section 2035 of WRDA 2007 requires that certain projects in design or under construction undergo 

independent peer review in cases where the Chief of Engineers determines a safety assurance 

review is necessary, such as where failure of a project would pose a significant threat to human life.

The Corps’ Process 

for Initiating Peer 

Review Is More 

Expansive and Less 

Flexible Than Section 

2034 Requirements

The Corps’ Process for 

Determining the Need for 

Peer Review Is More 

Expansive Than Section 

2034 Requirements
 

Page 21 GAO-12-352 Army Corps of Engineers

c

Section 2034 criteria include that the Chief of Engineers determines that the study is controversial 

considering whether (a) significant public dispute exists as to the size, nature, or effects of the project; 

or (b) significant public dispute exists as to the economic or environmental costs or benefits of the 

project.

Consequently, the Corps has selected some studies for review based in 

part on criteria included in EC 209 that are not required by section 2034, 

and others that are outside the scope of section 2034. For example, 

according to our analysis of review plans for 44 peer reviewed project 

studies,22 over one-third identified criteria that related to both section 

2034 and other authorities. In addition, the Corps process for determining 

whether a peer review is required has resulted in 30 project studies 

undergoing peer review that were outside the time parameters identified 

in section 2034. Based on our analysis of the characteristics of these 

studies, the Corps’ process was applied to all studies and reports 

regardless of when they were initiated,23 whereas section 2034 applies to 

project studies initiated from November 2005 through November 2014. 

Specifically, section 2034 applies to (1) project studies initiated from 

November 2005 through November 2007 and for which the array of 

alternatives had not been identified, and (2) project studies initiated from 

November 2007 through November 2014. As a result, over half (30 of 49) 

of the peer reviews conducted since the enactment of WRDA 2007 were 

for project studies that did not fall under the scope of section 2034 

because the studies were initiated before November 2005.24

Another reason the Corps’ process for selecting studies for peer review is 

more expansive than the scope of section 2034 is that Corps’ guidance 

 

22Of the 49 peer reviews that the Corps conducted, 4 review plans for studies that 

underwent peer review were not updated to reflect the decision to conduct peer review 

and indicate that peer review was not required, and 1 project study that underwent peer 

review did not have a review plan completed, so our analysis was based on 44 review 

plans.

23EC 209 includes an exception for cases where the final decision document package had 

been forwarded to headquarters before August 22, 2008.

24We requested a list of studies that came within the scope of section 2034 but the Corps 

does not track this information, however, and it requested this information from each of the 

districts. Through this effort, the Corps identified 134 studies with the caveat that the list

was not complete and 18 of the 49 studies that underwent peer review were included on 

this list. In addition, we made our initial request for this list of studies in June and did not 

receive it until mid-December. Given the delay and the list’s incompleteness, we did not 

evaluate the other 116 studies on the list for whether peer review is required or planned. 

We note it is likely that some of the 116 studies may not yet have review plans nor 

decisions on whether peer review is to be conducted. 
 

Page 22 GAO-12-352 Army Corps of Engineers

does not clearly define “project study.” The guidance refers to a wide 

range of project studies, decision documents, and work products that may 

be subject to peer review, whereas section 2034 defines a project study 

subject to peer review as a feasibility or reevaluation study, including the 

EIS, or any other study associated with the modification of a water 

resources project that includes an EIS. According to our analysis of the 

49 studies that underwent peer review, some of these studies did not fit 

this definition. Specifically, 34 of the 49 studies that underwent peer 

review were feasibility or reevaluation studies which are project studies as 

defined by section 2034 requirements, 8 were other kinds of reports that 

included an EIS and therefore may have been subject to section 2034 

requirements, and 7 were neither feasibility nor reevaluation studies and 

did not include an EIS and therefore did not fit the definition of a project 

study subject to peer review under section 2034. For more details on 

each of the studies that underwent peer review, see appendix II.

The Corps’ process for determining whether peer review is required for 

project studies does not include the flexibility provided in section 2034 to 

exclude certain project studies from otherwise mandatory peer review. EC 

209 states that most studies should undergo peer review, and the Corps’ 

process requires that for any decision document to forgo a peer review, 

an exclusion must be requested and approved by headquarters. In 

addition, guidance provided to Corps staff on how to implement EC 209 

discourages requests for exclusions, noting that time should not be 

wasted shopping around for exclusion requests. Furthermore, agency 

guidance and Corps headquarters officials, including the Director of Civil 

Works, highlight the value and importance of peer review in achieving the 

agency’s mission, noting that an extra set of eyes is beneficial. In 

addition, Corps headquarters officials told us, and agency guidance 

highlights language from the WRDA 2007 conference report, that 

“[s]ection 2034 permits the Chief of Engineers to exclude a very limited 

number of project studies from independent peer review.”25

 

25H.R. Conf. Rep. 110-280, at 267. 

We believe, 

however, that the Corps has misconstrued this statement and overstated 

its significance. This statement is part of the explanation of the exclusion 

paragraph (a)(5), and does not apply to the provision as a whole; 

therefore, this statement pertains to how many studies for which peer 

The Corps Does Not Use 

the Flexibility in Section 

2034 to Exclude Studies 

from Review
 

Page 23 GAO-12-352 Army Corps of Engineers

review is mandatory would be eligible for exclusion.26

In reviewing the exclusion requests that it receives, Corps headquarters 

determines whether the studies meet any of the mandatory requirements 

in EC 209 for undergoing peer review. Specifically, the Corps reviews 

whether the project has a cost estimate of greater than $45 million, 

represents a threat to health and safety, is controversial, and has had a 

request for peer review from a governor or the head of a federal or state 

agency. If studies do not meet any of these criteria, the Corps generally 

approves the study for exclusion from peer review. From our review of 50 

studies that had requested exclusion from peer review between 2009 and 

2011, we found that the Corps had granted an exclusion for 37 studies 

because they did not meet any of the criteria in EC 209 for studies that 

must undergo peer review except for one study, which did not fit the 

definition of a project study in section 2034.

Furthermore, 

another relevant statement in a House committee report on WRDA 2007 

suggests that 26 studies over 7 years, or about 4 studies per year, would 

be expected to be subject to peer review. Additionally, the 2002 NAS 

study—which is prominently mentioned throughout the subsequent 

legislative history of WRDA 2007—states that not all Corps water 

resources project planning studies will require external, independent 

review, but the Corps should institute external review for studies that are 

expensive, that will affect a large area, that are highly controversial, or 

that involve high levels of risk.

27

Under section 2034, however, the Corps also has the flexibility to exclude 

studies from peer review that exceed the $45 million threshold if they: do 

not have an EIS; are not controversial; are expected to have negligible 

adverse impacts on scarce or unique cultural, historic, or tribal resources; 

have no substantial impacts on fish and wildlife species or their habitats; 

and have no more than negligible impacts on threatened or endangered 

species or their critical habitat. Similarly, under section 2034, the Corps 

 

26H. Rep. 110-80, at 132 (discussing H.R. 1495 § 2037) (Mar. 29, 2007). 

27This study exceeded the threshold of $45 million, but Corps headquarters’ approval of 

the request for exclusion said it was excluded because it did not meet any of the other 

mandatory criteria for peer review. It was not a feasibility study or a reevaluation study and 

did not include an EIS. Of the 50 studies that went through the exclusion process as of 

November 2011, the Corps excluded 37. The Corps decided not to exclude 7 of these 

studies and determined that 4 did not need to request an exclusion. A decision is pending 

for 1 study and has been deferred for another study.
 

Page 24 GAO-12-352 Army Corps of Engineers

may exclude studies from peer review that involve (1) only the 

rehabilitation or replacement of existing hydropower turbines, lock 

structures, or flood control gates within the same footprint and for the 

same purpose as an existing water resources project; (2) an activity for 

which there is ample experience within the Corps and industry to treat the 

activity as being routine; and (3) minimal risk to human life and safety. 

Nevertheless, according to our analysis of exclusion request documents 

and headquarters’ responses to these requests, as of November 2011, 

the Corps had not granted an exclusion based on any of the flexibilities 

included in section 2034.

Several Corps officials expressed concerns about the Corps exclusion 

process. Specifically, some officials told us that they were concerned 

about the cost and time involved and said that the exclusion of projects 

that do not meet any of the mandatory criteria should be delegated to the 

division offices. In their opinion such delegation would help streamline the 

process. Moreover, some of the studies that underwent or are currently 

undergoing peer review did not warrant it, according to some Corps 

officials we spoke with. Specifically, we found the following examples of 

studies that may not have warranted a peer review:

• Two dredged material management plans underwent peer review. For 

example, the Green Bay Dredged Material Management Plan 

underwent peer review but is not a project study as defined by section 

2034. Officials we spoke with said that such plans should not 

generally require peer review because any significant impacts would 

be addressed under NEPA and because the Corps has sufficient 

expertise in the area of dredging.

• The Chacon Creek study in southern Texas underwent peer review 

but should not have, according to some Corps officials we spoke with. 

This study was for a project that would remove houses from a 

floodplain, but officials said it should not require peer review because 

there are no structural components and it did not exceed the $45 

million threshold. Corps headquarters denied the request for exclusion 

and stated that flood studies warrant peer review because of the 

nature of the hazard and the need to assess the extent and treatment 

of risk. Headquarters officials highlighted the importance of assessing 

and addressing such risks in light of Hurricane Katrina and said that 

flood studies such as Chacon Creek require peer review because of 

the importance of assessing and decreasing risks associated with 

flooding.
 

Page 25 GAO-12-352 Army Corps of Engineers

• The Yuba River General Reevaluation study is undergoing peer 

review but does not warrant it according to some Corps officials. 

District officials told us that the study does not warrant peer review 

because the construction work involved has already been completed, 

and the purpose of the study is to determine the amount the local 

sponsor should be reimbursed by the Corps.

Officials from several districts, divisions, and planning centers of expertise 

we spoke with told us that peer review should be focused on larger and 

more complex or controversial projects and should not be the default 

approach. Two Corps officials described the Corps’ peer review policy as 

a one-size-fits-all approach, and one of these officials stated that it is 

inflexible and risk averse.

The Corps has a process to review general information on contractors’ 

conflicts of interest and independence during its contractor selection 

process, but it does not have a process for reviewing project-specific 

information provided by contractors to determine if conflicts of interest 

and independence exist at the project level. The Corps’ contractors, 

however, have a process for reviewing the appropriate information related 

to the conflicts of interest and independence of the experts selected for 

peer review panels at the project level.

For its initial peer reviews, the Corps relied on Battelle to establish and 

manage the peer review panels. From 2007 to 2009, Battelle managed 15 

independent peer reviews for the Corps. The Corps had identified Battelle 

as a potential contractor for managing its peer review panels as early as 

August 2007, when WRDA 2007 was being considered. To ensure that 

Battelle could meet the section 2034 independence requirements, 

according to Corps officials, the Director of Civil Works and the Chief of 

Planning and Policy held discussions with Battelle, and officials from the 

Corps’ Institute for Water Resources met with Battelle to discuss 

Battelle’s existing review process and the independent peer review 

requirements of WRDA 2007. Battelle informed the Corps that it met all 

WRDA 2007 requirements for an eligible organization, and Battelle 

identified its existing contract with the Army Research Office as a vehicle 

for employing Battelle to establish and manage peer review panels under 

section 2034.

During that time, NAS also conducted one independent peer review for 

the Corps on the Louisiana Coastal Protection and Restoration Program, 

which charged the Corps with developing a full range of flood control, 

Gaps Exist in the 

Corps’ Process for 

Screening Its 

Contractors Who Are 

Responsible for 

Selecting Experts for 

Peer Review Panels
 

Page 26 GAO-12-352 Army Corps of Engineers

coastal restoration, and hurricane protection measures for South 

Louisiana. According to Corps planning-center-of-expertise officials, 

because of the extensive scope and breadth of the project, NAS was 

chosen instead of Battelle to conduct that peer review. But Corps 

headquarters and planning-center-of-expertise officials told us that, over 

the course of NAS’ review, they realized that NAS would not be the 

appropriate organization for reviewing individual projects studies because 

its process was too time-consuming and expensive. A member of that 

NAS peer review panel also told us that while he would recommend NAS 

review for larger projects, in his opinion NAS might not be the appropriate 

organization for reviewing smaller Corps projects.

In 2009, the Corps sought additional contractors to establish and manage 

peer review panels and began its contractor selection process by putting 

out a request for proposals. This solicitation included as contract 

requirements the section 2034 criteria that the organizations establishing 

and managing peer review panels be independent and free from conflicts 

of interest. The Corps received six proposals, including one from Battelle, 

and each of these proposals was then evaluated by a three-person 

review panel. The panel chairperson told us that the section 2034 criteria 

that eligible organizations be independent and free from conflicts of 

interest were considered as minimum qualifications for screening and 

selection.28 As a result of this process, the Corps awarded a contract to 

Battelle—this contract was in addition to the existing contract Battelle 

already had with the Corps through the Army Research Office—and one 

to Noblis.29

Although the Corps’ contractor screening and selection process identifies 

general contractor independence and areas of conflicts of interest, the 

Corps does not have a process for reviewing the selected contractors’ 

The Corps determines which of the two contractors it will use 

to manage individual peer reviews on the basis of the contractors’ 

responses to specific project study scope of work requests, described 

below.

 

28The summary evaluation of contract proposals does not discuss the section 2034 

eligibility criteria; although, the proposals from both Battelle and Noblis address the criteria 

and state that those organizations meet all section 2034 criteria for eligible organizations. 

29Initially, one contract was awarded to Battelle and one to a second organization whose 

contract was terminated 6 months later because of internal management issues at that 

organization. In November 2010, on the basis of Noblis’s proposal from the original panel 

evaluation, the Corps selected Noblis to replace this organization. 
 

Page 27 GAO-12-352 Army Corps of Engineers

project-specific independence and freedom from conflicts of interest. For 

each project study undergoing a peer review, the Corps sends both 

contractors a “scope of work” document, which describes the project 

study and lists the required contractor qualifications. These qualifications 

include independence and freedom from conflicts of interest related to the 

specific project study being reviewed. In response, the contractors send 

the Corps their proposals for conducting the peer review, which generally 

include statements that they are independent and free from projectspecific conflicts.30

• The Corps’ planning centers of expertise are expected to review the 

contractors’ overall proposals, but the Corps does not require the 

centers to ensure that contractors’ statements of independence within 

the proposals are reviewed and corroborated for each individual 

project. Although planning-center-of-expertise officials told us that 

they review the overall proposals, some of these officials also stated 

that they did not believe that the statements required review because 

Corps headquarters had already prescreened the contractors during 

the initial contractor screening and selection process. Furthermore, 

the Corps has not provided any guidance to the planning centers of 

expertise or other Corps offices that specifies how those officials 

should review the contractors’ project-specific statements at the 

proposal stage and ensure that they are accurate and that the 

contractors are in fact independent and free from conflicts of interest. 

Absent such guidance, the Corps cannot ensure that its contractors 

are independent and free from conflicts of interest at the project level.

Nevertheless, we identified a number of weaknesses 

in the Corps’ approach for reviewing and corroborating this information, 

including the following:

• The Corps neither conducts any internal conflicts-of-interest checks 

nor asks contractors for documentation about potential conflicts of 

interest so that it can determine whether a conflict exists; rather, the 

Corps allows the contractors to make that determination on their own. 

As a result, if the contractors do not provide this information to the 

Corps, the agency does not have a process for otherwise obtaining 

this information.

 

30Prior to the selection of Noblis as a contractor in 2010, the scope of work went solely to 

Battelle. Battelle was still required to provide the Corps with a proposal describing how it 

would conduct the peer review and responding to the contractor requirements.
 

Page 28 GAO-12-352 Army Corps of Engineers

Unlike the Corps’ review of contractor independence, the Corps’ 

contractors do solicit and review information on panel members’ 

independence and conflicts of interest at the project level. The contractors 

gather information about prospective panel members using screening 

questions developed from the scope of work for each peer review. These 

questions cover issues described in the NAS policy on committee 

composition and conflicts of interest, such as financial and employment 

interests and public statements and positions. The peer review reports 

from both Battelle and Noblis state that they follow both the OMB 

guidance on peer review and the NAS policy when selecting panel 

members. According to contractors and Corps officials, district and 

planning-center-of-expertise officials review the contractors’ screening 

questions, as well as the resumes of selected experts, and can provide 

the contractors with additional information about potential conflicts of 

interest, such as previous work a particular expert may have done for the 

Corps. Corps officials told us that the contractors follow up on such 

information where appropriate, but the contractors and a Corps official we 

spoke to said that it is the contractors who ultimately select the panel 

members and ensure their independence.

The Corps has adopted and incorporated most of the peer review 

recommendations it has received. Adoption of these recommendations 

has resulted in some technical improvements to project study reports but 

generally has not changed the Corps’ decisions in selecting preferred 

project designs. According to some Corps officials we spoke with, this is 

the result of the review occurring too late in the process to effect a 

change in decision making.

Of the 49 project studies that have undergone peer review, the Corps has 

provided a final written response for 17. The Corps has adopted 231 of 

274 recommendations, partially adopted 31, and rejected 12 for these 17 

peer reviews. Several Corps district officials told us that they make every 

effort not to reject peer review recommendations and that headquarters 

has directed them to adopt recommendations whenever possible. In fact, 

some district officials told us that they felt pressure from headquarters to 

adopt peer review recommendations even when the recommendations 

The Corps Has 

Adopted Most Peer

Review 

Recommendations, 

Resulting in Technical 

Improvements but 

Generally No Changes 

in Project Decisions

The Corps Has Adopted 

and Incorporated Most 

Peer Review 

Recommendations, 

Resulting in Technical 

Improvements
 

Page 29 GAO-12-352 Army Corps of Engineers

would not affect the study outcome and would be burdensome to 

implement.

The Corps’ adoption of peer review recommendations has improved the 

technical quality of its project study reports, according to Corps officials 

and panel members we spoke with. Corps officials complimented the 

quality and technical competence of panel members and stated that the 

panels’ recommendations have been helpful in clarifying and 

strengthening the arguments presented in the studies. Most of the 

recommendations either requested that the Corps add to or clarify the 

study report or stated that the study report did not sufficiently address 

certain issues. The Corps addressed these issues in almost all instances 

(193 of 201 recommendations) within its written responses to completed 

peer review reports.31

A smaller number of recommendations addressed the underlying 

assumptions and inputs to the project studies’ economic, engineering, 

and environmental analyses.32

 

31Forty-nine studies underwent peer review, but 3 of these studies resulted in 2 peer 

review reports, so the total number of reports was 52. For the 52 completed peer review 

reports, panel members made a total of 910 recommendations. Most of the 

recommendations (672 of 910) either requested that the Corps add to or clarify the study 

report or stated that the study report did not sufficiently address certain issues.

We identified four instances in which the 

Corps revised portions of its analysis on the basis of these kinds of 

recommendations. In none of these cases did the Corps indicate that the 

revised analyses would change the study decisions. In one case, 

according to Corps documents, the revised analysis served to strengthen 

arguments in favor of its recommended plan. In response to a 

recommendation concerning an environmental analysis from the peer 

review of the Mid-Chesapeake Bay Islands Ecosystem Restoration 

project study, the Corps conducted additional analyses to justify its 

calculations of environmental benefits. The Corps reported that the 

additional analyses led to the determination that the selected plan was 

appropriate but that by considering the ecosystem impacts of the project 

in a more detailed fashion, justification of the recommended plan was 

strengthened.

32Specifically, 34 out of the total 910 peer review recommendations indicated a problem 

with the economic analysis, 24 indicated a problem with the engineering analysis, and 19

indicated a problem with the environmental analysis.
 

Page 30 GAO-12-352 Army Corps of Engineers

Nevertheless, despite these technical improvements, some Corps officials 

have questioned the benefit of peer review, given the significant amount 

of time that district staff have to spend managing the process and 

responding to recommendations. The process for responding to 

recommendations begins with district officials drafting a written response, 

which they provide to the panel. The Corps’ response to the peer review 

recommendations includes a detailed description of the steps that the 

Corps has taken or will take to incorporate the recommendations into the 

project study. The contractor then convenes a teleconference at which 

district officials discuss the draft response with panel members. After this 

discussion, the panel members provide written feedback—“backcheck 

responses”—to the Corps stating whether they agree with the district’s 

response.33 The district then finalizes its response to the 

recommendations and forwards the response to its division office. After its 

review, the division forwards the response to headquarters, where the 

response is finalized. The final written response is generally published at 

the same time as the final decision document for the project study. The 

time between completion of the peer review report and public availability 

of Corps’ written response therefore varies greatly depending on the 

individual project. In one case it was 3 months, while in other cases peer 

review reports have been completed for more than 3 years without a final 

response from the Corps having been made public.

Corps officials we spoke with told us that peer review recommendations 

have generally had no impact on the Corps’ decision making process. 

These Corps officials were not aware of any project studies for which the 

study outcome changed as a result of peer review. Corps headquarters 

officials told us that one reason for the lack of impact of peer review on 

decision-making is because the Corps’ internal review process is 

identifying the same issues as peer review. Another reason cited by 

Corps officials for the lack of impact on decision making is the fact that 

peer review is occurring at the end of the study process. Peer review 

generally occurs concurrently with the public comment period for the draft 

study report, which comes after the preferred design has been selected. 

As a result, some recommendations about alternatives may not have 

 

33The backcheck responses are not part of the panel’s final peer review report which 

contains only the panel’s recommendations. The backcheck responses are given after the 

final peer review report is submitted to ensure that the panel’s opinion and objectivity are 

not influenced by the Corps.

For Most Studies, Peer 

Review Occurs Too Late in 

the Study Process to Affect 

Decision Making
 

Page 31 GAO-12-352 Army Corps of Engineers

been implemented because the decision on the preferred design had 

already been made. Selecting a different preferred design at that stage 

would require the Corps to revisit an already completed selection analysis 

and decision. For example, in the peer review report on the Cedar RiverCedar Rapids Iowa Flood Risk Management project study, the review 

panel recommended that the Corps further investigate one of the nonselected design alternatives, because panel members felt that the 

alternative might achieve project objectives better than the preferred 

design. The Corps, however, had already selected its design and decided 

to proceed. The Corps did not adopt this recommendation, stating that it 

believed its analysis of alternatives was sound and that there was no 

reasonable expectation that a more detailed analysis of the alternative 

would result in finding that it had greater net economic benefits than the 

preferred design.

In contrast, when the Corps has conducted a peer review earlier in the 

process, opportunities have arisen for positive impacts on a study 

decision. For example, the American River Common Features project 

study peer review was conducted early in the study process. According to 

Corps division and planning-center-of-expertise officials, they conducted 

peer review early to obtain external input on defining the problem and to

inform decision-making due to the complexity of the project. As a result, 

the peer review began before the alternative formulation briefing, when 

the without-project conditions were being identified. By employing this 

approach, the Corps received feedback from the review panel before 

selecting the preferred design. The panel’s recommendations included 

three suggested changes to the Corps’ analyses and model calibrations, 

which the Corps had time to incorporate before conducting the alternative 

analysis and selection. According to the contractor that managed the peer 

review, the panel members involved in the American River Common 

Features peer review also found the timing of the review to be beneficial 

and suggested that the Corps conduct peer review earlier for other project 

studies.

The timing of peer review was also addressed in the 2002 NAS study on 

peer review. NAS recommended that the Corps initiate peer reviews early 

enough in the study process so that the review results could be 

meaningfully incorporated into the study or project design and stated that 

conducting peer review before selecting a recommended plan is essential 

if the Corps is to benefit from the review. Corps officials nevertheless told 

us that they have generally chosen to conduct peer reviews later in the 

process to minimize effects on project study schedules. Corps 

headquarters officials noted that, for many studies, peer review occurred 
 

Page 32 GAO-12-352 Army Corps of Engineers

late in the process because the studies were under way at the time the 

Corps began requiring peer review. These officials also noted that it 

would be challenging to assemble a peer review panel to conduct a 

review early in the study process and retain the same panel to complete 

this review at the end of the study. Furthermore, Corps headquarters 

officials noted that a further challenge is implementing section 2033 

WRDA requirements along with section 2034. Section 2033 generally 

requires the Corps to complete feasibility studies within 2 years. 

According to Corps officials, there is tension between these requirements 

and it may be challenging to include peer review throughout the study 

process without altering project study schedules.

Section 2034 established a trial to look at the cost and impact of 

conducting peer review for controversial and costly projects over a 7-year 

period. After the trial period, based on information provided by the Corps, 

Congress could reconsider whether to retain or revise section 2034 or 

allow it to lapse. Because the Corps generally does not specify the 

authority under which peer review was conducted, however, it has not 

provided Congress with the information needed to evaluate the merits of 

the section 2034 requirements. In addition, the Corps’ implementation of 

peer review has not focused on the larger, more complex, and 

controversial projects that were contemplated when section 2034 was 

enacted and as recommended by NAS a decade ago. As a result, project 

studies are being selected to undergo peer review that may not be 

warranted and may thereby be increasing project costs and schedules 

needlessly. Further, essential to the integrity of the peer review process is 

the assurance that the Corps has effective processes not only to ensure 

overall contractor independence and freedom from conflicts of interest but 

also to ensure project-level independence and freedom from conflicts of 

interest. The Corps’ current process, however, has a number of 

weaknesses with respect to ensuring no conflicts of interest exist at the 

project level. Finally, with peer review generally occurring late in the 

Corps’ project study process, peer review serves more to strengthen the 

Corps’ presentation of its decisions than to influence its decision making. 

This effect runs counter to what NAS recommended in 2002, that 

realizing the benefits of peer review requires the results to be used as 

inputs in the decision-making process. By choosing to apply peer review 

late in the project study process, the Corps has effectively chosen to not 

use the results of peer review to enhance its decision-making process 

and ensure selection of the most effective project alternatives.

Conclusions 
 

Page 33 GAO-12-352 Army Corps of Engineers

We recommend that the Secretary of Defense direct the Chief of 

Engineers and Commanding General of the U.S. Army Corps of 

Engineers to take the following three actions:

To facilitate congressional evaluation of the 7-year trial period outlined in 

section 2034, the Corps should:

• Identify for each past and future peer review the specific statutory 

authority under which the peer review was conducted and the criteria 

triggering peer review under the Corps’ civil works review policy.

To better reflect section 2034 and provide more effective stewardship of 

public resources and ensure efficient and effective operations, the Corps 

should:

• Revise the criteria in the Corps’ process for conducting peer review to 

focus on larger, more complex, and controversial projects; to 

encourage peer review to occur earlier in the study process; and to 

include exclusions to peer review that align with section 2034.

• Develop a documented process to ensure that contractors are 

independent and free from conflicts of interest on a project-specific 

basis.

We provided a draft of this report to the Department of Defense for review 

and comment. In its written comments, reprinted in appendix III, the 

department generally concurred with our recommendations. Specifically, 

in response to our first recommendation, the department agreed that the 

Corps should, and stated that it will, identify for each past and future peer 

review the specific statutory authority under which the peer review was 

conducted and the criteria triggering peer review under the Corps’ civil 

works review policy. In response to our second recommendation, the 

department partially concurred, stating that it agreed that peer review 

should be focused on studies that will significantly benefit from peer 

review and that initiating reviews early is advantageous. Nevertheless, 

the department noted that early involvement must be balanced with 

having sufficient data and analysis available for review and also 

highlighted work under way at the agency to overhaul its planning 

processes, which includes efforts to better align product reviews for 

greater effectiveness. In response to our third recommendation, the 

department agreed that the Corps should develop a documented process 

to ensure that contractors are independent and free from conflicts of 

interest on a project-specific basis.

Recommendations for 

Executive Action

Agency Comments 

and Our Evaluation
 

Page 34 GAO-12-352 Army Corps of Engineers

Although the department generally concurred with our recommendations, 

it disagreed with our report’s finding that the Corps’ process does not use 

the flexibility provided in section 2034, and it disagreed that some studies 

have undergone review that did not warrant it. The department stated that 

the Corps has carefully deliberated in support of the agency decision to 

conduct peer review on the three studies noted in our report and also 

stated that the Corps stands by all of its decisions to date to grant or deny 

exclusions from peer review. Nevertheless, the department stated that as 

part of the Corps’ ongoing review of the civil works review policy, it will 

assess the effectiveness of its criteria and how the criteria are applied to 

determine which studies should be considered for exclusion. In addition, 

the department expressed concern about the level of weight given in the 

report to anecdotal remarks from field-level officials, who in the 

department’s opinion may not have had the benefit of the corporate vision 

supporting the Army Civil Works Program. We disagree with the 

department’s characterization of our methodology. As clearly described in 

the scope and methodology section of this report, we interviewed officials 

who had a corporate-level perspective, as well as those who had a 

project-level perspective. Specifically, to obtain a corporate-level view, we 

interviewed senior level officials from Corps headquarters, the Institute for 

Water Resources, and the planning centers of expertise involved in 

managing the peer reviews. In addition, to get a project-level perspective 

and to assess the impact of peer review on division and district offices, 

we interviewed officials in all of the Corps’ eight divisions, and from 10 

geographically dispersed Corps districts that had conducted studies that 

underwent peer review. We also interviewed the three contractors and 

selected peer review panel members and local sponsors of Corps civil 

works projects. We believe that the report provides a balanced 

perspective from both the headquarters and field levels.

As agreed with your office, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies of this report to the 

appropriate congressional committees, the Secretary of Defense, the 

Chief of Engineers and Commanding General of the U.S. Army Corps of 

Engineers, and other interested parties. This report will be available at no 

charge on the GAO website at http://www.gao.gov. 
 

Page 35 GAO-12-352 Army Corps of Engineers

If you or your staff members have any questions about this report, please 

contact me at (202) 512-3841 or mittala@gao.gov. Contact points for our 

Offices of Congressional Relations and Public Affairs may be found on 

the last page of this report. GAO staff who made key contributions to this 

report are listed in appendix IV. 

Sincerely yours, 

Anu K. Mittal

Director, Natural Resources and Environment
Appendix I: Objectives, Scope, and 

Methodology

Page 36 GAO-12-352 Army Corps of Engineers

Our objectives for this work were to examine (1) the number of Corps 

project studies that have undergone independent peer review in response 

to section 2034 of the Water Resources and Development Act (WRDA) of 

2007, (2) the cost of these peer reviews, (3) the extent to which the U.S. 

Army Corps of Engineers’ (Corps) process for determining if a project 

study is subject to peer review is consistent with section 2034, (4) the 

process the Corps uses to ensure that the contractors it hires and the 

experts the contractors select to review project studies are independent 

and free from conflicts of interest, and (5) the extent to which 

recommendations from peer reviews have been incorporated into project 

studies. We focused on peer reviews for which reports had been 

completed since WRDA 2007 was enacted.

To address all of these objectives, we reviewed relevant legal 

requirements, policy guidance, review plans, and peer review reports for 

project studies that were subject to a peer review and for which a peer 

review report had been completed since WRDA 2007 was enacted. In 

addition, we selected a nongeneralizable sample of six peer reviews to 

examine in greater depth to better understand the costs associated with 

conducting these reviews, as well as the overall impact of the process on 

the timeline of the project study and the study outcome. We chose these 

reviews as illustrative examples and selected one from each of the Corps’ 

planning centers of expertise and at least one for each of the three 

contractors the Corps has used to manage peer reviews since enactment 

of WRDA 2007. Although the information derived from analysis of these 

case studies cannot be generalized, these examples provide valuable 

insights into the peer review process. We conducted semistructured 

interviews with officials from Corps headquarters, the planning centers of 

expertise involved in managing the peer reviews, all of the Corps’ eight 

divisions, and from 10 geographically dispersed Corps districts that had 

conducted studies that underwent peer review. We also conducted 

semistructured interviews with the three contractors, as well as selected 

peer review panel members and local sponsors of Corps civil works 

projects.

To determine the number of studies that have undergone peer review in 

response to section 2034 of WRDA, we reviewed all completed peer 

review reports, plus Corps reports and information on completed peer 

reviews. We reviewed information on completed peer reviews obtained 

from headquarters, the planning centers of expertise, divisions, and 

selected districts. We also reviewed information on completed peer 

reviews obtained from the contractors that established the peer review 

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology

Page 37 GAO-12-352 Army Corps of Engineers

panels and the entities the Corps used to administer these contracts: the 

Institute for Water Resources and the Army Research Office.

To determine the cost of these reviews, we reviewed contract award 

documents and information on contract costs from the contractors. 

Generally, we relied on the contract award amounts reported in the 

contracts to determine the cost of the contracts awarded for establishing 

review panels. For four contract awards, the contract work included 

establishing a peer review panel and additional work. For these awards, 

we therefore relied on information provided by the contractor on the 

portion of the contract cost that was for the peer review. For the contract 

award for peer review of a local sponsor-led study, we relied on 

information from the local sponsor and the contractor on the cost of the 

award. In addition, for the six case study peer reviews, we analyzed 

information on costs associated with managing the review process, 

including cost data and estimates provided by districts with regard to 

district and other staff time involved in peer review. In cases where we 

reported cost data including staff time associated with completing peer 

review, we asked knowledgeable officials about the data system and the 

quality of the data and determined that they were sufficiently reliable for 

our purposes. In cases where we reported estimates of these costs, we 

asked officials about how these estimates were developed and 

determined that they were sufficiently reliable for our purposes.

To determine the extent to which the Corps’ process for determining if a 

study is subject to peer review is consistent with section 2034, we 

analyzed the legal requirements and relevant policy guidance for 

determining when to conduct peer review. We also reviewed 

documentation on decisions to conduct peer review included in review 

plans and documents requesting exclusion from peer review. In addition, 

we reviewed information on the characteristics of studies that underwent 

peer review, including date initiated, whether an environmental impact 

statement was included, and the type of study. We identified this 

information in review plans, study drafts, signed Chief’s reports, and other 

Corps study-related documents; Corps officials from relevant districts or 

divisions reviewed this information.

To determine the process the Corps uses to ensure that the contractors it 

hires and the experts the contractors select are independent and free 

from conflicts of interest, we reviewed information on contractor selection 

obtained from Corps headquarters and the Institute for Water Resources. 

We also reviewed documentation from the contractors that outlined 

contractor and reviewer qualifications, as well as the National Academy of 
Appendix I: Objectives, Scope, and 

Methodology

Page 38 GAO-12-352 Army Corps of Engineers

Sciences Policy on Committee Composition and Balance and Conflicts of 

Interest.

To determine the extent to which peer review recommendations are 

incorporated into project studies, we reviewed information obtained from 

headquarters, the planning centers of expertise, divisions, and selected 

districts on how the Corps responds to peer review recommendations. 

We also reviewed all peer review recommendations contained in 

completed peer review reports, as well as all responses to peer review 

recommendations contained in the Corps’ published responses to the 

completed peer review reports.

We conducted this performance audit from April 2011 to March 2012 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.
Appendix II: Characteristics of Studies That 

Underwent Peer Review

Page 39 GAO-12-352 Army Corps of Engineers

Forty-six of the 49 peer reviews completed in table 2 below were 

conducted by Battelle Memorial Institute; Noblis completed the Green Bay 

Dredged Material Management Plan and the Wood River Levee System 

General Reevaluation Report peer reviews, and the National Academy of 

Sciences conducted the Louisiana Coastal Protection and Restoration 

peer review. As table 2 shows, the studies that underwent peer review 

came under the areas of ecosystem restoration (19 of 49), flood risk 

management (15 of 49), deep draft navigation (7 of 49), coastal storm 

damage reduction (5 of 49), inland navigation (2 of 49), and water 

management and reallocation (1 of 49). According to our analysis of 

Corps documents, 32 of the 49 studies included an environmental impact 

statement (EIS), 19 of 49 were initiated after November 2005, and 42 of 

49 had an estimated total project cost greater than $45 million.

Table 2: Characteristics of Studies That Underwent Peer Review

Dollars in thousands

Study name 

and type Corps district Corps division

Corps 

planning 

center of 

expertise

Date of 

peer 

review 

report(s)

Feasibility 

or 

reevaluation 

studya

Initiated 

after 

Nov. 

2005

Estimated 

cost 

exceeding 

$45 million EIS

Cost of 

peer 

review 

contract 

Alton to Gale 

Organized Levee 

Districts 

(Continuing, 

Deficiency 

Corrections) Letter 

Report

St. Louis Mississippi 

Valley

Flood risk 

management

Aug. 18, 

2010

√ $145

American River 

Draft Natomas 

Post-Authorization 

Change Report 

and Draft EISb

 

Sacramento South 

Pacific

Flood risk 

management

June 10, 

2009 & 

Sept. 16, 

2010

√ √ √ √ 458

Barataria Basin 

Barrier Shoreline 

Restoration Draft 

Construction 

Report and Draft 

EIS 

New Orleans Mississippi 

Valley

Ecosystem 

restoration

Oct. 3, 

2011

√ √ √ 141

Biscayne Bay 

Coastal Wetlands 

Project 

Implementation 

Report

Jacksonville South Atlantic Ecosystem 

restoration

Dec. 1, 

2009

√ √ √ 164

Appendix II: Characteristics of Studies That 

Underwent Peer Review
Appendix II: Characteristics of Studies That 

Underwent Peer Review

Page 40 GAO-12-352 Army Corps of Engineers

Dollars in thousands

Study name 

and type Corps district Corps division

Corps 

planning 

center of 

expertise

Date of 

peer 

review 

report(s)

Feasibility 

or 

reevaluation 

studya

Initiated 

after 

Nov. 

2005

Estimated 

cost 

exceeding 

$45 million EIS

Cost of 

peer 

review 

contract 

Boston Harbor 

Navigation 

Improvement Draft 

Feasibility Study 

and Draft 

Supplemental EIS

New England North Atlantic Deep draft 

navigation

June 3, 

2008

√ √ √ 159

Brevard County 

Mid-Reach 

Shoreline 

Protection Project 

Draft Integrated 

General Reevaluation Report 

and Supplemental 

EIS 

Jacksonville South Atlantic Coastal storm 

damage 

reduction

Dec. 9, 

2009

√ √ √ 173

Calcasieu River 

and Pass Dredged 

Material 

Management Plan 

and Supplemental 

EIS 

New Orleans Mississippi 

Valley

Deep draft 

navigation

Aug. 29, 

2008

√ √ 135

Cedar Rapids 

Flood Risk 

Management 

Feasibility Study 

with Integrated 

Environmental 

Assessment

Rock Island Mississippi 

Valley

Flood risk 

management

Oct. 28, 

2010

√ √ √ 122

Chacon Creek, 

Rio Grande Draft 

Feasibility Report 

and Integrated 

Environmental 

Assessment

Fort Worth Southwestern Flood Risk 

management

Nov. 17, 

2010

√ 137

Chatfield Storage 

Reallocation Study 

and EIS

Omaha Northwestern Water 

management/ 

reallocation

Oct. 25, 

2011

√ √ 134

Clear Creek Risk 

Management 

General 

Reevaluation 

Report and 

Preliminary Draft 

EISb,c

Galveston Southwestern Flood risk 

management

May, 12, 

2009

√ √ √ 290
Appendix II: Characteristics of Studies That 

Underwent Peer Review

Page 41 GAO-12-352 Army Corps of Engineers

Dollars in thousands

Study name 

and type Corps district Corps division

Corps 

planning 

center of 

expertise

Date of 

peer 

review 

report(s)

Feasibility 

or 

reevaluation 

studya

Initiated 

after 

Nov. 

2005

Estimated 

cost 

exceeding 

$45 million EIS

Cost of 

peer 

review 

contract 

Columbia River at 

the Mouth Major 

Rehabilitation 

Evaluation Report

Portland Northwestern Deep draft 

navigation

Mar. 9, 

2011

√ 200

East Branch Dam, 

Clarion River, Elk 

County, Dam 

Safety 

Modification 

Report

Pittsburgh Great Lakes 

and Ohio River

Flood risk 

management

July 2, 

2010

√ √ 105

East St. Louis 

Flood Protection 

Limited 

Reevaluation 

Report and 

Environmental 

Assessment on 

Design Deficiency 

Corrections

St. Louis Mississippi 

Valley

Flood risk 

management

Aug. 3, 

2010

√ √ √ 159

Environmental 

DNA (eDNA) 

Science and 

Methodology

N/Ad Great Lakes 

and Ohio River

Ecosystem 

restoration

Dec. 7, 

2010

√ 151

Fargo-Moorhead 

Flood Risk 

Management 

Feasibility Studyb,e

St. Paul Mississippi 

Valley

Flood risk 

management

May 17, 

2010 & 

July 7, 

2011

√ √ √ √ 282

Freeport Harbor 

Draft Feasibility 

Report and EISe

Galveston Southwestern Deep draft 

navigation

Aug. 20, 

2008

√ √ √ 214

Green Bay 

Dredged Material 

Management Plan

Detroit Great Lakes 

and Ohio River

Inland 

navigation

June 27,

2011

√ √ 101

Jamaica Bay, 

Marine Park, and 

Plumb Beach 

Draft Interim 

Feasibility Report

New York North Atlantic Ecosystem 

restoration

Dec. 8, 

2010

√ √ 164

Kissimmee River 

Restoration PostAuthorization 

Change Limited 

Reevaluation 

Report 

Jacksonville South Atlantic Ecosystem 

restoration

Oct. 13, 

2010

√ √ √ 98
Appendix II: Characteristics of Studies That 

Underwent Peer Review

Page 42 GAO-12-352 Army Corps of Engineers

Dollars in thousands

Study name 

and type Corps district Corps division

Corps 

planning 

center of 

expertise

Date of 

peer 

review 

report(s)

Feasibility 

or 

reevaluation 

studya

Initiated 

after 

Nov. 

2005

Estimated 

cost 

exceeding 

$45 million EIS

Cost of 

peer 

review 

contract 

L-31N Seepage 

Management Pilot 

Project Draft 

Integrated Pilot 

Project Design 

Report and 

Environmental 

Assessment

Jacksonville South Atlantic Ecosystem 

restoration

Mar. 10, 

2009

102

Louisiana Coastal 

Area Amite River 

Diversion Canal 

Modification 

Integrated 

Feasibility Study 

and Supplemental 

EIS 

New Orleans Mississippi 

Valley

Ecosystem 

restoration

June 23, 

2010

√ √ √ 677f

Louisiana Coastal 

Area Convey 

Atchafalaya River 

Water to Northern 

Terrebonne 

Marshes, 

Lafourche 

Terrebonne, St. 

Mary Parish, 

Integrated 

Feasibility Study 

and EIS

New Orleans Mississippi 

Valley

Ecosystem 

restoration

June 25, 

2010

√ √ √ √ 677f

Louisiana Coastal 

Area Medium 

Diversion at White 

Ditch, 

Plaquemines 

Parish, Integrated 

Feasibility Study 

and Supplemental 

EIS 

New Orleans Mississippi 

Valley

Ecosystem 

restoration

June 23, 

2010

√ √ √ √ 677f

Louisiana Coastal 

Area Small 

Diversion at 

Convent/Blind 

River, St. James 

Parish, Integrated 

Feasibility Study 

and EIS

New Orleans Mississippi 

Valley

Ecosystem 

restoration

June 22, 

2010

√ √ √ √ 677f
Appendix II: Characteristics of Studies That 

Underwent Peer Review

Page 43 GAO-12-352 Army Corps of Engineers

Dollars in thousands

Study name 

and type Corps district Corps division

Corps 

planning 

center of 

expertise

Date of 

peer 

review 

report(s)

Feasibility 

or 

reevaluation 

studya

Initiated 

after 

Nov. 

2005

Estimated 

cost 

exceeding 

$45 million EIS

Cost of 

peer 

review 

contract 

Louisiana Coastal 

Area Terrebonne 

Basin Barrier 

Shoreline 

Restoration, 

Terrebonne 

Parish, Integrated 

Feasibility Study 

and EIS

New Orleans Mississippi 

Valley

Ecosystem 

restoration

June 25, 

2010

√ √ √ 677f

Louisiana Coastal 

Protection and 

Restoration 

Program Draft 

Final Technical 

Reportg

New Orleans Mississippi 

Valley

Coastal storm 

damage 

reduction

2009 √ N/A √ 595

Marlinton Detailed 

Project Report and 

EIS 

Huntington Great Lakes 

and Ohio River

Flood risk 

management

Nov. 16, 

2010

√ √ √ 191

Melvin Price Wood 

River 

Underseepage 

Limited 

Reevaluation 

Report and 

Environmental 

Assessment on 

Design Deficiency 

Corrections

St. Louis Mississippi 

Valley

Flood risk 

management

Apr. 6, 

2011

√ √ 130

Middle 

Chesapeake Bay 

Islands Ecosystem 

Restoration Final 

Integrated 

Feasibility Report 

and EIS and 

supporting 

documentation

Baltimore North Atlantic Ecosystem 

restoration

Jan. 23, 

2008

√ √ √ 121

Mississippi 

Coastal 

Improvements 

Program 

Comprehensive 

Plan

Mobile South Atlantic Coastal storm 

damage 

reduction

Nov. 7, 

2008

√ √ √ 161
Appendix II: Characteristics of Studies That 

Underwent Peer Review

Page 44 GAO-12-352 Army Corps of Engineers

Dollars in thousands

Study name 

and type Corps district Corps division

Corps 

planning 

center of 

expertise

Date of 

peer 

review 

report(s)

Feasibility 

or 

reevaluation 

studya

Initiated 

after 

Nov. 

2005

Estimated 

cost 

exceeding 

$45 million EIS

Cost of 

peer 

review 

contract 

Mississippi River 

Gulf Outlet 

Ecosystem 

Restoration Plan 

Feasibility Study 

and EIS

New Orleans Mississippi 

Valley

Ecosystem 

restoration

June 3, 

2011

√ √ √ √ 373

Mohawk Dam 

Major 

Rehabilitation 

Report

Huntington Great Lakes 

and Ohio River

Flood risk 

management

Apr. 22, 

2011

√ √ 240

Navigation and 

Ecosystem 

Sustainability 

Program, Project 

P2, Lock and Dam 

22 Fish Passage 

Improvement 

Project 

Implementation 

Report

Rock Island Mississippi 

Valley

Ecosystem 

restoration

Dec. 17, 

2009

√ √ 141

Olmsted PostAuthorization 

Change Reportb,e

Louisville Great Lakes 

and Ohio River

Inland 

navigation

Nov. 15, 

2010 & 

Nov. 4, 

2011

√ √ √ 201

Port Everglades 

Harbor Science 

Reports for the 

Feasibility Study 

and EIS

Jacksonville South Atlantic Ecosystem 

restoration

Aug. 17, 

2011

√ √ √ 76

Programmatic EIS 

for the Mechanical 

Creation and 

Maintenance of 

Emergent 

Sandbar Habitat 

on the Upper 

Missouri River

Omaha Northwestern Ecosystem 

restoration

Mar. 17, 

2010

√ √ 120

Sabine Neches 

Waterway 

Channel 

Improvement Plan 

Draft Feasibility 

Report, Draft EIS, 

and Supporting 

Documentatione

Galveston Southwestern Deep draft 

navigation

Dec. 13, 

2007

√ √ √ 313
Appendix II: Characteristics of Studies That 

Underwent Peer Review

Page 45 GAO-12-352 Army Corps of Engineers

Dollars in thousands

Study name 

and type Corps district Corps division

Corps 

planning 

center of 

expertise

Date of 

peer 

review 

report(s)

Feasibility 

or 

reevaluation 

studya

Initiated 

after 

Nov. 

2005

Estimated 

cost 

exceeding 

$45 million EIS

Cost of 

peer 

review 

contract 

Sacramento River 

Deep Water Ship 

Channel Limited 

Reevaluation 

Study and 

Supplemental EIS

San 

Francisco

South Pacific Deep draft 

navigation

Sep. 30, 

2011

√ √ √ 164

San Clemente 

Storm Damage 

and Shoreline 

Protection 

Feasibility Study 

Los Angeles South Pacific Coastal storm 

damage 

reduction

July 23,

2010

√ √ 145

Savannah Harbor 

Expansion Project 

General 

Reevaluation 

Reporte

Savannah South Atlantic Deep draft 

navigation

Feb. 11, 

2011

√ √ √ 484

St. Johns Bayou 

and New Madrid 

Floodway 

Consolidated 

NEPA Document 

and Work Plan 

(Phase I)h

Memphis Mississippi 

Valley

Flood risk 

management

Oct. 23, 

2009

√ √ 453

Success Dam 

Remediation Dam 

Safety Assurance 

Program Letter 

Report

Sacramento South Pacific Flood risk 

management

Mar. 24, 

2011

√ √ 285

Surf City and 

North Topsail 

Beach Draft 

Integrated 

Feasibility Report 

and EIS

Wilmington South Atlantic Coastal storm 

damage 

reduction

Apr. 16, 

2010

√ √ √ 187

Tamiami Trail 

Limited 

Reevaluation 

Report

Jacksonville South Atlantic Ecosystem 

restoration

June 2, 

2008

√ √ 142

Tres Rios del 

Norte, Pima 

County, Feasibility 

Study

Los Angeles South Pacific Ecosystem 

restoration

Sep. 26, 

2011

√ √ √ 159
Appendix II: Characteristics of Studies That 

Underwent Peer Review

Page 46 GAO-12-352 Army Corps of Engineers

Dollars in thousands

Study name 

and type Corps district Corps division

Corps 

planning 

center of 

expertise

Date of 

peer 

review 

report(s)

Feasibility 

or 

reevaluation 

studya

Initiated 

after 

Nov. 

2005

Estimated 

cost 

exceeding 

$45 million EIS

Cost of 

peer 

review 

contract 

Western C-111 

Spreader Canal 

Project 

Implementation 

Report

Jacksonville South Atlantic Ecosystem 

restoration

Oct. 30, 

2009

√ √ √ 184

White Oak Bayou 

Flood Damage 

Reduction Plan 

General 

Reevaluation 

Report

Galveston - 

Local sponsor 

led

Southwestern Flood risk 

management

Jan. 11, 

2011

√ √ 124

Wood River Levee 

System Limited 

Reevaluation 

Report for Design 

Deficiency 

Corrections

St. Louis Mississippi 

Valley

Flood risk 

management

Aug. 24, 

2011

√ √ √ 79

Source: GAO analysis of Corps documents. 

Note: In addition to the 49 peer reviews included on this list, the Louisiana Coastal Area Beneficial 

Use of Dredged Material Program Preliminary Study Report— which addressed the methodology to 

select and prioritize projects under the program—underwent peer review in July 2008. Section 

7006(d) of WRDA 2007 authorized this $100 million program for the beneficial use of dredged 

material. The review was overseen by the U.S. Geological Survey and conducted by the Louisiana 

Coastal Area Science Board, and about $12,000 in Corps funds were used to support its completion. 

However, because the completion of this review was overseen by the U.S. Geological Survey, and 

did not involve the Corps’ typical contract process or costs, it was not included in our list or analysis of 

completed peer reviews.

a

Postauthorization change reports are generally considered reevaluation reports, and project 

implementation reports are generally considered feasibility studies.

b

Peer review included multiple reports.

c

As part of this contract, a follow up report was completed in July 2010 to determine whether 

recommendations from the peer review report were addressed. This follow up report was not included 

in our analysis because it was not provided until our review was complete.

d

The eDNA science and methodology work was done by the University of Notre Dame, and the study 

is part of the Great Lakes Regional Initiative.

e

Peer review was completed under multiple contracts, and the contract cost provided is the sum of 

the cost of those contracts.

f

Multiple peer reviews were conducted under one contract for five of the Louisiana Coastal Area 

studies.

g

This review does not include a preliminary report done by Battelle before WRDA 2007. Also, 

according to officials, the estimated cost of the project for this study is not applicable (N/A) because 

the study made no recommendations for construction.
Appendix II: Characteristics of Studies That 

Underwent Peer Review

Page 47 GAO-12-352 Army Corps of Engineers

h

As part of this contract, a report on Phase I and Phase II of the project were completed. The Phase II 

report was completed in April 2010 with an addendum in November 2010 but was not included in our 

analysis because it was not provided until our review was complete.
Appendix III: Comments from the Department 

of Defense

Page 48 GAO-12-352 Army Corps of Engineers

Appendix III: Comments from the 

Department of Defense 
Appendix III: Comments from the Department 

of Defense

Page 49 GAO-12-352 Army Corps of Engineers
Appendix III: Comments from the Department 

of Defense

Page 50 GAO-12-352 Army Corps of Engineers
Appendix III: Comments from the Department 

of Defense

Page 51 GAO-12-352 Army Corps of Engineers
Appendix IV: GAO Contact and Staff 

Acknowledgments

Page 52 GAO-12-352 Army Corps of Engineers

Anu K. Mittal, (202) 512-3841 or mittala@gao.gov

In addition to the individual listed above, Vondalee R. Hunt, Assistant 

Director; Darnita Akers; Elizabeth Beardsley; and Janice Ceperich made 

significant contributions to this report. Ellen Wo Chu, Cindy Gilbert, 

Richard P. Johnson, Ben Shouse, and Kiki Theodoropoulos also made 

key contributions.

Appendix IV: GAO Contact and Staff 

Acknowledgments 

GAO Contact

Staff 

Acknowledgments 

(361279)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (www.gao.gov). Each weekday afternoon, 

GAO posts on its website newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations 

Public Affairs

Please Print on Recycled Paper.