Missouri River Flood and Drought: Experts Agree the Corps Took Appropriate Action, Given the Circumstances, but Should Examine New Forecasting Techniques
Released: 2014-09-12T00:00:00

Letter

Background

Missouri River Basin and the Mainstem Dams

The Missouri River basin extends from the Rocky Mountains across portions of the Midwest and Great Plains, covering roughly one-sixth of the continental United States (see fig. 1).

Of the six dams along the mainstem of the Missouri River, one is in Montana (Fort Peck), one is in North Dakota (Garrison), three are in South Dakota (Oahe, Big Bend, and Fort Randall), and one is along the South Dakota-Nebraska border (Gavins Point). This reservoir system is the largest in the United States and contains about 73.1 million acre-feet (MAF) of water storage capacity. A majority of the system’s storage capacity is in the three upstream reservoirs—Fort Peck Lake, Lake Sakakawea, and Lake Oahe. Gavins Point dam is the furthest downstream of the six dams, and its water releases support all uses of the river below the reservoir system. Gavins Point dam is about 811 miles upstream from the mouth of the Missouri, where it enters the Mississippi River near St. Louis; water released from Gavins Point dam takes about 10 days to reach the Mississippi River.

The Master Manual and Authorized Purposes

The Master Manual lays out procedures for the Corps’ management of the six Missouri River mainstem dams as a system. In the Master Manual, the Corps attempts to balance the eight congressionally authorized purposes of the river. The current Master Manual was developed over the course of 17 years and involved extensive consultation between the Corps and basin stakeholders, as well as multiple lawsuits. Key changes in the Master Manual revision include more rapid measures taken in response to drought conditions, changes in the water levels in the upper three reservoirs during the spring to support fish spawning, and measures to support endangered species along the river.

The Master Manual allocates water within the reservoir system to four different storage zones (see fig. 2):

The Permanent Pool includes about 25 percent of the system’s storage capacity and is intended to be full at all times to maintain a minimum amount of water in the reservoirs for hydropower production, fish and wildlife in and along the reservoirs, and reservoir-based recreation.

The Carryover Multiple Use Zone stores water for irrigation, navigation, hydropower, water supply, recreation, water quality control, and fish and wildlife. This zone is intended to maintain downstream river flows, although at lower levels, even in a succession of dry years. When the basin is not experiencing a drought, this zone is designed to be full when the runoff year begins on March 1. During times of drought, water from this zone is used to support the aforementioned authorized purposes, though at lower levels.

The Annual Flood Control and Multiple Use Zone provides storage space for spring and summer runoff that can be used throughout the year to support all authorized purposes. The Master Manual sets a goal of having this zone empty on or about March 1 of every year, so any water that is stored here during the spring and summer is meant to be released prior to the start of the next runoff season, which is approximately March 1.

The Exclusive Flood Control Zone is only used to store floodwaters in extreme and unpredictable floods and is emptied as rapidly as downstream conditions permit.

The eight authorized purposes of the system have different water needs, and the Master Manual addresses each of these purposes (see table 1).

In addition, some of the Corps’ reservoir management is related to compliance with the Endangered Species Act of 1973, as amended. Two bird species nest along the river from May to August: the endangered least tern and threatened piping plover. Releases from Gavins Point, Fort Randall, and Garrison dams have been modified to accommodate these bird species by adapting releases to prevent, as much as possible, inundation of bird nests along the river. In addition, one endangered fish species, the pallid sturgeon, lives in the Missouri River. For the pallid sturgeon, the Master Manual calls for two “pulses” of water (temporary higher releases) from Gavins Point dam in the spring to mimic the higher spring river flows that occurred prior to the construction of the mainstem reservoirs. The last spring pulse was implemented in 2009; pulses were cancelled in 2010 and 2011 due to high water levels downstream. According to Corps officials, a 2011 independent review panel questioned the efficacy of the spring pulse and the Fish and Wildlife Service and the Corps are, therefore, currently reevaluating the pulse.

Hydrologic Data and Forecasting for Reservoir Management

The Corps uses numerous types of hydrologic data—data relating to the movement and distribution of water in the basin—to track current conditions in the basin. Most of these data are collected by other federal agencies as part of nationwide efforts to gather weather and hydrologic data (see table 2).

Reports by federal agencies and others have highlighted limitations in some of these data collection efforts in the Missouri River basin.

Streamflow. According to USGS data and an October 2012 report by the Corps assessing post-flood vulnerabilities, loss of streamgages in the basin has reduced available information about streamflows. For example, according to USGS data, operation of 79 streamgages in the Missouri River basin has been discontinued in the last 10 years; this represents about 9 percent of the streamgages in the basin.

Soil Moisture. The October 2012 Corps’ vulnerability report and a December 2011 Independent Technical Review Panel commissioned by the Corps indicate that data on soil moisture in the Missouri River basin is currently limited. The October 2012 Corps’ vulnerability report recommended that soil moisture be measured at predefined locations in plains states. Data on soil moisture can indicate how much of the precipitation that falls can be expected to run off into the reservoir system. For example, if soils are dry then precipitation is more likely to soak into the soil than to runoff into nearby rivers or streams.

Plains snowpack. Three reports have recently identified limitations in plains snowpack data: the October 2012 Corps’ vulnerability report, the December 2011 Independent Technical Review Panel Report, and a May 2012 assessment of forecasting during the Missouri River flood For example, the May 2012 NWS report noted that the by the NWS.National Operational Hydrologic Remote Sensing Center (NOHRSC) provides modeled information on snow-water equivalent, but that observational data in the basin are sparse and not always representative of basin-wide conditions.

Precipitation. The May 2012 assessment of NWS forecasting also noted that precipitation gauge and radar data on precipitation in the Missouri River basin were insufficient during the flood.

Agencies have begun taking steps to address some data limitations. For example, in response to the December 2011 Independent Technical Review Panel Report, the Corps worked with officials from NOAA and NRCS, among others, to develop an interagency proposal, released in February 2013, to create a snowpack and soil moisture monitoring system in the plains. Under the proposal, the agencies would (1) enhance existing climate stations with snow depth and soil moisture sensors; (2) install new climate stations in the basin to enhance existing coverage; (3) enhance NOHRSC airborne surveys; (4) identify and train volunteer or part-time hires to conduct manual snow sampling; and (5) fund state coordinator positions in Montana, Nebraska, North Dakota, South Dakota, and Wyoming to coordinate snow surveys and other snow data networks at a state level. The Water Resources Reform and Development Act (WRRDA) of 2014, enacted into law in June 2014, included a requirement that the Secretary of the Army, in coordination with other specified agencies, develop this type of monitoring system in the Upper Missouri River Basin. In addition, NWS has developed a new technology, the Multi-Radar Multi-Sensor system, which integrates information from NWS, Canadian, and other radar systems with on-the- ground precipitation gauge information and model data to provide better estimates of precipitation. According to NWS officials, the Multi-Radar Multi-Sensor system is also capable of mitigating some gaps in radar coverage by extending the effective range of radar-based precipitation estimates from individual radars. According to NWS officials, this technology will be implemented nationwide by the end of 2014.

According to Corps documents and officials, the Corps uses hydrologic data as an input to forecasts used to manage the reservoir system. The Corps runs two key forecasts to generate information for basin stakeholders and to make reservoir release decisions.

Monthly forecast. On a monthly basis, or more frequently as needed, the Corps produces a forecast of the expected annual runoff for the remainder of the calendar year. This forecast takes into consideration current basin conditions, such as soil moisture and snowpack, as well as long-range weather outlooks and historical trends. The Corps produces a “basic” forecast, and then adjusts that forecast by a predetermined percentage to generate “upper basic” and “lower basic” forecasts to create a range of potential runoff conditions. According to Corps officials, the upper and lower basic forecasts are designed to be approximately one standard deviation away from the basic forecast and cover approximately 80 percent of the likely variation in expected runoff based on an analysis of historic runoff records. Each month, these runoff forecast estimates are used as input to the 3-week forecast, which forecasts reservoir inflows, releases, storage levels, and hydropower generation, among other things. According to Corps officials, this forecast is used by basin stakeholders to make business decisions that are affected by reservoir releases. For example, the Western Area Power Administration, which is responsible for marketing all the hydropower generated by the six dams, makes power purchase decisions based on this forecast. In addition, the Corps makes some reservoir release decisions based on the monthly forecast, particularly to move water between the six reservoirs to adjust to current weather conditions or support downstream uses.

Three-week forecast. On a weekly basis, or more frequently as needed, the Corps produces a forecast of reservoir inflows, outflows, storage, and power generation over the next 3-to-5 weeks. According to Corps officials, this model uses “water on the ground” information— specifically streamflows and reservoir levels—combined with information from the basic monthly forecast. Officials said this is the primary model they use to set daily and weekly reservoir releases and that they try not to deviate significantly from projected releases at Fort Peck and Garrison dams in this forecast, unless there are unusual circumstances. Adjustments at the other four dams are routinely made to respond to changing conditions on the ground, such as rainfall below the reservoir system.

The Corps’ current runoff forecasts are deterministic, meaning that the models create a single forecast based on the existing hydrologic data. Although the monthly runoff forecast also includes the upper basic and lower basic conditions, these are still deterministic because they are generated by using a multiple of the basic forecast. However, according to NOAA documents, error can be introduced into deterministic forecasts when initial hydrologic conditions are not fully known. According to NWS officials, a different type of forecasting—called probabilistic forecasting— attempts to account for uncertainty in the forecast by, for example, using statistical techniques to simulate multiple, slightly different initial conditions. These officials said that probabilistic forecasts provide a range of potential outcomes and their likeliness. Probabilistic techniques are used extensively in weather forecasting both for routine forecasts and for more rare events such as hurricanes, according to NWS officials.

The 2011 Flood and 2012 and 2013 Drought

Annual runoff into the Missouri River reservoir system can vary significantly from year to year (see fig. 3). The lowest runoff year was 1931, with 10.6 MAF of runoff. The highest runoff year was in 2011, with 61 MAF; 61 MAF is about enough water to cover nearly the entire state of Oregon (61.4 million acres) in water 1 foot-deep. Prior to 2011, the highest runoff, which also caused flooding along the river, was 49 MAF in 1997. Runoff in 2011 was about 25 percent greater than in 1997 and 148 percent greater than the historical median of 24.6 MAF.

According to the May 2012 assessment of NWS forecasting, several factors combined in 2011 to produce record runoff: wet soil conditions throughout the basin leading into the winter of 2010-2011, high snowpack in both the plains and mountains, and extreme rainfall in May and June of 2011.

Wet basin conditions. After experiencing a drought between 2000 and 2008, the Missouri River basin experienced relatively wet years in both 2009 and 2010. According to a December 2013 NOAA report examining climate extremes in the Missouri River basin, 2010 was the fifth wettest year on record. This precipitation created wet soil moisture conditions throughout the upper basin in the fall of 2010.

Plains snowpack. Snow can accumulate on the plains through late March or early April. In 2011, total snowfall in the Missouri River basin plains states was well above average, and the snowpack was greater than usual. Numerous cities in the basin set new seasonal snowfall records. For example, Williston, North Dakota, had 107 inches of snow, compared with a long-term average of about 35 inches.

Mountain snowpack. Snowpack generally accumulates in the mountains of Montana and Wyoming throughout the winter, peaking in mid-April and then providing runoff as it melts through May and June. As of March 1, 2011, the mountain snowpack was slightly above average at about 110 percent of normal (see fig. 4). However, late April and May were extremely wet and cold, and mountain snowpack continued to build to record levels in many areas. Mountain snowpack in 2011 peaked in early May at approximately 140 percent of normal.

Rainfall in May and June. Record rain fell in Montana, northern Wyoming, and the western Dakotas in May and early June 2011. Areas of south central and southeast Montana received as much as 15 inches of rain in May, which is 12 inches above normal. Most of eastern Montana received at least three times more precipitation than normal, and the month of May and was ranked as one of the wettest Mays on record in Montana, Wyoming, North Dakota, South Dakota, and Nebraska. Rain continued to fall in June, with Montana, North Dakota, South Dakota, and Nebraska receiving 3 to 8 inches more rain than normal.

As these weather conditions unfolded in 2011, the Corps continued to modify its release rates from the reservoir system (see fig. 5). In early April, the Corps began flood control operations by increasing releases from Gavins Point in response to the above average mountain snowpack. The Corps continued increasing releases throughout April and May, reaching 50,000 cubic feet per second (cfs) on May 9 and surpassing the previous high release rate of 70,000 cfs (set in 1997) on May 29. Release rates increased particularly fast between late May and late June, when releases peaked at about 160,000 cfs, more than double the previous high release rate; 160,000 cfs is about the amount of water from two Olympic-sized swimming pools going past a single point in 1 second. Gavins Point release rates remained above 100,000 cfs until August 31, and it was not until December that releases returned to a more normal rate of 35,000 cfs.

In the fall of 2011, as basin stakeholders and the Corps were repairing infrastructure and recovering from the flood, there was concern that additional flooding would occur in 2012. However, 2012 brought drought throughout the Missouri River basin. Nebraska and Wyoming experienced their driest year in 118 years of recordkeeping, and several other states in the basin also had very dry years. For example, Missouri had its seventh driest year, Iowa had its 11th driest year, and South Dakota had its 13th driest year.

Drought intensity, as defined by the U.S. Drought Monitor, increased from January through July of 2012, at which point moderate and severe drought conditions were present in southern Montana, western South Dakota, western Nebraska, and Wyoming. Conditions worsened during the summer, and by October 2012, extreme and exceptional drought conditions were present across Wyoming, South Dakota, Nebraska, and western Iowa. Total runoff into the Missouri River mainstem reservoir system in 2012 was 19.5 MAF, or about 77 percent of normal runoff. In addition, runoff into the Missouri River below the reservoir system was also extremely low at 51 percent of normal runoff.

According to the Master Manual, March 15 reservoir levels must be at least 54.5 MAF to initiate full-service to navigation. July 1 reservoir levels must be at least 57 MAF to continue full-service to navigation for the remainder of the season. and the Master Manual called for minimum winter releases of 12,000 cfs from Gavins Point. However, water intake owners in the lower basin were concerned about maintaining access to the river at those low flows, particularly since the 2011 flood scoured the river bottom in many areas. According to the Corps report describing reservoir management in 2012, the Corps exercised the flexibility in the Master Manual and elected to keep winter releases at 14,000 cfs to prevent municipalities and power plants from losing access to the river.

Drought conditions persisted into 2013, and the reservoir system was 7.4 MAF below the top of the Carryover Multiple Use zone on April 1. Due to the low volume of water in the reservoirs, the Corps continued implementing drought conservation measures, according to Corps officials. For example, navigation releases during April through June were at a minimum service level, meaning flows were high enough for an 8- foot-deep channel. The drought began to ease in parts of the basin during the summer due to rainfall and associated runoff. The higher volume of water in the reservoirs in July led to a slight increase in release rates for navigation, as well as a full 8-month navigation season. Runoff into the Missouri River reservoirs was about average in 2013 at 25.1 MAF, although water levels in the upper three reservoirs remained low.

Experts Agreed the Corps Made Appropriate Release Decisions during the Flood and Drought, under the Circumstances

Experts who participated in our meeting agreed that the Corps made appropriate release decisions during the flood and drought, given that neither the flood nor drought could have been predicted and the Corps’ need to follow the guidelines in the Master Manual. These experts did not suggest changes to the Master Manual due to the 2011 flood or subsequent drought.

Experts Said the Corps Could Not Have Predicted or Prevented Flooding Due to Extreme Weather in 2011 and Its Release Decisions Were Appropriate

Experts who participated in our meeting discussed the above normal snowpack in the mountains and the plains, but they agreed the flood was triggered by the extreme rain in eastern Montana in May and June 2011. This conclusion is consistent with the December 2013 NOAA report examining climate extremes in the Missouri River basin that stated the record-setting rains were the final and, perhaps most critical, meteorological factor leading to high runoff and flooding in 2011. The experts agreed that no existing forecasting tools, including those used by the Corps and NOAA, could have accurately predicted the extreme rainstorms that occurred in Montana more than a week in advance. The December 2011 Independent Technical Review Panel Report commissioned by the Corps also reached this conclusion, noting that accurate prediction of precipitation more than a week in advance is beyond the current state of science.

Prior to our meeting, one of the experts reviewed information that was available on March 1, 2011, to determine what long-range forecast models were projecting about precipitation in the Missouri River basin for spring 2011. This expert noted that of the models he examined, only one forecasted a wet spring, and all other models forecasted normal or dry conditions. Based on the information the Corps had available in March 2011—these forecasts as well as evidence of the slightly above-normal mountain snowpack—experts who participated in our meeting said they considered the Corps release decisions early in the spring to be appropriate.

Experts who participated in our meeting also agreed that the Corps could not have prevented flooding in 2011. Snow continued to accumulate in the mountains in April and May—well past the average date of maximum snow accumulation. The experts said that, by June 2011, the volume of water coming into the reservoirs from the extreme rains and melting snow was so great that the Corps had no choice in June and July but to release water to accommodate the inflow and prevent damage to dam infrastructure, such as spillways in danger of being overtopped. The December 2011 Independent Technical Review Panel Report reached a similar conclusion. This report cited the absence of major dam failures as evidence of the Corps’ success during the flood and noted that dam failure beginning at Fort Peck would have caused a catastrophic disaster of unprecedented magnitude.

Even if the Corps had decided on March 1, 2011, to increase releases due to the slightly larger-than-average snowpack in the mountains and plains, experts who participated in our meeting agreed that action would not have significantly reduced peak flows because of the extremely large amount of runoff in 2011. One of the experts said it would have taken several months for the Corps to release enough water from the reservoirs to make space for the runoff from the rainstorms and melting snow, and that action also could have resulted in downstream flooding. Specifically, this expert noted that high releases during the winter can cause flooding because of ice on the river, so the Corps would have needed to know in October 2010 about the upcoming extreme spring rain to release enough water in the fall to create more space in reservoirs.

One of the experts said that having additional space in the reservoirs on March 1 was the only way the Corps could have significantly reduced the peak downstream flooding. This expert also noted, however, that taking steps to lower reservoir levels in this way may not be consistent with the Master Manual. Another of the experts who participated in our meeting noted that while having more flood control storage available on March 1 each year reduces the chances of flooding, it could have negative effects on the other authorized purposes of the mainstem dams in nonflood years.

Experts Said the Corps Could Not Have Predicted the Drought in 2012 and Made Release Decisions in Accordance with the Master Manual

Experts who participated in our meeting generally agreed that the Missouri River basin’s rapid descent into drought could not have been predicted. One of the experts qualified this statement, noting that the drought could not have been predicted with sufficient certainty to change reservoir decisions, given the high costs of the forecast being incorrect. Prior to the meeting, one of the experts reviewed information that was available in the spring of 2012 to determine what long-range forecast models were projecting about precipitation in the Missouri River basin for the remainder of 2012. This expert noted that there was no predictability; some of the models were forecasting wet conditions, and others were forecasting dry conditions. He also explained that some of the models predicting dry conditions frequently forecast dry conditions that do not materialize.

As the drought took hold, experts who participated in our meeting said the Corps followed procedures as laid out in the Master Manual. For example, the experts noted that, in 2012, the Corps released water for a full-service navigation season. The experts said that the navigation season was in accordance with the Master Manual, but it drained the reservoirs relatively quickly during the very dry summer of 2012. Specifically, according to the Corps report describing its management of the reservoir system in 2012, 22 percent of the water in storage was released in 2012, which would have reduced the amount of water available for future years if the drought lasted for several years.

Experts who participated in our meeting also agreed that the Corps appropriately exercised the reservoir release flexibility granted by the Master Manual. For example, the experts agreed it was appropriate that, in the winter of 2012-2013, the Corps kept winter releases higher than normal to ensure that water intakes along the river had continued access for municipal and industrial uses.

Experts Did Not Suggest Changes to the Master Manual

Experts who participated in our meeting agreed that the Corps does not need to change the Master Manual due to the 2011 flood or 2012 and 2013 drought, noting that there are no obvious deficiencies in the Master Manual. One of the experts noted that occurrence of similar extreme events should be incorporated in analyses that support any potential future changes in operating rules. In addition, several of the experts mentioned that developing the Master Manual took 17 years and that Missouri River basin stakeholders have agreed to the trade-offs and compromises in the current Master Manual. Other experts who participated in our meeting noted, however, that if the Corps could develop improved forecasting tools, it might be useful to evaluate whether changes to the Master Manual would help the Corps to act on information from the new tools. These experts explained that they were not sure whether such an evaluation would find that changes to the Master Manual would significantly help the Corps manage the reservoirs and balance the authorized purposes, but they thought it was worth examining if new forecasting tools are developed.

Finally, experts who participated in our meeting also discussed challenges the Corps faces in balancing reservoir releases for all eight authorized purposes. Some of the experts thought that the Corps should restart a study called the Missouri River Authorized Purposes Study (MRAPS). MRAPS was authorized in the Corps’ fiscal year 2009 appropriations act to examine the extent to which the current authorized purposes of the river meet the needs of the residents of the Missouri River basin. The Corps worked on MRAPS for 2 years before it was defunded by Congress in fiscal year 2011 appropriations.experts thought that an examination of the purposes was warranted, in part, because of the number of reservoir regulation decisions made for the purpose of navigation. Another of the experts cautioned, however, that such a study might also open the idea of operating the Missouri River to benefit stakeholders outside the basin, such as navigators along the lower Mississippi River. This expert said that the navigation on the Mississippi River is a $1.2 billion industry and, in some years, could benefit from flow support from the Missouri River. He pointed out, however, that such an action could use a significant amount of water from the reservoirs, perhaps to the detriment of current authorized purposes. According to Corps officials, they are not authorized by Congress to make reservoir release decisions to support Mississippi River navigation.

Experts Identified Improvements in Data Collection and Forecasting That Could Help the Corps Make Better Release Decisions in Nonextreme Events

Experts who participated in our meeting suggested that collecting more hydrologic data, improving existing hydrologic data, and incorporating probabilistic forecasting techniques could improve the Corps’ ability to make release decisions in nonextreme events. The experts stated that these data and forecasts would not have predicted the 2011 flood. However, they explained that these data and forecasts could be helpful in future, less extreme, floods.

Experts Said Improving Hydrologic Data Could Assist the Corps in Making Release Decisions

Experts who participated in our meeting suggested that improving existing hydrologic data and collecting new data could improve the Corps’ ability to make release decisions. The experts mentioned that streamflow and precipitation data could be improved, and that new soil moisture, plains snowpack, and archaeological flood and drought data could be collected. The experts said they did not believe that having these data would have materially impacted the Corps’ response to the 2011 flood. One of the experts said, while improved data would not have prevented the flood, it might have helped the Corps reduce the severity of the flood to a small degree. However, it is important to note that the hydrologic data systems discussed by the experts are not managed by the Corps but by other federal agencies as part of nationwide efforts to gather this data. Therefore, the Corps cannot directly control the extent to which improvements in these systems are made.

Streamflow Data

Experts who participated in our meeting said that maintaining and improving the USGS streamgage network is critical because it provides important data on current and historical streamflows. The experts said that historical streamflow records can also help modelers describe how flow conditions persist in streams, which enables them to create probabilistic forecasts of possible future river flows. As previously mentioned, USGS data indicates that about 9 percent of streamgages in the Missouri River basin have been discontinued in the last 10 years. USGS officials said that streamgages are often discontinued due to funding shortages, either at USGS or from the cooperative partner agencies which help fund the streamgages. According to USGS officials, the Corps provides funding for 264 of the 892 streamgages in the Missouri River basin. According to Corps officials, when their support for streamgages is reduced, they prioritize saving downstream streamgages on tributaries with more than one streamgage because downstream streamgages capture more of the river’s flow. Corps officials said that under normal circumstances, losing data from upstream streamgages is not a serious problem, but that during a flood it can become a major challenge. For example, during the 2011 flood, the sole streamgage on the Judith River—a Missouri River tributary that runs through central Montana—was destroyed when the bridge it was attached to was washed away. Without streamflow data on the Judith River, the Corps had to estimate the Judith River flows, which resulted in less accurate information.

USGS officials said that they do their best to maintain the integrity of the streamgage network. USGS officials told us that most streamgages are funded through cooperation with federal, state and local government agencies. When streamgages are in danger of losing their funding, USGS officials work with their cooperative partners to find other funding sources to maintain the streamgage and are usually successful in finding funding for the most crucial ones. However, one of the experts who participated in our meeting said that the current cooperative funding model relied on by USGS to support most of the streamgages makes it a challenge to maintain the network since the cooperative partners may have other priorities. For example, USGS officials told us that one reservoir manager in Illinois pulled funding for a streamgage and used the money to build an outhouse. USGS officials began the National Streamflow Information Program in 2003 to federally fund a core network of 4,756 streamgages throughout the country. This program is designed to, among other things, improve USGS’ ability to continue operating high-priority streamgages when partners discontinue funding. According to USGS officials, the National Streamflow Information Program received a $6 million funding increase in FY 2014 to $33 million.

Precipitation Data

Experts who participated in our meeting also identified gaps in the weather radar and precipitation gauge collection systems. Specifically, one of the experts said that weather radar does not do a good job of measuring winter precipitation and, even if it did, radar coverage in many parts of the basin is limited. For example, this expert noted that areas approximately 30 miles north of Pierre, South Dakota, have relatively poor radar coverage. Weather radar data is supplemented by precipitation gauge data, such as through the volunteer Community Collaborative Rain, Hail and Snow (CoCoRaHS) Network. However, the experts said that the basin is sparsely populated, which limits the pool of volunteer observers, potentially making it more difficult to collect the data needed to supplement the weather radar network. Corps officials said that the CoCoRaHS Network can compensate for gaps in the radar coverage, and few stakeholders are seeking to expand the radar network. NWS officials agreed that radar coverage was poor in the northern and western parts of the basin, such as southeastern Montana and central South Dakota. However, NWS officials told us that they do not currently have plans to expand radar coverage because current off-the-shelf radar does not have the same capabilities as the NWS’s current system. Integrating off-the- shelf radars into the system would be difficult, and building radars that match the capabilities of the current system would be expensive, according to these officials. NWS officials said that new technology—such as the Multi-Radar Multi-Sensor system—will help mitigate the gaps in precipitation data in areas where radar coverage is sparse.

Soil Moisture and Snowpack Data

Experts who participated in our meeting said that making improvements to soil moisture and snowpack data would be very useful for making long- term forecasts because these conditions can be observed months before the associated runoff reaches the reservoir system. Some of the experts noted that there are major gaps in plains snowpack and soil moisture monitoring data, and improving these data would be useful in improving Corps’ forecasting models. The experts said that gathering this data could be accomplished if the resources were available and stakeholders were willing to participate. A NOAA official working on the February 2013 interagency proposal to create a snowpack and soil moisture monitoring system said that NOAA is working with stakeholders to develop the interagency proposal, but implementation was on hold while stakeholders were waiting to see if it would be included in the then pending 2014 WRRDA. However, this official noted that, even though the 2014 WRRDA requires the development of a monitoring system for soil moisture and snowpack data, there may be challenges in funding the proposal, which has a projected up-front cost of $6.25 million. Specifically, agencies supporting the proposal—such as the Corps, NOAA and NRCS—will need to find money for upfront costs in their existing budgets, which could take funds away from other programs and priorities. In addition, according to the February 2013 interagency proposal, maintaining the network once it is built would cost $1.46 million per year.

Archaeological Data

Some of the experts who participated in our meeting also recommended collecting archaeological data on floods and droughts, which could be used to provide a better understanding of the extreme floods and droughts in the basin before recordkeeping began in 1898. For example, USGS has ongoing archaeological work in the Black Hills of South Dakota that could allow a better understanding of how large a 10,000- year flood would be compared with a 100-year flood. the experts cautioned that these data may not be useful for the Corps. According to a Corps official, these data would not be used in developing the manuals they use to guide regulation decisions, although it would provide information about the risk of larger floods.

The “100-year flood” is a flood that has a 1 percent chance of occurring each year. This flood magnitude is used by federal agencies to administer floodplain management programs.

Experts Identified Forecasting Techniques That Could Improve the Corps’ Ability to Anticipate Weather Developments and Make Release Decisions

Experts who participated in our meeting agreed that by incorporating probabilistic techniques into runoff forecasts, the Corps could improve its ability to make release decisions in nonextreme events. Two of the experts indicated that probabilistic forecasting techniques could also improve the Corps’ ability to make release decisions in extreme events. Experts who participated in our meeting agreed that probabilistic forecasting techniques could help the Corps manage risks and make decisions in the face of uncertainty. One of the experts said that these techniques are useful for communicating risk information—such as the risk of severe excesses or deficiencies of water—to the public and public officials. Another of the experts said that probabilistic techniques could allow the Corps to provide increased benefits to the system. These benefits could include higher reliability of water supply without increasing flood risk, increased flood protection, small increases in hydropower production, and easier implementation of variable river flows to create and maintain specific fish and wildlife habitats.

The primary type of probabilistic forecasting techniques discussed by experts who participated in our meeting was ensemble forecasting. Ensemble forecasting combines multiple forecasts to generate a sample of potential future weather developments. The individual forecasts, called ensemble members, can be created either using several different models in concert or multiple runs of the same model with slightly different initial conditions. These forecasts are then compared to determine how much agreement there is between the various ensemble members. Some of the experts said that ensemble forecasts can help forecasters correct for both uncertainty about initial conditions and uncertainty about the how a model is constructed, which are common causes of forecasting error. One method of generating ensemble forecasts that the experts said could be potentially useful in the Missouri River basin is the Hirsch method. This method uses correlations between 1 month’s streamflow and the previous month’s flow to generate ensemble members, since flow conditions from the previous month generally persist into the next. For example, Hirsch model forecasts examine the historical statistical relationships between streamflow in March and streamflow in April since March’s conditions persist into April. When this relationship exists, it allows for forecasts with less variance and uncertainty compared with other methods, leading to more accurate forecasts.

Another way experts who participated in our meeting suggested that the Corps could incorporate ensemble modeling into its forecasts would be to leverage the existing ensemble streamflow forecast created by the NWS Advanced Hydrologic Prediction Service (AHPS). The AHPS forecasts use ensemble streamflow predictions to determine the chances of a river exceeding minor, moderate, or major flood levels over the next 90 days. One of the experts noted that the current AHPS forecast locations in the Missouri River basin are not located where the Corps likely needs them to be to have enough information to make their decisions. NWS officials told us that they produce probabilistic forecasts at 465 AHPS forecast locations in the basin but that none of these locations are along the mainstem of the Missouri River because probabilistic forecasting along the mainstem would require integrating the Corps’ reservoir management procedures into the NWS probabilistic models. One of the experts said the Corps could overcome this challenge by identifying statistical relationships between the existing AHPS locations and the locations that the Corps would want to use for decision making and conduct a pilot project to see how useful these statistical relationships would be for reservoir management decisions. This expert said that this pilot effort would not be a difficult undertaking and could be accomplished for roughly $100,000, but the Corps would need to coordinate closely with NWS.

According to experts who participated in our meeting, reservoir managers in several basins throughout the United States currently use probabilistic techniques to manage reservoirs. For example, reservoir managers in the Occoquan River basin (which provides drinking water to Fairfax County, Virginia) successfully used Hirsch method forecasts in the 1970s to support implementation of drought mitigation measures that were less onerous than were thought necessary based on their deterministic forecasts. In 2009, the New York City Department of Environmental Protection (DEP) began developing a tool that incorporates both Hirsch method and NWS forecasts to help manage drinking water reservoirs that serve 9 million residents of the city and surrounding areas, as well as other competing demands on the system, such as release requirements on the Delaware River, flood control, and recreational fisheries. DEP officials said that a similar tool could help the Corps better manage other reservoir systems, such as along the Missouri River. DEP officials said the tool, which cost about $8 million, was developed in stages: in late 2010, it included modeling based on the Hirsch method and, in November 2013 it incorporated NWS hydrologic ensemble forecasts. DEP officials said the tool uses these ensemble forecasts to model potential reservoir management scenarios to meet water quality goals—such as reducing the amount of sediment in the city’s drinking water—without having to install expensive new water filtration plants. DEP officials also said the tool supports reservoir operations decisions, including preemptive releases in advance of large storm events to create space in the reservoirs and releases to support downstream communities. DEP officials said the tool is more effective than their previous method of forecasting, which, much like the Corps’ Missouri River forecasting, used historical data and runoff volume calculations. DEP officials said that ensemble forecasts have been effective in modernizing management of their reservoir system by reducing uncertainty and helping them to better assess risk and make informed decisions, and that similar systems could help the Corps make risk-based decisions about reservoir releases informed by real probabilities.

Corps officials told us that they have not considered using probabilistic techniques, such as the Hirsch method or NWS forecast products, in the Missouri River system because they are not sure the benefits would outweigh the difficulty of creating the models or explaining the new methods to their stakeholders. Corps officials told us that deterministic forecasts are easy to maintain and simpler to explain to stakeholders than probabilistic methods. These officials also said that their current methods work well in all but the most extreme events and are a more efficient use of their limited staff resources. Corps officials said that assigning probabilities to their current runoff forecast could give them and their stakeholders more certainty about the likelihood that a very high or very low runoff year will develop. However, the Corps would nonetheless have to select one of the forecasts on which to base their real-time operations. This would, in effect, require the application of the same engineering judgment that is used in their deterministic forecast, according to Corps officials. In addition, Corps officials said that Missouri River basin stakeholders who see the results of the Corps models would face similar challenges, but they would not have the years of engineering expertise to determine how to act on a range of potential very high or very low releases with the same likelihood of occurrence. Furthermore, Corps officials said that basing system operations on probabilistic forecasts would require changes to many of their current operational procedures, and thus changes to the Master Manual.

However, experts who participated in our meeting agreed that Corps should investigate probabilistic techniques. Some of the experts said that the Corps can achieve better outcomes for the basin using probabilistic techniques than their current methods. One of the experts noted that using probabilistic techniques can help the Corps focus on the risks of flood and drought in less extreme years than 2011, which may help to increase the benefits to basin stakeholders from the six mainstem dams. The experts also agreed on the importance of evaluating any new forecasting methods using hindcasting, which uses historical weather and stream information to determine how effectively a given forecasting approach would have predicted past events. One of the experts said that hindcasts are a powerful tool for showing the Corps and their stakeholders that a new probabilistic forecasting model would have provided useful information in the past, will be able to provide useful information in the future, and that changes to operating rules based on the ensemble forecasts would create win-win outcomes.

Stakeholders Were Generally Satisfied with Corps’ Communication during the Flood and Drought

During both the 2011 flood and the subsequent drought, the Corps communicated with Missouri River stakeholders in a variety of ways, which most stakeholders we interviewed said were effective. Nearly all stakeholders we interviewed were generally satisfied with the Corps’ communication with them during these events, saying that the information they received from the Corps was timely and sufficient for their purposes.

Stakeholders Received Information from the Corps in a Variety of Ways and Most Said Methods Were Effective

Stakeholders we interviewed said that the Corps communicated with them in a variety of ways during both the flood and drought. For most stakeholders, the Corps was the primary source of information during the flood, but fewer than half of these stakeholders used the Corps as their primary source of information during the drought. In both events, most stakeholders said that the methods through which they received information from the Corps were the most effective way for the Corps to communicate with them.

As shown in table 3, many of the communication methods mentioned by the stakeholders we interviewed were used by the Corps in both the flood and drought. One of those methods was the Corps’ conference calls. According to Corps officials, these calls began in May 2011 as a way to provide daily updates and information during the flood to congressional representatives, state and local officials, and the news media about (1) planned water releases from the six Corps dams on the Missouri River, (2) weather forecasts, and (3) the Corps’ repair schedule. The conference calls also gave these stakeholders an opportunity to ask the Corps questions about their own communities. In 2012 and 2013, the Corps continued these calls on an almost monthly basis during the winter and spring runoff season. Nearly all of the stakeholders we interviewed reported that they took part in the conference calls at some point. Stakeholders also reported receiving information from the Corps via e- mail. For example, a few stakeholders mentioned receiving information during the flood in e-mails from a Corps district official who updated them on release rates and provided other useful information. One of these stakeholders reported that, if they had questions, they could contact this official by e-mail and would obtain a response. During the drought, one stakeholder said that he received monthly updates about the reservoir by e-mail. In addition, some stakeholders mentioned communication methods specific to the drought. For example, four stakeholders mentioned receiving a letter from the Corps that was sent to water intake owners in April 2013 warning of a potential need to reduce water releases from Gavins Point dam to as low as 9,000 cfs in the fall of 2013 as a conservation measure.steps to ensure that their intakes would operate at the low levels.

Nearly All Stakeholders Were Generally Satisfied with the Corps’ Communication during the Flood

Nearly all of the stakeholders we interviewed were generally satisfied with the Corps’ communication during the flood. The information they received from the Corps helped them take a number of actions, as shown in table 4. For example, officials from several state and local governments agencies said they used information from the Corps to identify infrastructure that would be affected by floodwaters and either protect it by sandbagging or, if possible, relocate it out of the flood zone. Other stakeholders, such as one state parks department official, used the Corps’ information to plan for facility closures or remove equipment to prevent damage. Stakeholders also shared information they received from the Corps with other people. For example, one state agency official shared the Corps information with others in his agency, with other state agencies, and with farmers and levee districts throughout the state. Similarly, one nonprofit organization official disseminated information to its members who are located throughout the Missouri River basin.

Most of the stakeholders we interviewed said that the information they received from the Corps was sufficient for their purposes. For example, one state agency official praised the inundation maps received from the Corps for having considerable amounts of useful information, and also praised the Corps’ status updates about which levees were in danger of failing or had already failed due to the effects of the flood. In contrast, several stakeholders said that the information was not sufficient for their purposes. For example, 8 stakeholders said that the Corps changed their water release estimates from the dams too frequently. Specifically, one stakeholder reported that during May 2011, the Corps revised its release estimates upward five times in a 2-week span. The final estimate, provided in early June, was nearly three times the size of the first estimate from mid-May. This stakeholder said his agency had to revise flood control plans after each estimate, and that his agency eventually decided to plan for the highest possible releases rather than frequently revising its plans.

Most of the stakeholders we interviewed said that the Corps’ information was generally timely. For example, one state agency official said that the Corps provided data in a timely manner, and that the agency was always informed when the Corps planned to increase releases and by how much. Similarly, a local government official said that while he did not always like what the Corps was telling him, the Corps always provided accurate and timely data about when the releases were going to change, and by how much. In contrast, 5 stakeholders said that the Corps information was generally untimely during the flood, and 5 others said that the Corps’ information was untimely in the beginning stages of the flood but improved over time. Seven of the 10 stakeholders who cited problems with timeliness were located in North and South Dakota, the states where five of the six Corps dams are located.

Although nearly all of the stakeholders we interviewed were satisfied with the Corps’ communication during the flood, most stakeholders offered at least one suggestion for how the Corps could improve its communication during future floods. However, there was no consensus among stakeholders on these suggestions. A few stakeholders suggested that the Corps communicate any uncertainty associated with its release estimates. One of these stakeholders explained that this could include the Corps telling them the worst-case scenario for releases, not just the most likely case. Corps officials expressed concerns about communicating this type of information with their release estimates since it could be difficult to use without the appropriate context. In addition, a few stakeholders suggested that the Corps hold a conference call specifically for agency officials. One of these stakeholders said that having the news media on the call made it difficult to discuss sensitive response-related information. Corps officials also said that having separate conference calls for agency officials and elected officials and media is feasible and that they would consider this in the future. In contrast, some stakeholders said there is nothing more the Corps could do to improve communication in the event of another flood. A few stakeholders identified actions the Corps took during the flood that they appreciated such as having a Corps official embedded in their emergency operations center.

Nearly All Stakeholders Were Generally Satisfied with Corps’ Communication during the Drought

Nearly all of the stakeholders we interviewed who were in contact with the Corps were generally satisfied with the Corps’ communication during the drought. These stakeholders used the information they received to take a number of actions, as shown in table 5. For example, officials from one local government used the information they received from the Corps to analyze options to update their intake to ensure that it would remain operational if the Corps’ winter releases dropped to 9,000 cfs. As was the case during the flood, some of these stakeholders said that they shared information with others during the drought. For example, one nonprofit official said that he disseminated Corps information to Missouri River navigators, shipping companies, and agricultural producers, among others. Nearly all of these stakeholders said that the Corps gave them sufficient information for their purposes. One state agency official said that he had a “good handle” on the problems caused by drought and that the Corps has explained things well.

Nearly all of the stakeholders we interviewed said that the Corps communicated with them in a timely fashion. However, 8 of these stakeholders said that the issue of timeliness is different in a drought than during a flood. As one stakeholder explained, droughts do not present the same type of near-term safety issues that must be dealt with immediately. Instead, droughts stress the water system at a lower level over a longer period of time. Although nearly all of the stakeholders we interviewed were satisfied with the Corps’ communication during the drought, some stakeholders offered suggestions for how the Corps could improve its communication during future droughts. However, there was no consensus among stakeholders on these suggestions. The most common suggestion, mentioned by several stakeholders was that the Corps make better use of technology in presenting information, such as by improving their website. Corps officials acknowledged the importance of a user friendly website and that it can be hard to find information on their current website. These officials noted that redesigning a website would take a significant effort and that doing so is not a high priority given current staffing and funding levels. However, these officials mentioned that some website improvements have been made recently, such as adding a map of streamgages within the basin with links to the raw data, which could be on Corps, U.S. Bureau of Reclamation, USGS, or NWS websites. Most stakeholders did not have suggestions for the Corps on improving communication in future droughts.

Conclusions

The extreme flood of 2011 followed by severe drought in 2012 and 2013 created challenging conditions on the Missouri River for the Corps. Experts who participated in our meeting agreed that the Corps made appropriate release decisions during the flood and drought, given the circumstances. However, the experts agreed that techniques such as probabilistic forecasting have the potential to improve the Corps’ ability to make release decisions in nonextreme events. Probabilistic forecasting could allow the Corps to make better risk-based decisions and provide increased benefits to residents in the Missouri River basin, such as higher reliability of water supply, increased flood protection, small increases in hydropower production, and easier implementation of variable river flows to create fish and wildlife habitats. However, the Corps currently uses deterministic forecasting methods, and Corps officials told us that they have not assessed the pros and cons of using probabilistic techniques because they are not sure that the benefits of more sophisticated probabilistic modeling techniques would outweigh the difficulty of creating the models or explaining the new methods to the stakeholders in the Missouri River basin. New forecasting methods can be evaluated—using a technique known as hindcasting—to determine how effectively a new forecasting approach would have predicted past events. According to the experts, hindcasting is a powerful tool for showing the Corps and their stakeholders that a new probabilistic forecasting model would have provided useful information in the past, will be able to provide useful information in the future, and that changes to operating rules based on the ensemble forecasts would create win-win outcomes.

Recommendation for Executive Action

To ensure the U.S. Army Corps of Engineers considers a full range of forecasting options to manage the Missouri River mainstem reservoir system, we recommend that the Secretary of Defense direct the Secretary of the Army to direct the Chief of Engineers and Commanding General of the U.S. Army Corps of Engineers to evaluate the pros and cons of probabilistic forecasting techniques that could improve the U.S. Army Corps of Engineers’ ability to anticipate weather developments, and to evaluate whether forecasting changes are warranted.

Agency Comments

We provided a draft of this product to the Departments of Commerce, Defense, and the Interior for comment. In its written comments, reprinted in appendix III, the Department of Defense concurred with our recommendation and noted that it will take steps to address the recommendation. The Department of Commerce provided technical comments that we incorporated as appropriate. The Department of the Interior had no comments.

We are sending copies of this report to the appropriate congressional committees; the Secretaries of Commerce, Defense, and the Interior; and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-3841 or fennella@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: List of Experts and Discussion Questions

Experts

Larry Cieslik, retired, U.S. Army Corps of Engineers (Corps)

Discussion Questions

Does the Corps have access to adequate data and forecasting information that allows them to make timely release decisions during droughts and floods? If not, why? What are the consequences of not having this data and forecasting?

If data elements needed for decision making are missing, what technical challenges, if any, are there in collecting these data?

Based on information presented by the Corps about how they use data collected by federal and state agencies, are those data being used to support reservoir operations decisions, as appropriate? Why, or why not?

How should the Corps make reservoir operations decisions in the face of uncertainty in runoff forecasts? How, if at all, should the Corps incorporate uncertainty in runoff forecasts into information provided to basin stakeholders or the public?

What are the pros and cons of the Corps’ policy to regulate releases based on “water on the ground”?

What steps, if any, could the Corps take to improve runoff forecasting?

In the 2011 flood, were there additional reservoir release actions the Corps might have taken to better manage flood risks and, if so, what may have been the consequences of not taking these actions? What constraints might exist on those actions?

In the 2012 and 2013 drought, were there additional reservoir release actions the Corps might have taken to better manage drought risk and, if so, what may have been the consequences of not taking these actions? What constraints might exist on those actions?

Does the Corps have appropriate flexibility to regulate the river to manage risk of flood and drought? What are the pros and cons of the current amount of flexibility the Corps has?

During low water conditions, the Corps works to balance the competing interests of the eight authorized purposes of the Missouri River reservoirs. How well is the Corps balancing these purposes and what, if any, improvements could the Corps make? What constraints might exist on those actions?

The most recent changes to the Master Manual raise the threshold below which the Corps regulates the system in drought conservation mode. What are the pros, and cons, of this new threshold?

Appendix II: Objectives, Scope, and Methodology

This report describes (1) experts’ views on the Corps’ release decisions during the 2011 flood and 2012 and 2013 drought; (2) additional actions, if any, experts recommend to improve the Corps’ ability to make future release decisions; and (3) stakeholders’ views on how the Corps communicated information during the flood and drought and improvements, if any, stakeholders suggest.

To address all of our objectives, we reviewed relevant laws, including the Flood Control Act of 1944 that authorized the construction of dams along the Missouri River. In addition, we reviewed the documents that the U.S. Army Corps of Engineers (Corps) uses to guide their release decisions, including the Missouri River Mainstem Reservoir System Master Water Control Manual (Master Manual) (last updated in 2006) and the Annual Operating Plans for 2010-2011, 2011-2012, and 2012-2013. We also reviewed documents produced or commissioned by the Corps that describe the details of the Corps’ release decisions during the flood and drought, including “Summary of Actual 2011 Regulation,” “Summary of Actual 2012 Regulation,” and the December 2011 Independent Technical Review Report. In addition, we reviewed National Oceanic and Atmospheric Administration (NOAA) and U.S. Geological Survey (USGS) documents about existing hydrologic data collection systems and National Weather Service (NWS) documents about weather and flood forecasts during 2011. We conducted a technical review of the December 2011 Independent Technical Review Report and the NOAA Climate Assessment Report to ensure that the methodologies used were appropriate to support the conclusions reached. We also interviewed Corps officials at the Missouri River Basin Water Management Division who are responsible for making release decisions and conducted a site visit to the Oahe project in South Dakota to gather information about dam operations and water levels during the 2011 flood.

To obtain expert views on the Corps’ information and release decisions, we convened a meeting of experts to discuss these issues. This meeting was held at the National Academy of Sciences (NAS) in February 2014, and staff at NAS assisted in identifying experts for the meeting. To identify the experts appropriate for this meeting, NAS staff solicited nominations from: current and former members of the NAS Water Science and Technology Board; current and former members of the Water Science and Technology Board study committees; select members of the NAS and the National Academy of Engineering; National Research Council staff; and other experts. Experts were selected based on knowledge of: (1) reservoir operations and river basin system modeling (both in the United States and abroad); (2) weather conditions and forecasts, and river stage forecasting, specifically in the Missouri River system; and (3) modeling of the relationship between precipitation and runoff; and Corps operational decisions. The experts identified by NAS included individuals with a broad set of viewpoints and knowledge, including experts from federal and state government agencies, the private sector and consultants, academia, and retired federal water and engineering experts. The range of the experts’ expertise included reservoir system operations and modeling, hydrology and hydraulics, civil engineering, meteorology, rainfall-runoff modeling, and Corps reservoir operations.

The nine experts were evaluated for conflicts of interest. A conflict of interest was considered to be any current financial or other interest that might conflict with the service of an individual because it (1) could impair objectivity and (2) could create an unfair competitive advantage for any person or organization. All potential conflicts were discussed by NAS and GAO staff. The nine experts were determined to be free of conflicts of interest, and the group as a whole was judged to have no inappropriate biases. See appendix I for a list of the experts and the questions discussed during the 2-day meeting.

The 2-day expert meeting began with a 1-hour presentation by the Chief of the Missouri River Basin Water Management Division and an opportunity for the experts to ask questions. After questions, the Corps’ representative left the meeting and experts began discussing the Corps’ data, forecasts, and release decisions. The meeting was recorded and transcribed to ensure that we accurately captured the experts’ statements, and we reviewed the transcripts as a source of evidence. We analyzed the transcripts to identify key statements the experts made regarding the Corps’ data, forecasts, and release decisions. We sent these statements, via e-mail, to each of the experts to ensure that they all agreed with our characterization of the findings of the 2-day meeting. We received replies from all nine experts generally agreeing to the statements and making some additions and clarifications, which we incorporated as appropriate.

To obtain stakeholders’ views of the Corps’ communication during flood and drought, we used a standard set of questions to interview a nonprobability sample of 45 stakeholders in the Missouri River basin. We identified these stakeholders based on our preliminary interviews and sought to include organizations from each of the seven states included in our review and related to the eight authorized purposes of the Missouri River reservoir system. For example, to obtain perspectives related to flood control, we identified communities of different sizes along the river and interviewed officials from those communities responsible for emergency management and public works. These local communities included cities as large as Kansas City, Missouri (more than 450,000 people), and as small as Fort Pierre, South Dakota (roughly 2,000 people). We also spoke with state emergency management agencies in each of the seven states. Similarly, to obtain perspectives related to navigation, we interviewed a barge operator, a terminal operator, and nonprofit organizations that advocate for navigation along the Missouri River. In addition, to obtain perspectives related to fish and wildlife, we interviewed an official at the U.S. Fish and Wildlife Service active in addressing endangered species in the Missouri River basin and state officials involved in fish and wildlife issues from each of the seven states. See table 6 for a complete list of the stakeholders we interviewed about the Corps’ communication efforts.

To obtain the views of these 45 agencies and organizations, we developed a structured interview guide that included questions about the Corps’ mode of communication with stakeholders (for example, via e- mail, phone, or letter), and how effective that communication was during the recent flood and drought. We conducted two pretests of the questionnaire and made appropriate changes based on these pretests. We used the structured interview guide to obtain the views of these organizations either via phone or, in cases where the respondent preferred, via e-mail. We analyzed the responses to provide insight into organizations’ views on the Corps’ communication during the flood and drought. These stakeholder interviews provide key insights and illustrate opinions concerning Missouri River basin issues; however the results of our interviews cannot be used to make generalizations about all views. In some cases, interview questions were skipped, as appropriate. For example, some stakeholders did not interact with the Corps during the drought, and we did not ask these stakeholders questions about the Corps’ communication during that time.

We conducted this performance audit from August 2012 to September 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: Comments from the Department of Defense

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Vondalee R. Hunt (Assistant Director); Cheryl Arvidson; Elizabeth Beardsley; Michelle Cooper; Cindy Gilbert; Geoffrey Hamilton; Armetha Liles; Perry Lusk, Jr.; and Janice Poling made key contributions to this report.