Open-File Report 2011–1236

U.S. Department of the Interior

U.S. Geological Survey

Objectives, Priorities, Reliable Knowledge, and 

Science-Based Management of Missouri River 

Interior Least Terns and Piping Plovers
Cover. Background photo: Sandbar habitat on the Gavins Point Reach of the Missouri River, South Dakota. Inset photos (bottom left 

to top right): Least tern nest, Garrison Reach of the Missouri River, North Dakota. Newly hatched least tern chick, Gavins Point Reach 

of the Missouri River, South Dakota. Newly hatched piping plover chicks, Lake Sakakawea, North Dakota. Adult piping plover with two 

fledglings, Lake Sakakawea, North Dakota.
Objectives, Priorities, Reliable Knowledge, 

and Science-Based Management of 

Missouri River Interior Least Terns and 

Piping Plovers

By Mark Sherfy, Michael Anteau, Terry Shaffer, Marsha Sovada, and 

Jennifer Stucker

Open-File Report 2011–1236

U.S. Department of the Interior

U.S. Geological Survey
U.S. Department of the Interior

KEN SALAZAR, Secretary

U.S. Geological Survey

Marcia K. McNutt, Director

U.S. Geological Survey, Reston, Virginia: 2011

For more information on the USGS—the Federal source for science about the Earth, its natural and living 

resources, natural hazards, and the environment, visit http://www.usgs.gov or call 1–888–ASK–USGS.

For an overview of USGS information products, including maps, imagery, and publications, 

visit http://www.usgs.gov/pubprod

To order this and other USGS information products, visit http://store.usgs.gov

Any use of trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the 

U.S. Government.

Although this report is in the public domain, permission must be secured from the individual copyright owners to 

reproduce any copyrighted materials contained within this report.

Suggested citation:

Sherfy, M., Anteau, M., Shaffer, T., Sovada, M., and Stucker, J., 2011, Objectives, priorities, reliable knowledge, and 

science-based management of Missouri River interior least terns and piping plovers: U.S. Geological Survey Open-File 

Report 2011–1236, 26 p.

ISBN 978–1–4110–3236–2
iii

Contents

Abstract...........................................................................................................................................................1

Slide 1—Introduction....................................................................................................................................2

Slide 2—Abstract...........................................................................................................................................3

Slide 3—Science-Based Missouri River Management ..........................................................................4

Slide 4—Why is Science a Common Theme?...........................................................................................5

Slide 5—Missouri River Recovery Program Science Guidance ...........................................................6

Slide 6—Tools of Management for Listed Species..................................................................................7

Slide 7—Efficiency vs. Effectiveness.........................................................................................................8

Slide 8—What is Data?.................................................................................................................................9

Slide 9—What is an Objective?.................................................................................................................10

Slide 10—Example–Piping Plovers...........................................................................................................11

Slide 11— Example–Piping Plovers..........................................................................................................12

Slide 12— Example–Piping Plovers..........................................................................................................13

Slide 13—Prospective vs. Retrospective Questions .............................................................................14

Slide 14—Types of Goals............................................................................................................................15

Slide 15—Types of Data Required ............................................................................................................16

Slide 16—Spatial Scales ............................................................................................................................17

Slide 17—Tern and Plover Monitoring Program (TPMP)......................................................................18

Slide 18—Where the TPMP has Done Well............................................................................................19

Slide 19—Future of Tern and Plover Monitoring....................................................................................20

Slide 20—1) Set Objectives........................................................................................................................21

Slide 21—2) Make Predictions and Test Them.......................................................................................22

Slide 22—3) Ask the How Questions........................................................................................................23

Slide 23—4) Manage to Learn...................................................................................................................24

Slide 24—5) Prioritize Information Needs ...............................................................................................25

Slide 25—Relationship to Missouri River Recovery ..............................................................................26
iv
Objectives, Priorities, Reliable Knowledge, and ScienceBased Management of Missouri River Interior Least Terns 

and Piping Plovers

By Mark Sherfy, Michael Anteau, Terry Shaffer, Marsha Sovada, and Jennifer Stucker

Abstract

Supporting recovery of federally listed interior least tern (Sternula antillarum athalassos; tern) and piping plover (Charadrius melodus; plover) populations is a desirable goal in management of the Missouri River ecosystem. Many tools are 

implemented in support of this goal, including habitat management, annual monitoring, directed research, and threat mitigation. 

Similarly, many types of data can be used to make management decisions, evaluate system responses, and prioritize research 

and monitoring. The ecological importance of Missouri River recovery and the conservation status of terns and plovers place a 

premium on efficient and effective resource use. Efficiency is improved when a single data source informs multiple high-priority 

decisions, whereas effectiveness is improved when decisions are informed by reliable knowledge. Seldom will a single study 

design be optimal for addressing all data needs, making prioritization of needs essential. Data collection motivated by wellarticulated objectives and priorities has many advantages over studies in which questions and priorities are determined retrospectively. Research and monitoring for terns and plovers have generated a wealth of data that can be interpreted in a variety 

of ways. The validity and strength of conclusions from analyses of these data is dependent on compatibility between the study 

design and the question being asked. We consider issues related to collection and interpretation of biological data, and discuss 

their utility for enhancing the role of science in management of Missouri River terns and plovers. 

A team of USGS scientists at Northern Prairie Wildlife Research Center has been conducting tern and plover research on 

the Missouri River since 2005. The team has had many discussions about the importance of setting objectives, identifying priorities, and obtaining reliable information to answer pertinent questions about tern and plover management on this river system. 

The objectives of this presentation are to summarize those conversations and to share insights about concepts that could contribute to rigorous science support for management of this river system.
2 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 1
Slide 2 

3

Slide 2
4 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 3

This presentation initiates a session on management of least terns and piping plovers, but it is really about science and how 

it is conducted in support of the many important decisions that are made while managing the Missouri River system. There are 

many documents and plans that guide how this system is managed including the Missouri River Biological Opinion, various 

recovery and management plans, the Missouri River Ecosystem Recovery Plan, National Environmental Policy Act documents, 

and others. Throughout these there is a recurring theme of importance of science in managing the system. In fact, the Missouri 

River Natural Resources Conference has a principal role of exchanging scientific information, which is a key expression of how 

important science is on the Missouri River.
Slide 4 5

Slide 4

With science being such a common element in river management, we might ask a seemingly nonsensical question—why is 

science a common theme? The most obvious answer is that it is important to make the best decisions when faced with a choice, 

whether it is selecting a management action, making a decision about allocating resources, or reaching a conclusion about 

whether a management goal has been reached. It is not necessary to use science to make a decision. Alternative options would 

be to select the cheapest, easiest or most popular pathway forward. However, if ecological recovery of the river is the desired 

endpoint, then science is the most logical tool to support decisionmaking.
6 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 5

Looking more closely at the Missouri River Recovery Program, the importance of scientific information quickly becomes 

evident. The Program’s science guidance prioritizes science by stating “Incomplete knowledge and understanding of this complex river system that spans a large, diverse landscape creates challenges for making good recovery decisions.” This presentation 

will look at several components of science-based decisionmaking and reflect on how they are being implemented, specifically 

with reference to least terns and piping plovers.

The last four words in this guidance are a key reminder of the context for this discussion. Not only is it important to make 

good decisions, but it is important that they be focused on recovery of the river system and its sensitive species.
Slide 6 7

Slide 6

Many tools are used to implement the Missouri River Recovery Program guidance as it relates to threatened and endangered species on the Missouri River. These include habitat management (such as sandbar construction), water management (such 

as manipulation of dam releases to minimize flooding of nests), annual monitoring of habitat and productivity, directed research 

related to tern and plover biology, and mitigation of threats that may cause mortality or other negative effects. Each of these has 

its own unique set of decisions that go into implementation and influence ultimate outcomes. Each of these also generates some 

sort of response, which may be a biological response, a physical response, or, more likely, many complex responses. Additional 

challenges arising from these responses relate to whether or not they can be detected and properly interpreted, and whether the 

resulting information can be used appropriately to make subsequent decisions. However, as usual, there is a catch. Resources are 

limited, which constrains how and even if these tools are implemented.
8 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 7

Therefore, it becomes very important to consider the tradeoff between efficiency and effectiveness of data used to make 

decisions. Efficiency can be maximized by using a single data source to make multiple decisions, whereas effectiveness of a 

given decision can be maximized if the decision is based on scientifically valid, directly applicable information. This kind of 

information is also known as reliable knowledge. Ideally we’d like to be both efficient and effective, but it is rarely the case that 

a single type of data can inform a wide variety of decisions. Therefore, it becomes important to understand the data needs of 

each potential decision. This tradeoff between effectiveness and efficiency is especially important for the Missouri River system 

because of the ecological importance of recovery, the conservation status of key resources such as threatened and endangered 

species, and the limited resources available to collect data.
Slide 8 9

Slide 8

To further understand this challenge, let us reflect a moment on what we mean by data. David Anderson wrote a paper in 

2001 titled “The need to get the basics right in wildlife field studies”, and in that paper he defined data as implying information 

content with respect to some objective. This way of thinking emphasizes a couple of important points. First, numbers are not 

always data. Numbers are only data when they are supported, collected, and interpreted in reference to an objective. Second is 

the sample size trap. Most of us believe more is better when it comes to numbers, but simply having lots and lots of numbers 

does not guarantee good data. If the fundamentals, such as the guiding objectives or the sampling design, are flawed, then no 

quantity of numbers can produce good data. Third is the underlying objective that motivated data collection in the first place. If 

the objective is not clearly stated and well understood, then there is a reduced likelihood the data will be, which has subsequent 

implications for whether decisionmaking is based on the best information.
10 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 9

Based on that definition, data and their supporting objective are inseparable. Therefore, it is also important to understand 

what an objective is. An objective is a clearly worded statement that identifies a desired outcome in measurable terms that is 

stated in advance of an action. The context presented in this report is relevant mostly to tern and plover monitoring, but these 

same ideas apply in conducting research and in implementing management actions.
Slide 10 11

Slide 10

To illustrate these concepts, consider an information need about abundance of piping plovers. Here are two potential ways 

to phrase an objective to drive data collection on piping plover abundance. Objective A recognizes the need for information on 

abundance of piping plovers, but it’s very general and leaves a lot of uncertainties. Objective B, however, provides a lot more 

information. The very first word is important—it tells us an estimate is needed, as opposed to a complete census of piping plovers. The second word is also important—it tells us these estimates are needed annually. Next, Objective B says that breeding 

population will be measured. This is much more specific about information needs than the word “abundance” in Objective A. 

Finally, Objective B states the spatial scale at which the estimate is needed—segment X of the Missouri River.
12 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 11

The importance of clearly stated objectives can be seen by examining some data collected on breeding population of piping plovers on the Missouri River. This graph shows estimates of the breeding population for each of 4 years on a segment of 

the river. This graph might prompt a couple of management-relevant questions, such as “what was the breeding population of 

plovers in 2008?” It is simple to read a value off the graph and make management decisions based on that information. One also 

might be interested in the trend in breeding population between 2006 and 2009 and feel pretty certain it declined based on this 

graph.
Slide 12 13

Slide 12

However, imagine what would happen if a second measure of breeding population from the exact same area was available, but it was measured using different methods. Now that question about the breeding population in 2008 becomes a lot less 

certain, because there are two very different estimates—one approximately 375 birds, and the other approximately 625 birds. 

Now it becomes really important to understand the differences in how these two datasets were collected and what objectives 

motivated collection of the data. Both datasets illustrate similar patterns of declining plover populations, but very different 

ones about the true number of breeding birds. Thus, it should be evident that simply labeling the response variable (in this case, 

breeding population) is insufficient to interpret a dataset.

In the prior slide, when only one dataset was presented, it was easy to rapidly make several conclusions about both trends 

and absolute abundance of breeding birds. In practice it is very rare that two complementary datasets are available to force 

thinking about how and why data were collected, but understanding the motivating objectives for a dataset should always be a 

priority when interpreting data. 
14 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 13

It is often tempting, and sometimes useful, to pose questions after a dataset has been collected, which might be labeled as 

a retrospective question. A prospective question is one that is stated in clear and measurable terms in advance of data collection, 

enabling a sampling design to assess predictions and competing hypotheses. In contrast, a retrospective objective is one stated 

after the data are already in hand that may be only loosely related to the original motivating objective. This can lead to inferences that are weaker and knowledge that is less reliable.

Does this mean that retrospective questions should not be asked and cannot be answered? No, but it does mean that caution 

is needed to ensure that retrospective questions remain aligned with the original objectives and design principles that motivated 

the data collection.
Slide 14 15

Slide 14

Several different types of objectives are relevant to monitoring terns and plovers on the Missouri River. Each of these 

implies a different response variable that may be measured at a different scale.

Legal mandates include things like the Biological Opinion, which has very familiar goals for fledge ratios, suggesting measures of fledglings and breeding population are needed for the river system as a whole.

Species and population recovery plans for terns and plovers provide breeding population targets for the midcontinent population of both species. 

Ecosystem recovery efforts such as the Missouri River Ecosystem Recovery Program are looking at system-wide targets for 

populations and productivity of terns and plovers. These kinds of targets will be used to evaluate success of ecosystem restoration, and they’re different in spatial scale than the targets used in recovery plans or the Biological Opinion.

There are many management actions made to benefit terns and plovers, such as building sandbars, managing vegetation, 

caging nests, and managing predation. Each of these is taken at a different spatial scale and has a unique desired outcome, and 

these ought to be taken into account when setting objectives for monitoring.

Detecting and responding to threats is another objective that might require localized supporting data that matches the spatial extent of the threat. 

And finally, river management decisions, such as manipulating flow rates to avoid loss of nests, might require information 

about everything that is happening at a given band of elevations that are affected by a water management decision.
16 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 15

As diverse as these types of objectives are, the types of data needed to address them are equally diverse. Some objectives 

might require a census or a total count of individuals, whereas others may only require an estimate of mean and variation in 

abundance. It might not be necessary to know the absolute value of abundance, but just the relative abundance of individuals 

between two areas or two habitat types. It might not even be necessary to measure abundance at all as long as it is possible to 

detect trends, and it might not be necessary to quantify trends as long as one can determine where individuals are present and 

where they are absent. There are profound differences in how these different types of data are collected, and choosing the correct 

data type is linked intimately with the supporting objective.
Slide 16 17

Slide 16

Likewise, there are many spatial scales at which data can be collected, ranging from very fine scales like nests or sandbars, to very coarse scales such as river reaches, lakes, or the entire Missouri River system. It might be necessary to sample in 

between these extremes, such as at the scale of a certain management action or within a certain range of elevations. Again, there 

are profound differences in how data are collected at each of these scales, and a clearly articulated objective will identify the 

proper spatial scale to ensure resulting data are appropriate for inference at the correct scale and informing the decision at hand.
18 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 17

So how does all of this apply to the tern and plover monitoring program on the Missouri River? The program is clearly an 

extremely rich resource that has produced an abundance of numbers, but one of the uncertainties is how much “data” is contained in those numbers. Teasing the data out of the monitoring program requires an understanding of the objectives that have 

guided the program since its inception. Likewise, making adjustments to the program for the future requires an understanding of 

current and future objectives.
Slide 18 19

Slide 18

Despite these uncertainties, there are several areas where the monitoring program has done well. First, the program has 

assembled unique datasets on Federally listed species through its prioritization of long-term monitoring. Second, by complementing long-term monitoring with directed research, the program has shown that diverse sources of scientific information 

are important in decisionmaking. Third, the program has emphasized exchange and use of scientific information as a means to 

promote recovery of the river system.
20 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 19

There has been much discussion recently about future adjustments to the monitoring program, and many discussions 

focused on how data are collected. In the future we will probably find that collecting and analyzing data will be the easy part. 

However, decisions about what data should be collected and why will likely be a necessary and much more difficult precursor.
Slide 20 21

Slide 20

The concepts illustrated in this presentation can be summarized in five points that might be considered as the tern and 

plover monitoring program moves forward. First, take actions with a purpose by setting objectives that are clearly articulated, 

measurable, and stated in advance. This up-front investment will lead to reliable and defensible conclusions and should be a key 

component of management, research, and monitoring.
22 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 21

Second, is to make predictions and test them. Often it is the case that the likely outcome of a management action can be 

anticipated. If two or more potential outcomes can be articulated along with the associated biological metrics that would be 

associated with each outcome, then implementation and monitoring approaches could be readily designed to distinguish among 

the outcomes and support or refute these predictions. 
Slide 22 23

Slide 22

Third, is asking “how” questions. Often a very simple question, such as “Does process X affect response Y?” has a very 

simple but not very interesting answer—“Yes, it does.” It is far more interesting and relevant to management if the question 

“How does process X affect response Y?” is asked and the study design and data collection are approached accordingly.
24 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 23

Fourth, the program could learn more by implementing management practices in accordance with experimental design 

principles, such as replication, randomization, and experimental controls. There are several examples where the program already 

has taken these steps and many more examples where learning could be enhanced dramatically by subtle, but very important 

adjustments to standard practices.
Slide 24 25

Slide 24

The final, and perhaps most important point, is prioritizing information needs. It is important to understand that not all 

questions can be answered with one dataset, but if objectives are prioritized and resources are committed accordingly, the greatest chance of meeting the most important needs occurs. It may be that the highest priority objective cannot be achieved because 

it is too expensive or beyond the recovery program’s means, which would require priorities to be adjusted, compromises be 

made, and the implications of those changes be understood.
26 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Slide 25

The guidelines for presentations at the Missouri River Natural Resources Conference asked authors to show how results 

were related to Missouri River Recovery. The instructions to presenters contained two statements that were very interesting, very 

encouraging, and very relevant not only to how science is being presented in this venue, but also to the actions to be taken when 

attendees leave the conference on Friday—Objectives should be clearly stated, and conclusions should relate back to objectives. 

This simple and concise statement is an excellent summary of the important points in this presentation.
Slide 25 27

Publishing support provided by:

Rolla Publishing Service Centers 

For additional information concerning this publication, contact:

Director, USGS Northern Prairie Wildlife Research Center

8711 37th Street SE

Jamestown, ND 58401

(701) 253–5566

Or visit the Northern Prairie Wildlife Research Center Web site at:

http://www.npwrc.usgs.gov
28 Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers

Sherfy and others—Objectives, Priorities, Reliable Knowledge, and Science-Based Management of Missouri River Interior Least Terns and Piping Plovers—OFR 2011–1236

Printed on recycled paper