Towards Policy Relevant Environmental Modeling:

Contextual Validity And Pragmatic Models

By

Scott B. Miles

1

Open-File Report 00-401

2000

This report has not been reviewed for conformity with U.S. Geological Survey

editorial standards or with the North American Stratigraphic code. Any use of trade,

product, or firm names is for descriptive purposes only and does not imply endorsement

by the U.S. Government.

U.S. DEPARTMENT OF THE INTERIOR

U.S. GEOLOGICAL SURVEY

1 Department of Geography, Box 353550, University of Washington, Seattle, WA 98125,

smyles@u.washington.edu, http://salish.wr.usgs.gov/smiles/

1
Towards Policy Relevant Environmental Modeling:

Contextual Validity And Pragmatic Models

Scott B. Miles

"Nothing in the world, no object or event, would be true or false if

there were not thinking creatures." (Davidson, 1990, p. 279)

"Ground Truth has given way to Ground Fuzziness." (Openshaw,

1996, p. 765)

Abstract

"What makes for a good model?" In various forms, this question is a question

that, undoubtedly, many people, businesses, and institutions ponder with regards

to their particular domain of modeling. One particular domain that is wrestling

with this question is the multidisciplinary field of environmental modeling.

Examples of environmental models range from models of contaminated ground

water flow to the economic impact of natural disasters, such as earthquakes.

One of the distinguishing claims of the field is the relevancy of environmental

modeling to policy and environment-related decision-making in general. A

pervasive view by both scientists and decision-makers is that a "good" model is

one that is an accurate predictor. Thus, determining whether a model is

"accurate" or "correct" is done by comparing model output to empirical

observations. The expected outcome of this process, usually referred to as

"validation" or "ground truthing," is a stamp on the model in question of "valid"

or "not valid" that serves to indicate whether or not the model will be reliable

before it is put into service in a decision-making context. In this paper, I begin

by elaborating on the prevailing view of model validation and why this view

must change. Drawing from concepts coming out of the studies of science and

technology, I go on to propose a contextual view of validity that can overcome

the problems associated with "ground truthing" models as an indicator of model

goodness. The problem of how we talk about and determine model validity has

much to do about how we perceive the utility of environmental models. In the

remainder of the paper, I argue that we should adopt ideas of pragmatism in

judging what makes for a good model and, in turn, developing good models.

From such a perspective of model goodness, good environmental models should

facilitate communication, convey  not bury or "eliminate"  uncertainties,

and, thus, afford the active building of consensus decisions, instead of promoting

passive or self-righteous decisions. 

2
It's A Model World

To [many] folk, a model is either a miniature version of some realworld object, like an airplane fuselage, or an idealized version of a realworld person ... Models like these are what we might label experimental

models because they constitute material representations of reality that have

either had some real-world features abstracted away (as with a model car

or ship in a bottle) or they have had imperfect real-world features, like

broken noses or bowed legs, replaced by idealized version (such as the

perfect proportions of a fashion model). In either case, the model then

serves to answer by direct experiment certain types of questions, such as

how air flows over an airplane wing under different atmospheric

conditions or how a particular types of clothes ... look when draped over a

perfect shape. (Casti, 1997, p. 13) (Original emphasis)

In introducing us to models in his book Would-Be Worlds, John L. Casti illustrates

how pervasive the use of models are in society and, in fact, how we all use models in our

everyday lives. Casti uses this discussion to introduce the distinction between the

concrete, material models that we use in our everyday lives and the abstract, symbolic

models, like Newton's equation of motion (F = ma), which he implies are the domain of

only scientists and engineers. Ignoring the significance of Casti's distinction for now, his

discussion helps us to see what models in general are: those things that humans interact

with in lieu of interacting with the real (or scarce) thing. We use models in place of the

real thing because interacting with the real thing may be too costly, too dangerous, or may

be practically impossible due to extreme temporal or spatial characteristics. Thus, we can

see that models represent things in a similar way as political representatives serve as

delegates for a group of people: models "stand in for." This broad conception of what

models are subsumes other general definitions of models, such as models as analogies or

metaphors (see Coyne, 1995). Models, of course, can be used over and over again

(although in some cases that may mean rebuilding the model), whereas it may only be

possible to do it once (or some other finite number, including zero) with the real thing.

Knowing, in a general sense, what a model is and why we use models, a likely question

that comes to mind is: "What makes for a good model?"

Undoubtedly, many people, businesses, and institutions ponder various forms of this

question with regards to their particular domain of modeling (don't forget the examples of

models given by John L. Casti). One particular domain that is wrestling with this

question is the multidisciplinary field of environmental modeling. To avoid the

contention of the meaning of "environment," the definition given by the Oxford English

Dictionary will suffice for characterizing what environmental models, in fact, model: the

sum-total of influences which modify and determine the development of life or character.

Examples of environmental models range from models of contaminated ground-water

flow (see Corwin and Loague, 1996) to the economic impact of natural disasters, such as

earthquakes (see Earthquake Spectra, 1997, vol. 13, no. 4). One of the distinguishing

claims of the field is the relevancy of environmental modeling to policy and environmentrelated decision-making in general. Because of the potential social and economic impacts

3
(both positive and negative) of basing a decision on a environmental model, it is easy to

realize the import of the question "What makes for a good model?" for the field of

environmental modeling. Briefly recounting some of my own experiences that led me to

think on these issues will help to illustrate the importance and difficulty of answering this

question, in addition to illustrating the common view of how to determine whether a

model is a good.

Recently, I was tasked with developing computer software for decision-support

related to regional earthquake-induced landslide hazard  the subject of my expertise.

Moderate to large earthquakes can trigger landslides, numbering in the thousands,

distributed over areas exceeding 100,000 km2(Keefer, 1984). Earthquake-triggered

landslides have caused huge economic losses and casualties. In fact, in many

earthquakes, triggered landslides have accounted for most of the damage or casualties.

The most devastating example is likely that of an M=7.8 earthquake in China that

resulted in over 120,000 deaths caused by landslides, over half of the total fatalities

(Close and McCormick, 1922).

The primary user considered in the requirements development of the software was a

utility company, who wanted a means of prioritizing their finances and efforts for

retrofitting and replacing gas pipeline. I encountered an obstacle in the very initial stages

of conceptualizing the software design: "What earthquake-induced landslide model

should the software be designed around?" Even though I was partial to the simulation

that I developed (Miles and Ho, 1999a), I decided to compare several models to try to

find an answer to my question. Each of the models were based on the method of Wilson

and Keefer (1983), most were simplified approaches  regression equations  derived

from applying the method to a generic set of inputs, while one was a direct

implementation of the algorithm. After applying each model to a real-world problem and

obtaining the respective results, I found that I could not arrive at a conclusion, in any

absolute sense, regarding which model was better. So, I proceeded to design the software

independently of a specific model to allow use of any (and many) models in the decision

process (see Miles et al., 1999). During the first review meeting for the project, similar

questions to the one I had asked myself were posed. "What good will having many

models do?" "Why not just use the best model?" In presenting my work at geology and

earthquake hazards conferences, I was met with even more direct questions. "But which

is the correct model?" "Why not just compare each model to data gathered from recent

earthquakes and determine the most accurate model?" A referee of a related paper (Miles

and Keefer, 2000) likened it to eating a fine meal (the model comparison) and not being

given dessert (the correct or accurate model). 

This episode illustrates the potential role of environmental modeling in decisionmaking and, thus, the importance of basing decisions on a good model. This brief

narrative also shows the common expectation, of both scientists and decision-makers, for

determining what makes for a good model: ascertain the most "accurate" or "correct"

model by comparing it to empirical observation. The expected outcome of this process,

usually referred to as "validation" or "ground truthing," is a stamp on the model in

question of "valid" or "not valid" that serves to indicate whether or not the model will be

4
reliable before it is put into service in a decision-making context. Within environmental

modeling, one can trace this commonly held view to two origins. The first is a carry-over

of theory confirmation from physics and chemistry (i.e., "We expect A to exhibit behavior

X. Through controlled experiment, we observe X. Thus, our behavioral prediction is

confirmed.") (Sarewitz and Pielke, 1999). The second, of course, is the equivalent

practice advocated within cartography. Board (1967) noted that "[t]here are ... two major

stages to the cycle of map making. First, the real world is concentrated in model form;

secondly the model is tested against reality." (p. 672) The map then is given the stamp of

approval if there is an "adequate fit" between the map and reality. This common view,

however, must change, on the part of modelers, decision-makers, and stakeholders, as it is

impossible to determine the truth of a model or its reliability prior to its use (Oreskes et

al., 1994). 

In the following section, I will elaborate on the prevailing view of model validation

and why this view must change (and why this view has changed in various intellectual

fields). Drawing from concepts coming out of the studies of science and technology, I

will propose a contextual view of validity that can overcome the problems associated

with "ground truthing" models as an indicator of validity. The problem of how we talk

about and determine model validity has much to do, I will argue, about how we perceive

the overall utility of environmental models. Above, I made the assumption that

environmental modeling is relevant to policy and decision-making. While this may be

obvious to environmental modelers (including myself), the simple question of "why?" is

not only justifiable, but is important in understanding what makes for a good model.

Returning to the quote from Casti (1997), the reason why environmental models are

useful for decision-making is not so different, I think, from the reason why the model

fuselage is useful to aeronautical engineers or the fashion model to fashion designers.

Casti's label of "experimental models" is misapplied; the value of all models lies in the

facilitation of experimentation. In the remainder of the paper, I will argue that pragmatic

perspective can help us to judge what makes for a good model and, in turn, develop good

models. From such a perspective of model goodness, good environmental models should

facilitate communication, convey  not bury or "eliminate"  uncertainties, and, thus,

afford the active building of consensus decisions, instead of promoting passive or selfrighteous decisions. In brief, it is not the environmental model that can be "valid" or "not

valid," rather it is the decision making context and the emergent actions or outcomes.

Truth In Modeling

"Every philosopher you ask will attack logical positivism, either on

details or on some general principle, but it remains the working philosophy

of modern science, medicine, and engineering." (Kosko, 1993, p. 8) 

Although, foundationalist thought  the belief that there is a unique ultimate basis

either in experience or rational thought (Kleindorfer et al., 1998)  has been out of

vogue within the philosophy of science since at least the 1970's thanks to Thomas Kuhn's

The Structure of Scientific Revolutions (1962), as observed by Bart Kosko and illustrated

by my anecdote, such perspectives persist outside of philosophy. Obviously, the

5
epistemology that one adopts, whether knowingly or tacitly as part of one's discipline, has

a profound affect on one's view of model validity. The implications of foundationalist

philosophies of science, including Popper's critical rationalism (falsification) (Giere,

1999), on model validation is stated well by Barlas and Carpenter (1990):

If one adopts a logical empiricist, foundationalist philosophy of model

validation, then validation is seen as a strictly formal, algorithmic,

reductionist, and "confrontational" process. Since the model is assumed to

be an objective and absolute representation [i.e., proposition] of the real

system, it can be either true or false. And given that the analyst uses the

proper validation algorithms, once the model confronts the empirical facts

[data], its truth (or falsehood) is automatically revealed. (p. 157)

Kleindorfer et al. (1998) likens this view of model validation to measles immunization;

once the validity of a model has been established it is immune to further attack. A good

example of such "immunity" among models is that of geologic or soil survey maps.

Unquestionably, maps fit our general view of models as those things that we interact with

in lieu of the actual thing. However, as argued by Board (1967), once "ground truthed,"

maps are typically seen as immune from further evaluation:

It is unlikely that geological or soil surveys would be undertaken twice

for the same area ... The expense of repeating such surveys often means

that they are done but once. Maps of such surveys can therefore be

properly regarded as initial data with which to begin an investigation of

some problem within an areal context. They are as good as raw data

derived directly from observation of reality if such raw data cannot feasibly

be collected. (p. 714-715) (Author's emphasis)

Naomi Oreskes and others point out that veracity (truth) cannot be established for

any proposition except for one that refers to a closed system, such as mathematics,

symbolic logic, or computer code (Oreskes et al, 1994). They provide the following

simple example to demonstrate this statement.

[I]f I say, "If it rains tomorrow, I will stay home and revise this

paper." The next day it rains, but you find that I am not home. Your

verification has failed. You conclude that my original statement was

false. But in fact, it was my intention to stay home and work on my paper.

The formulation was a true statement of my intent. Later, you find that I

left the house because my mother died, and you realized that my original

formulation was not false, but incomplete. It did not allow for the

possibility of extenuating circumstances. Your attempt at verification

failed because the system was not closed. (p. 642) (Author's emphasis)

Systems such as mathematics and computer code are closed because the system's

elements (i.e., symbols) have precisely defined, fixed meanings, which are not contingent

upon empirical input. However, how many  if any  elements in an environmental

system have precisely defined, fixed meanings? Thus, the logical consistency of the

mathematics or computer code of environmental models can be verified, but the models

6
never portray closed systems and so can not be verified. 

Ronald N. Giere argues that models, in fact, are not (linguistic) propositions, and,

thus, the concept of correspondence truth  exact conformance to an antecedently

determinate reality (Rorty, 1991)  has little meaning with regard to models and the

representation of environmental systems (Giere, 1999). Giere uses the familiar example

of maps to illustrate that models represent particular perspectives of the system as

determined by human interests, and these perspectives are necessarily partial.

Imagine, for example, four different maps [models!] of Manhattan

Island: a street map, a subway map, a neighborhood map, and a geological

map. Each ... represents [stands in for] the island of Manhattan from a

different perspective, appropriate, for example, for a taxi driver, a subway

rider, a social worker, and a geologist. (Giere, 1999, p. 81)

Because these map-cum-models, are embedded within each respective practice (e.g., taxi

driving or geology), metaphysical questions of truth and correspondence (e.g., "How do

maps represent physical spaces?") will not lead to meaningful answers. Rather, Giere

suggests naturalistic questions of the sort: "How do we humans manage to use maps to

represent physical spaces?" This is not to suggest the abandonment of realism (i.e., the

world exists independently of the mental) in any way. In fact, Michael Devitt, an

outspoken proponent of realism, now admits "that no doctrine of truth is in any way

constitutive of Realism." (Devitt, 1997, p. 306)

The response of modelers to discussions of establishing the truth of a model as way

of validation is often one of the semantics of truth. Rykiel (1996) suggests that useful

definitions of truth for model validation are "(1) consistent with available data, (2) in

accord with current knowledge and beliefs, and (3) in conformance with design criteria."

(p. 237) Thus, Rykiel argues that model validation is the establishment of truth in the

above sense, under specified conditions. In other words, a modeler specifies some a

priori performance criteria (e.g., some statistical indication of fit), applies the model to a

case study exhibiting appropriate conditions, and compares the output with the best

available data (i.e., empirical observations). If the model meets the specified

performance criteria, the model is worthy of the stamp of "valid" and can be set on the

shelf at the model supermarket for, apparently, general consumption. Rykiel (1996)

stresses that modelers must observe truth-in-labeling "laws" and provide, with the model,

its purpose and the performance criteria and conditions for application that earned the

model a stamp of "valid" so that any potential consumer can make an informed choice.

Unfortunately, this view of model validity is not much better than validity as

correspondence truth. This view of validity presupposes that the phenomenon in question

(1) is observable and measurable, (2) permits the collection of ample data and (3) is

spatially and temporally invariant with respect to conditions not specified by the modeler

(Oreskes, 1998). Miles et al. (2000) demonstrate that these criteria for model validation

do not fully apply to many, if not most, environmental systems, and that it is not

uncommon for models to be applied (sometimes successfully) to conditions or uses

inconsistent with those intended. An example of such "misapplication" could be the

7
common use of geology or similar maps(-cum-models) for supporting the estimation of

various environmental properties, such as soil strength or permeability, which are then

assigned as attributes to the original map units  something that the map's developer(s)

likely never intended or foresaw. The diffusion of geographic information systems (GIS)

and the ease in which many environmental models can be implemented in GIS, has

certainly led to many models being employed for regional analysis when they were not

originally developed with GIS or large geographic scales in mind. Different assumptions

and decisions with respect to model parameterization and model implementation (e.g.,

with different GIS software or data models) effect model output (Miles et al., 2000).

Thus, whether or not output of a particular model matches empirical observations or

meets some specified criteria may be the result of a serendipitous combination of data

and decisions. Lastly, this view of validation may be viable if the models were only used

within a particular scientific community that understands the applications, limitations,

etc. of a given model (Oreskes, 1998) . But because environmental models are promoted

and expected to support policy and decision making, this isn't the case. 

An example of a class of models that is handicapped by the requirements of model

validation as suggested by Rykiel (1996) are models that yield probabilistic output 

inarguably, a popular class of models. The ontological status of probability as applied to

environmental systems is questionable. (An extreme view is that of Kosko (1993, p. 50)

who writes, "The ultimate fraud is the scientific atheist who believes in probability.")

Can probability be observed or measured in environmental systems? Consider an

example from the August, 1999 Izmet, Turkey earthquake (M=7.4). Stein et al. (1997)

calculated a 12% probability of a large earthquake (M > 6.7) occurring within a 30 year

period on the particular segment of the North Anatolian fault that experienced the recent

quake. Obviously, we can make the observation that the earthquake did occur. Does this

observation validate or invalidate the model? Can we determine the "actual" probability?

Is it possible that an a priori performance criteria be specified for this model? For this

class of models, these questions simply don't make sense. Thus, any common view of

model validation doesn't help us to assess whether this model is, in fact, a "good" or

"bad" model. I submit that, to varying degrees, this is the case with all environmental

models.

(Actor-)Networked Models

Before I discuss ways to help us understand what makes for a good model, it is

helpful to reorient our concept of validity as it relates to environmental models. I agree

with Naomi Oreskes who stresses that we should never describe any model as valid

(Oreskes, 1998). To better understand what it is that we can describe as "valid" (or "not

valid") or "successful" (or "partially successful"), it is important to refocus upon the

decision-making process that a potential environmental model would be recruited (i.e.,

selected or developed) to play a part in, rather than just the model itself. Towards this

end, I suggest that it is useful to conceptualize a decision-making context as an actor

network. Actor Network Theory (ANT) developed out of the studies of science,

technology, and society (Latour, 1987; Law, 1992). Briefly, ANT provides an ever8
evolving framework and vocabulary for describing and, arguably, explaining society's

successes and failures with(in) science and technology. For our purposes, it is not

necessary to learn the extensive jargon of ANT, nor do we need to take on any

metaphysical baggage associated with it (see Bloor, 1999). Understanding what is an

actor network will suffice.

An actor network takes inventory of the contents of the context of an action

(Suchman et al., 1999). A simple, everyday example of an actor network is when we

drive down the freeway. Our actions to any given situation that may arise (e.g., debris on

the roadway) are affected by any number of factors, both human and non-human, such as

our experience and training in driving, the car we are driving, traffic regulations, road

conditions, the cars around us, and the drivers in those cars. An actor network then

considers (links) all of the influential factors, referred to as actors (or actants to remind us

that it is not just humans that need to be considered), for the purposes of describing the

situation and our actions. Thus, within an actor network, human and non-human actors

are given equal explanatory status; at the outset, neither is treated as more important or

more influential. The value of this conceptualization is in the detail of description that

can be obtained and that an explanation does not make any a priori assumption that an

action is the result of (only) social or technologic factors, rather the action will be the

result of interactions between social and technical actors. This cursory definition of

"actor network" is more than enough to help us refocus our attention to the decisionmaking context, rather than just the model, to understand what it is that can be evaluated

for labels, such as, "valid."

Just as the attribute "distance" applies to the relationship between, for example,

Seattle and Memphis, "valid" and "not valid" (and other common adjectives of validation)

only have contextual meaning. The adjectives only apply to the network of relationships

that emerge out of a particular decision-making context, that is, the actor network. As

with all actor networks, any decision-making context is comprised of human and nonhuman actants. Human actants include scientists and engineers (the modelers and their

peers), decision-makers (at any number of levels), stakeholders (those people that will be

affected by the resulting decision or policy), and any funding agencies or the like. While

non-humans actants include, but are not exclusive to, the study area (which could be

unpacked further: e.g., soil type, climate, building codes, environmental impact laws,

etc.), data, model(s), software, hardware, etc. Without this network and the interactions

among the actors, the label of "valid" is not helpful, if not meaningless. Within the actor

network, the model plays an important role, but potentially no more important than the

human actants, such as stakeholders, and no less important than non-human actants, such

as the study area or data. A model may be well liked and well used, and perhaps even

carries the label "valid" thanks to some research publication. However, if the model has

been recruited to inform policy the model is situated within an actor network. The model

may have served well some previous decision-making context, but its success within the

current context is not assured and may be hamstrung by, for instance, the practice of local

scientists or out-of-date computing hardware. Thus, the context that must be considered

is not simply, as Rykiel (1996) argues, the physical conditions for which the model was

developed. 

9
Hopefully, at this point it is easy to see why "valid" (etc.) is not an attribute of

environmental models (rather, all models). Instead, the adjective is contextual and

applies to the particular actor network (or the emergent actions and influences), in which

the model has been recruited. Doubtless, any decision-making context will be complex

and dynamic. It may not always be obvious who or what the actants are within the actor

network. Of course, actants may change with time or be substituted (e.g., higher

resolution data) or the interactions between actants may be modified (e.g., with

experience). Because of the complexity and dynamism of any such network, the label

"valid" or "adequate" is always fuzzy  a matter of degree  and is apt to be short-lived.

As a result, evaluation of the actor network is necessarily a never-ending affair, and,

hence, the idea of validation, as a formal process, loses much of its meaning.

What's The Pragmatic Application?

To recapitulate, we have a working idea of what models, in general, are; we are

confident that environmental models are relevant to policy and decision making; we

realize that the process of environmental-model validation is meaningless, if simply not

helpful; and, lastly, we have reoriented ourselves, with the help of ANT, so as to evaluate

the decision making context, rather than the isolated model. Not a short journey; but

we're better for making the trip. However, we haven't answered the question that we

started with: "What makes for a good model?" 

The popular notion, within environmental modeling, is that a good model is one that

is a good predictor; the model provides a (set of) number(s) or categorizations that

accurately forecasts reality. No doubt, this view is what led Casti to distinguish the

abstract, symbolic models of scientists from that of the aeronautical engineer's model

fuselage or the fashion model. This view of model goodnes is pervasive. For instance,

Charles Groat, director of the U.S. Geological Survey advocated such a notion in his

FY2001 budget request before the Subcommittee on the Interior and Related Agencies

Committee on Appropriations when he said, "Policy-makers at all levels, from the U.S.

Congress, to a business, to a local community, need accurate forecasts so that they can

envision the consequences of the many choices they face today." But having made our

journey, this path (i.e., good models are good predictors) is likely to lead us astray. 

Hooke and Pielke (2000) reemphasizes this point in their assessment of short-term

weather prediction, while providing us a potential path that is compatible with our

contextual view of validity.

Prediction [products] are produced in the environment of a broader

prediction process, which includes the production of forecasts, but also

communication of forecast information and the incorporation of that

information in user decisions. The process might be thought of as a

symphony orchestra in which the different sections must work together

harmoniously to produce music. The analogue to music in the decision

process is effective decision making with respect to weather. Often some

mistakenly ascribe a linear relation to the three sub-processes, ... [predict

Æ communicate Æ use] These three sub-processes are instead better

10
thought of as occurring in parallel, with significant feedbacks and

interrelations between them. (Hooke and Pielke, 2000, p. 68) (Original

emphasis)

Thus, "perfectly accurate" predictions that are miscommunicated or misused will

result in undesirable outcomes of the prediction process (or actions emerging from the

actor-network). Similarly, a prediction can be effectively communicated, but the

prediction product may not be of a form useful to actants within the decision context. As

a result,

when policy makers or other users ... ask the ... question "What is the

value of an improved [prediction product]?" and expect to get an aggregate

answer in dollars or lives, they ask the wrong question. They ought

instead to ask "What changes to the existing [prediction] process ... can we

expect to lead to better outcomes?" and expect the answer to be

contextual, multidimensional, and [perspectival]. (ibid, p. 69) (Author's

emphasis)

With the emphasis on the process, rather than the product, Hooke and Pielke (2000)

have adopted a decidedly contextual and pragmatic view of the goodnes of models.

However, in dealing with short-term weather forecasting, they understandably maintain

that model predictions, while not necessarily the sole foundation of model goodness, are

nonetheless important. We can conditionally accept this. But later, I will argue that other

properties of a model are equally important, or more so, in judging and guiding the

development of good models. Before we get to that, I want to take a cue from the

healthy, pragmatic perspective of Hooke and Pielke (2000) and suggest that we (i.e.,

scientists, decision makers, and stakeholders) adopt the pragmatist's way of thinking to

help us understand both how we should (and I would argue we actually do) go about

evaluating model goodness. As I briefly introduce the ideas of pragmatism, we will see

that our criticism of "ground truth" and our newly acquired contextual view of validity fit

quite nicely. 

The roots of pragmatism date back to the turn of the previous century with Charles S.

Peirce, William James, and John Dewey. Pragmatism has experienced a resurgence in

the latter half of the twentieth century, thanks in part to Richard Rorty's Philosophy and

the Mirror of Nature (1979). Pragmatism's popularity is continuing to grow on into the

twenty-first century. Environmental pragmatism is a particularly good example of this

popularity. Environmental pragmatism marries a pragmatist perspective to the

explanation, solution and discussion of environmental ethics and related issues (see Light

and Katz, 1996). So it is quite natural for the field of environmental modeling to adopt a

similarly pragmatic disposition. There are several hallmarks of pragmatism that I think

are quite valuable to understanding model goodnes and to environmental modeling, in

general, as it relates to policy and decision making. 

First and foremost, pragmatism is united against any type of foundationalist thought

(Giere, 1999), such as the notion of "ground truth" or other common notions that results

in a model being stamped as "valid." From the perspective of pragmatism, all products of

human endeavor are both provisional and fallible. To a pragmatist

11
[k]nowledge ... is a name for the product of competent inquiries ...

The "settlement" of a particular situation by a particular inquiry is no

guarantee that that settled conclusion will always remain settled. The

attainment of settled beliefs is a progressive matter; there is no belief so

settled as not to be exposed to further inquiry. (Dewey, 1938) (Original

emphasis)

As part of his fallibilism, John Dewey argued that we may get things truer and truer,

but this is "truth" in the pragmatic sense. To a pragmatist, truth in the foundational sense

(e.g., correspondence truth) is circular and, hence, unhelpful (James, 1907). For example,

to say that the model is "correct" or the model is "valid" does not explain the strengths

and weakness of recruiting a particular environmental model for a given context and

tends to prevent further inquiry of the model, which ultimately would provide a better and

better sense of model goodness. As the name implies, pragmatism is concerned with

human activities (e.g., a decision-making process) and the effects and outcomes of these

activities. Just as Hooke and Pielke (2000) focus our attention on the outcomes of the

prediction process, any mention of truth, by a pragmatist is referring to practical

consequences. "Truth is not something passively obtained, either by the contemplation of

absolutes or by the passive accumulation of data, but by activity shot through with the

theory [model] that guides it." (Rosenthal and Buchholz, 1996, p. 39) It follows that in

order for us to evaluate the goodnes of a model, we need to recruit it within our particular

decision-making context (actor network) and experiment with it. 

To help us understand this, it is useful for us to think of cars again, but this time in

the context of buying not driving. Few of us would purchase a car without taking it for a

test drive. The car may meet every criteria we set out (e.g., color, cargo capacity, engine

size, etc.), but it's not until we take it for a spin around the block and talk it over with

friends  participate in an activity and experiment with the car  that we can decide

whether or not the car is for us. Of course, the pragmatist knows that this choice to

purchase the car based on a test drive is not foolproof. But this should not and does not

prevent (most of) us from buying a car. In time, now that we own the car, we may

discover some annoying  even dangerous  features (e.g., a blind spot) about the car

that weren't addressed by our initial criteria or didn't "appear" to us during the test drive.

But with more time behind the wheel, we become practiced and the annoying feature is

not so much an obstacle, but something familiar. Of course, it is very possible that we

may not be able to practically overcome some annoying or dangerous features. But from

experience, which includes not only driving the car but our conversations with people

about our car and cars in general, we are better prepared to return to the car lot (or model

supermarket) and make a "truer" decision  one that will meet our practical

expectations.

At the risk of over-using a metaphor, the activity of buying a car helps to highlight

another hallmark of pragmatism: plurality. Except in rare cases of "love at first

experimentation," we would not buy a car without test driving multiple cars. From the

buyer's perspective, plurality of car models is typically seen as a good thing. Our

experiences from test driving one model helps us in experimenting with the next model;

12
and sometimes, we realize that we can't do without more than one model in our garage.

This is true for environmental models as well. I would like to suggest that we have a

responsibility to expose ourselves to a plurality of environmental models so that each

model has a chance to "choose" us. After evaluating four earthquake-induced landslide

models (the same four introduced in the first section), Miles and Keefer (2000) stress the

importance of keeping all four models in the "garage." Though each is a model of the

same thing (all derived from the same method in fact), one model may be more useful in

one context because of, say, available data, while a different model may be more usable

in another context, perhaps because of technical know-how of decision makers. Ideally,

several models can be used so as to encourage us to explore and explain differences in

model output  that is, foster experimentation. 

A Model Of Consensus

So, what makes for a good model? The environmental pragmatist Bryan Norton

provides the following answer:

[M]odels [should] improve our understanding in the specific sense

that they illuminate environmental problems and allow us to focus on

those natural dynamics that are causally related to important social values.

... The ... models chosen in normative sciences must therefore pass a

double criterion. They must help us to understand nature, but they must

also encourage us to understand nature in a way that will help us to

formulate and measure environmental goals effectively and to propose and

implement policies to achieve these goals. (Norton, 1996, p. 125-126)

That is, a good model (or models!) is one that guides our particular decision-making

process to the desired outcomes (or nearly so) in a not entirely unexpected way, which

would, of course, constitute a "valid" (or partially so) actor network.

Our motivating question has been answered, at least to some degree. The answer

may seem obvious to some, but of course it is the activity of answering (discussing) the

question that is most helpful. (I hope this discussion continues well beyond this paper.)

Even so, the environmental modelers among us are likely thinking: "Pragmatism guides

us to the obvious answer of what is a good model, but can it help me to develop good

models of environmental systems?" My answer is that it can.

Richard Rorty describes an "objective" explanation as one that is open and

transparent (Rorty, 1991). Similarly, if we adopt the perspective of pragmatism, we

should strive to develop objective models  models that are lucid and easily

understandable by the human actors within the respective actor network (i.e., scientists,

decision-makers and stakeholders). This is not to say that some environmental modelers

do not already attempt to do this, but that we should put greater emphasis on research to

meeting this goal. 

Complex mathematical equations or numerical simulations can be difficult for even

scientists to comprehend, let alone other actors in the decision-making context.

Following the suggestion of Richard Coyne for the design of information technology, the

13
transparency or ease of understanding of environmental models can be increased by

treating models as metaphors (Coyne, 1995). This can be accomplished through effective

use of conceptual models, diagrams, analogies, and the like, which helps to describe the

features and workings of the otherwise largely unintelligible model through juxtaposition

with something familiar or more intelligible (e.g., landslides as blocks on an inclined

plane; electricity as water flowing through a pipe network; or gas molecules as billiard

balls). Such practice used to be common place, but seems to have waned with the

increasing complexity of environmental models and emphasis on prediction. This

practice needs to resume and expand for the sake of building transparent environmental

models. 

Alternatively, fuzzy systems provides an encouraging means for developing

pragmatically objective models of complex environmental systems. Because the essence

of fuzzy inference is computing with words, fuzzy systems can be used to model any

complex system (Kosko, 1992). For the same reason, the salient features of any fuzzy

inference model can be read and understood by modelers, policy makers, and

stakeholders, regardless of complexity.

Openness and transparency also applies to the decisions and choice with regards to

how we go about parameterizing and implementing environmental models. Paul N.

Edwards points out that the distinction between data and models is not always clear

because most data is collected or generated with the assistance of models and vice versa

(Edwards, 1999). In a similar way, models are not easily distinguished from the means in

which they are implemented because models must often be modified to fit the particular

means of implementation (e.g., using a raster data model versus a vector data model

within GIS) (Miles and Ho, 1999). Thus, a good model includes actively (rather than

passively) making explicit choices regarding parameterization and implementation, so

that all actors in the decision making context are aware of the effects of such choices, and

possible alternatives (Miles et al., 2000).

An objective decision is a decision arrived at by consensus (Rorty, 1991). Therefore,

good models should assist in the building of consensus among the human actors in a

particular decision-making process. So how can models help to build consensus? 

Before I answer that question, I want to say that models should not be recruited as

tools of self-righteousness that prevent a consensus solution. With the prevailing

overemphasis on the value of model predictions and, of course, validity as "ground truth,"

I would argue that within any given decision-making context there is real potential for

this. Predictions are apt to be used to avoid making decisions, rather the "decision" is

"made" by the predictions of the environmental model (Pielke et al., 2000). For this

reason, while model-based predictions certainly have heuristic use, as in the example of

short-term weather-forecasting, complete reliance on model predictions (and stamping

models as "valid") can only be interpreted as an effort to avoid liability and

accountability.

Avoiding this pitfall, how can environmental models help to build consensus? I

would like to suggest that environmental models can greatly assist in consensus building

through persuasion, that is, encouraging experimentation and facilitating communication

14
by and among scientists, decision makers, and stakeholders. 

Let's look again at the latter part of Casti's quote on experimental models:

... the model then serves to answer by direct experiment certain types

of questions, such as how air flows over an airplane wing under different

atmospheric conditions or how a particular types of clothes ... look when

draped over a perfect shape. (Casti, 1997, p. 13)

This is precisely what any good model should do: help us to answer (and pose) questions

 certainly not a new revelation!

Thus, as I argued in the beginning, all good models are experimental models. A

good experimental environmental model is one that is pragmatically objective and is

outfitted with suitably numerous and appropriate "knobs and levers." What I mean by

"knobs and levers" can be anything from variables in an equation that represent some

aspect of the particular environmental system (e.g., earthquake magnitude) to literal

buttons or sliders of a graphical user interface. These "knobs and levers" must be relevant

to the needs (and ability) of the human actors in a decision-making context so as promote

their twisting and pulling  that is, encourage experimentation. We can see then,

because experimentation is a quality of a good model, as well as, being an important part

of evaluating model goodness, development of new and better experimental

environmental-models will likely lead to their use and (provisional) adoption.

I think that if we can make environmental models more engaging by improving their

ease of understanding (objectivity) and making them more and more suited to

experimentation, then environmental models will do a great deal in facilitating

communication. This is consistent with Richard Coyne's view of models as metaphors:

A metaphor need not be predictive. It is primarily a discursive tool to

keep a conversation alive, and its efficacy resides in its adoption and use in

discourse ... (Coyne, 1995, pg. 245)

The example of hikers huddled around a map (model!) trying to decide the best

route up the mountain, provides a good, if overly-simple, example of what I mean.

Because the map, hopefully, has appropriate (ocular) "knobs and levers," the hikers, who

may have varying levels of expertise and experience, can experiment with different routes

up the mountain without actually hiking each proposed route  precisely why we need

environmental models. In experimenting with the easily understandable model,

communication among the hikers is facilitated and a consensus of which route to take can

be built. Let's say for instance that a route is identified as being "good" by several of the

hikers. However, because the map helps to convey that the route may involve some

amount of perceived danger, some of the hikers communicate their apprehension and so

another route is chosen through further interaction with the map.

Based on this example, I am sure there are some of us that want to insist that this is

exactly why "ground truthing" must be the measure of model goodnes  the map must

correspond to reality (the mountain) so the hikers can accurately predict what they will

encounter. This insistence of "ground truth" leads to the call for reducing or eliminating

15
uncertainty in environmental models. However, this is another foundationalist view that

needs to change because uncertainty is a fact of life. The typical notion of uncertainty

must be expanded beyond empirical uncertainty (e.g., the degree of fit of a regression

equation to data points) to include epistemological uncertainty (questions about our

knowledge or beliefs) (Edwards, 1999). Epistemological uncertainty cannot be

eliminated. Further, any attempt at reducing uncertainty (in the broader sense advocated

by Edwards, 1999) is itself associated with uncertainty because absolute certainty is an

unseen target (i.e., "How can we be conscious of what we are not conscious of?"). This is

the essence of Lotfi Zadeh's Principle of Incompatibility, which forms the foundation of

the science and philosophy of fuzzy logic (Kosko, 1993). Zadeh argues that the closer

you look at an environmental system the fuzzier the understanding of the system or

solution to the problem becomes. The following discussion comes out of a paper on

decision processes:

As the complexity of a system increases, our ability to make precise

and yet significant statements about its behavior diminishes until a

threshold is reached beyond which precision and significance (or

relevance) become almost mutually exclusive characteristics. It is in this

sense that precise analysis of the behavior of humanistic systems are not

likely to have much relevance to the real-world societal, political,

economic, and other types of problems which involve humans either as

individuals or in groups. (Zadeh, 1973, p. 339)

While we cannot eliminate uncertainty from environmental models  Is it the model

that is uncertain or is it us?  and any reduction of uncertainty is itself associated with

uncertainty, I do think that environmental models can help to facilitate the understanding

and communication of the uncertainties within a particular decision-making context.

Thus, rather than spending vast amounts of resources trying to eliminate the uncertainties

in models that we can't be fully aware of anyway, we should develop environmental

models so that they afford the characterizing and conveying of uncertainty to the best of

our ability. While work is definitely being done towards this end, we need to consider the

point of facilitating communication. Take the widespread use of probability to describe

uncertainty and whether this helps to communicate uncertainty between scientists,

decision makers and stakeholders. The common understanding of probability is that of

the chance of, for example, rolling an even number using a six-sided die (50%

probability). In this case, the probability value communicates the near-absolute certainty

of the chance of rolling an even number. Thus, probability is commonly understood as

characterizing the uncertainty of an outcome (e.g., an even number or an earthquake),

rather than the uncertainty of our understanding of the system (e.g., a die or fault system)

or the uncertainty in the model output. Considering this, it can only be beneficial to

explore more (and more varied) means of facilitating the disclosure of uncertainty

through the development of pragmatic environmental models.

It Really Is A Model World

In writing this paper, I have two objectives beyond helping myself to think through

16
the issues surrounding environmental-model validation. The first is to add my voice, as a

practicing environmental modeler, to the fray in hopes of affecting further change in the

attitudes of environmental modelers, and perhaps even more so, decision makers, and

stakeholders since they both have equal say if environmental modeling is not only to

influence policy, but influence it in a positive and anticipated way. That we cannot ever

establish, with absolute certainty, the (correspondence) truth and reliability of an

environmental model and its output, is something that we must accept. By accepting this,

our view of model validation (or "ground truthing") has to shift. Rather than focusing

upon the environmental model, we need to take a pragmatic, contextual view of validity,

which is determined by the practical consequences emerging out of a particular decisionmaking context (actor network), that broadens our consideration to human and nonhuman actors.

My second objective is to attempt to reconstruct the role of environmental models in

the face of current philosophical perspectives and specific critiques of model validation.

From my unapologetically biased standpoint, I think environmental modeling can and

will continue to serve an important role in society. That role is decidedly pragmatic  to

help in the building of consensus (objective) decisions by encouraging experimentation

and facilitating communication (especially of uncertainties). In no way is this a

diminishment of the role of models or the importance of environmental modeling

research. If anything, it is a promotion in both respects. The building of consensus, in

even the most simple decision-making context, is rarely a trivial thing. The role of

affording consensus will help to allay fears surrounding the uncertainties and fallibility of

prediction, and reduce the risk of putting absolute faith in any prediction. To serve this

role well won't be a small matter and will require as much or more research effort on the

part of environmental modelers, who must take the lead, decision makers, and

stakeholders to find novel ways of encouraging experimentation and facilitating

communication. Continuing and building upon current work towards characterizing and

conveying uncertainty is more important than ever if environmental models are to take on

this modified role.

Being a pragmatist, I have no expectations that this paper has provided "the" answer

to making environmental modeling (more) relevant to policy. Perhaps, it provides no

answers. Certainly however, it provokes a wealth of questions that provide for further,

active and fruitful discussion. I will consider this paper a success if it manages to involve

a wider cross-section of discussants, beyond just a handful of scientists and historians,

sociologists, and philosophers of science. As for the judges and lawyers, if we change

our expectations and our practices, they will have no choice but to follow.

Acknowledgments

I would like to thank Herman Karl and Dave Keefer (U.S. Geological Survey) for

their constructive reviews of the manuscript. I would also like to thank, without

implicating, Michael Brown (University of Washington), Nick Chrisman (University of

Washington), Bryan Norton (Georgia Tech), Naomi Oreskes (U.C. San Diego), Barbara

Poore (U.S. Geological Survey), Matt Sparke (University of Washington), and Paul

17
Taylor (University of Washington) for discussing various issues during the preparation of

the manuscript.

References

Barlas, Y., and Carpenter, S., 1990, Philosophical roots of model validation: two

paradigms: System Dynamics Review, vol. 6, no. 2, pp. 148-146.

Bloor, D., 1999, Anti-Latour: Studies of the History of the Philosophy of Science, vol. 30,

no. 1, pp. 81-112.

Board, C., 1967, Maps as models, in Models in Geography, Chorley, R. J., Haggett, and

P., eds.: London, Mathuen, pp. 671-725.

Coyne, R., 1995, Designing Information Technology in the Postmodern Age: Cambridge,

MA, MIT Press.

Corwin, D. L., and Loague, K., eds., 1996, Applications of GIS to the Modeling of NonPoint Source Pollutants in the Vadose Zone: Soil Science Society of America, Special

Publication 48.

Close, U., and McCormick, E., 1922, Where the mountains walked: National Geographic,

vol. 41, no. 5, pp. 445-464.

Casti, J. L., 1997, Would-Be Worlds: New York, John Wiley and Sons.

Dewey, J., 1938, Logic: The Theory of Inquiry: New York, Holt and Company.

Devitt, M., 1997, Realism and Truth (2nd edition with new afterword): Princeton, NJ,

Princeton University Press.

Davidson, D., 1990, The structure and content of truth: Journal of Philosophy, vol. 87,

pp. 279-328.

Edwards, P. N., 1999, Global climate science, uncertainty and politics: Data-laden

models, model-filtered data: Science as Culture, vol. 8, no. 4, pp. 437-472.

Giere, R. N., 1999, Science Without Laws: Chicago, The University of Chicago Press.

Hooke, W. H., and Pielke, R. A., 2000, Short-term weather prediction: An orchestra in

need of a conductor, in Prediction: Science, Decision Making, and the Future of

Nature, Sarewitz, D., Pielke, R. A., and Byerly, R., eds.: Washington, D.C., Island

Press, pp. 61-84.

James, W., 1978;1907, Pragmatism: Cambridge, MA, Harvard University Press.

Kuhn, T., 1962, The Structure of Scientific Revolutions: Chicago, University of Chicago

Press.

Kosko, B., 1992, Fuzzy systems as universal approximators, in IEEE Conference on

Fuzzy Systems,, .:,, pp. 1153-1162.

Kosko, B., 1993, Fuzzy Thinking: New York, Hyperion.

Kleindorfer, G. B., O'Neill, L., and Ganeshan, R., 1998, Validation in simulation: Various

positions in the philosophy of science: Management Science, vol. 44, no. 8, pp. 1087-

1099.

Keefer, D. K., 1984, Landslides caused by earthquakes: Geological Society of America

18
Bulletin, vol. 95, pp. 406-421.

Light, A., and Katz, E., eds., 1996, Environmental Pragmatism: New York, Routledge.

Law, J., 1992, Notes on the theory of the actor network: Ordering, strategy and

heterogeneity: Systems Practice, vol. 5, pp. 379-393.

Miles, S. B., and Keefer, D. K., 2000, Evaluation of seismic slope-performance models

using a regional case study: Environmental and Engineering Geoscience, vol. 11, no.

1, pp. 25-39.

Miles, S. B., Keefer, D. K., and Nyerges, T. L., 2000, A case study in GIS-based

environmental model validation using earthquake-induced landslide hazard, in 4th

International Symposium on Spatial Accuracy Assessment in Natural Resources and

Environmental Sciences, Heuvelink, G. B. M., and Lemmens, M. J. P. M., eds.:

Amsterdam, Delft University Press, pp. 481-493.

Miles, S. B.,. Keefer D. K.,. and Ho C. L., 1999, Seismic slope-performance analysis:

from hazard map to decision support system, in Optimizing Post-Earthquake Lifeline

System Reliability, Elliot, W. M., and McDonough, P., eds.: Reston, VA, ASCE, pp.

71-80.

Miles, S. B., and Ho, C. L., 1999a, Rigorous landslide hazard zonation using newmark's

method and stochastic ground motion simulation: Soil Dynamics and Earthquake

Engineering, vol. 18, no. 4, pp. 305-323.

Miles, S. B., and Ho, C. L., 1999b, Applications and issues of GIS as tool for civil

engineering modeling: Journal of Computing in Civil Engineering, vol. 13, no. 3, pp.

144-152.

Norton, B. G., 1996, Integration or reduction: Two approaches to environmental values,

in Environmental Pragmatism, Light, A., Katz, and E., eds.: New York, Routledge, pp.

105-138.

Oreskes, N., 1998, Evaluation (not validation) of quantitative models: Environmental

Health Perspectives, vol. 106, Sup 6., pp. 1453-1459.

Oreskes, N., Shrader-Frechete, K., and Belitz, K., 1994, Verification, validation, and

confirmation of numerical models in the earth sciences: Science, vol. 263, no. 5147,

pp. 641-646.

Openshaw, S., 1996, Fuzzy logic as a new scientific paradigm for doing geography:

Environment and Planning A, vol. 28, pp. 761-768.

Pielke, R. A., Sarewitz, D. A., and Byerly, R., 2000, Decision making and the future of

nature: Understanding predictions and using predictions, in Prediction: Science,

Decision Making, and the Future of Nature, Sarewitz, D., Pielke, R. A., and Byerly,

R., eds.: Washington, D.C., Island Press, pp. 361-387.

Rykiel, E. J., 1996, Testing ecological models: the meaning of validation: Ecological

Modeling, vol. 90, pp. 229-244.

Rosenthal, S. B., and Buchholz, R. A., 1996, How pragmatism is an environmental ethic,

in Environmental Pragmatism, Light, A., and Katz, E., eds.: New York, Routledge, pp.

38-49.

Rorty, R., 1991, Objectivity, Relativism, and Truth: Cambridge, Cambridge University

19
Press.

Rorty, R., 1979, Philosophy and the Mirror of Nature: Princeton, NJ, Princeton

University Press.

Suchman, L., Blomberg, J., Orr, J. E., and Trigg R., 1999, Reconstructing technologies

as social practice: American Behavioral Scientist, vol. 43, no. 3, pp. 392-408.

Stein, R. S., Barka, A. A., and Dietrich, J. H., 1997, Progressive failure on the North

Anatolian fault since 1939 by earthquake stress triggering: Geophysical Journal

International, vol. 128, pp. 594-604.

Sarewitz, D., and Pielke, R. A., 1999, Prediction in science and policy: Technology in

Science, vol. 21, pp. 121-133.

Sarewitz, D., Pielke, R. A., and Byerly, R., eds., 2000, Prediction: Science, Decision

Making, and the Future of Nature: Washington, D.C., Island Press.

Wilson, R. C., and Keefer, D. K., 1983, Dynamic analysis of a slope failure from the 6

August, 1979 Coyote Lake, California, earthquake: Seismological Society of America

Bulletin, vol. 73, no. 3, pp. 863-877.

Zadeh, L. A., 1973, Outline of a new approach to the analysis of complex systems and

decision processes: IEEE Transactions of Systems Management and Cybernetics, vol.

1, pp. 28-44.

20