# Try, try again: Lessons learned from success and failure in participatory modeling

Eleanor Sterling, Moira Zellner, Karen Jenni, Kirsten Leong, Pierre Glynn, Todd Bendor, Pierre Bommel, Klaus Hubacek, Antonie Jetter, Rebecca Jordan, et al.

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Eleanor Sterling, Moira Zellner, Karen Jenni, Kirsten Leong, Pierre Glynn, et al. Try, try again: Lessons learned from success and failure in participatory modeling. Elementa, 2019, 7, pp.9. 10.1525/elementa.347 hal-05173749

HAL Id: hal-05173749

https://hal.science/hal-05173749v1

Submitted on 21 Jul 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# PRACTICE BRIDGE

## Try, try again: Lessons learned from success and failure in participatory modeling

Eleanor J. Sterling*, Moira Zellner†, Karen E. Jenni‡, Kirsten Leong§, Pierre D. Glynn||, Todd K. BenDor¶, Pierre Bommel**††, Klaus Hubacek††‚§§||||, Antonie J. Jetter¶¶, Rebecca Jordan***, Laura Schmitt Olabisi†††, Michael Paolisso††† and Steven Gray†††

Participatory Modeling (PM) is becoming increasingly common in environmental planning and conservation, due in part to advances in cyberinfrastructure as well as to greater recognition of the importance of engaging a diverse array of stakeholders in decision making. We provide lessons learned, based on over 200 years of the authors' cumulative and diverse experience, about PM processes. These include successful and, perhaps more importantly, not-so-successful trials. Our collective interdisciplinary background has supported the development, testing, and evaluation of a rich range of collaborative modeling approaches. We share here what we have learned as a community of participatory modelers, within three categories of reflection: a) lessons learned about participatory modelers; b) lessons learned about the context of collaboration; and c) lessons learned about the PM process. First, successful PM teams encompass a variety of skills beyond modeling expertise. Skills include: effective relationship-building, openness to learn from local experts, awareness of personal motivations and biases, and ability to translate discussions into models and to assess success. Second, the context for collaboration necessitates a culturally appropriate process for knowledge generation and use, for involvement of community co-leads, and for understanding group power dynamics that might influence how people from different backgrounds interact. Finally, knowing when to use PM and when not to, managing expectations, and effectively and equitably addressing conflicts is essential. Managing the participation process in PM is as important as managing the model building process. We recommend that PM teams consider what skills are present within a team, while ensuring inclusive creative space for collaborative exploration and learning supported by simple yet relevant models. With a realistic view of what it entails, PM can be a powerful approach that builds collective knowledge and social capital, thus helping communities to take charge of their future and address complex social and environmental problems.

**Keywords:** Participatory modeling; Collaborative modeling; Stakeholder engagement; Planning; Environmental management

## Introduction

Participatory Modeling (PM) is becoming increasingly common in environmental planning and conservation due to advances in cyberinfrastructure and to greater recognition of the importance of engaging a diverse array of stakeholders in decision making. We initiated this reflective article at the first of a series of workshops on PM, sponsored by the National Socio-Environmental Synthesis of the importance of engaging a diverse array of stakeholders in decision making. We initiated this reflective article at the first of a series of workshops on PM, sponsored by the National Socio-Environmental Synthe-

* Center for Biodiversity and Conservation, American Museum of Natural History, New York, US

† Department of Urban Planning and Policy, University of Illinois at Chicago, Chicago, Illinois, US

‡ U.S. Geological Survey, Science and Decisions Center, Denver, Colorado, US

§ NOAA Fisheries, Pacific Islands Fisheries Science Center, Honolulu, Hawaii, US

|| Water Mission Area, U.S. Geological Survey, Reston, Virginia, US

¶ Department of City and Regional Planning, University of North Carolina at Chapel Hill, North Carolina, US

** CIRAD Green, Univ. Montpellier, FR

†† UCR Universidad de Costa Rica, Cieda, San José, Costa Rica, US

††† Department of Geographical Sciences, University of Maryland, College Park, Maryland, US

§§ Department of Environmental Studies, Masaryk University, Brno, CZ

||| International Institute for Applied Systems Analysis, Laxenburg, AT

¶¶ Department of Engineering and Technology Management, Portland State University, Oregon, US

*** Department of Human Ecology, Rutgers University, New Brunswick, New Jersey, US

††† Department of Community Sustainability, Michigan State University, East Lansing, Michigan, US

†††† Department of Anthropology, University of Maryland, College Park, Maryland, US

Corresponding author: Eleanor J. Sterling (sterling@amnh.org)
sis Center (SESYNC). The goal of the workshop series is to organize and consolidate scholarship around the practice of PM. PM encompasses the use of a broad range of modeling approaches in various forms of collaboration among practitioners, academics, and other stakeholders who engage in a purposeful learning process that elicits and formalizes the implicit and explicit knowledge of participants to support decision-making and action. Our knowledge community has learned important lessons from their experiences, successes, and failures. Given our experience, we firmly believe that PM has the potential to help communities guide themselves toward more positive futures. We offer these lessons learned in the hope that they will guide practitioners as they assist communities with using meaningful, appropriate, and illuminating modeling tools in participatory processes.

The reflections in this paper emerge from the authors' over 200 years of cumulative and diverse experience conducting PM processes with communities concerned with different issues. Some examples of our work can be found in **Table 1**, at www.participatorymodeling.org, and in Gray et al., 2018.

Our collective interdisciplinary background allowed for reflection on a rich range of collaborative modeling approaches, incorporating insights and experiences with environmental modeling, spatial analysis, urban and regional planning, psychology, anthropology, computer science, social science, and economics. The reflections in this paper are also informed by prior summary and review articles. These prior articles, covering multiple participatory studies, describe commonalities, articulate different overarching structures for the PM processes, describe lessons learned, and propose principles for effective PM (e.g., Voinov and Brown Gaddis, 2008; Voinov and Bousquet, 2010; Oteros-Rozas et al., 2015; Voinov et al., 2016; Gray et al., 2018). This paper focuses less on summarizing previous PM projects, and more on what we have learned about the challenges of implementing PM, especially in creating meaningful stakeholder and modeler partnerships within the complex social and political contexts where PM is most needed.

These reflections and lessons learned were generated and organized as follows. During our first workshop (February 2016), each participant wrote a summary of the

key lessons that they had learned in their PM research and practice. We used inductive logic (Charmaz and Belgrave, 2007) to identify patterns arising from compiled data and grouped them into the three key themes described below. We built on this synthesis through several iterations, drawing from diverse literature to provide the foundation for this paper.

We have not previously shared these lessons widely because their articulation is frequently extraneous to traditional academic scholarship. We are often discouraged from publishing about failures (Becu et al., 2007). Our intent herein is to share our experience beyond advancing the technological dimension of modeling. We seek to encourage, and maybe even inspire others to embrace the uncertainty and messiness inherent to PM through lessons we have learned via many trials, successes, and even more errors, very much like any modeling process (Railsback and Grimm, 2012). It is our hope that sharing these lessons will help other practitioners skip some of the more painful learning steps we ourselves have worked through, and more effectively build the powerful collective knowledge and social capital that can emerge from PM processes.

**Figure 1** illustrates a common evolution of assumptions and practices in PM, where researchers, who are eager to put their skills and knowledge to good use in supporting decisions, move from a strictly technical perspective towards full embrace of the partnership perspective. Each puzzle piece depicted in **Figure 1** is necessary but not sufficient in describing how best to achieve innovative solution-building and action. For example, a common initial assumption is that providing decision-makers with the “right” information (e.g., more accurate/precise data, an understanding of interaction effects) and the “right” tools (e.g., models that accurately represent processes, interactions, and provide meaningful results), is sufficient for them to solve their problems. This is not enough to lead to improved decisions, actions, and effective solutions, possibly because decision-makers distrust modeling processes and tools that exclude their involvement.

Faced with falling short of their goals, researchers add increasingly more nuance to their approaches, moving next to an emphasis on communicating in the “right”

<table><caption>**Table 1:** Examples of authors' diverse experience conducting PM processes. DOI: https://doi.org/10.1525/elementa.347.t1</caption><thead><tr><td>How to...</td><td>Citation(s)</td></tr></thead><tbody><tr><td>Address flooding using decentralized green infrastructure</td><td>Zellner et al., In Press</td></tr><tr><td>Ensure the sustainability of groundwater supplies</td><td>Zellner et al., 2012</td></tr><tr><td>Protect biodiversity and ecosystem functions, and understand drivers of bushmeat trade</td><td>Nayaki et al., 2014</td></tr><tr><td>Model the social and economic impacts of climate change on coastal resources</td><td>Gray et al., 2014</td></tr><tr><td>Provide decision-support for wildlife managers in the Pacific</td><td>Htun et al., 2016</td></tr><tr><td>Support conservation planning in relation to invasive species management</td><td>Gray et al., 2017</td></tr><tr><td>Enact with farmers water resource planning at times of drought</td><td>Douglas et al., 2016</td></tr><tr><td>Address land use conflicts and manage trade-offs of a range of ecosystem services</td><td>Hubacek et al., 2009; Schmitt Olabisi et al., 2017</td></tr></tbody></table>
<figure><img src="image_3.png" /><figcaption>**Figure 1: Progression of researchers' evolving assumptions and practices in Participatory Modeling.** Researchers move from a strictly technical perspective towards full embrace of the partnership perspective. Light blue boxes are evolving assumptions; blue boxes are challenges researchers face when acting on those assumptions, leading to new actions based on the assumptions. Each puzzle piece is necessary but not sufficient in describing how best to achieve innovative solution-building and action, building towards a fuller set of foundations for effective application.
DOI: https://doi.org/10.1525/elementa.347.f1</figcaption></figure>

ways (e.g., clearly translating and presenting information and models to both decision-makers and stakeholders), and when that too fails, moving towards more inclusion and participation. Researchers may assume that engaging selected stakeholders in data collection, analysis, and modeling will facilitate ownership and make stakeholders feel empowered and informed. When anticipated outcomes do not occur, researchers then turn to the need for diversity to enhance innovation and representation. Conflict may arise in engaging multiple stakeholders. In response, while keeping in mind previous assumptions and responses, the researchers focus on social processes, towards an understanding that attending to emotions, transparency of models and assumptions, power dynamics, and tradeoffs are key elements of effective PM.

This evolution takes place in individuals as well as across whole fields where initial technocratic approaches are enhanced by and eventually embrace more inclusive

approaches. We have seen similar evolutions in risk management (Fischhoff, 1995), in probabilistic decision analysis (Spetzler et al., 2016), and in stakeholder engagement (Sterling et al., 2017). Recognition of greater complexity leads to consideration of processes as well as outcomes; more inclusive public input and consultation approaches, including possibilities for civic engagement and co-management (Leong et al., 2009, 2012); and a "commitment to action" based on the entire process (Leong et al., 2011; Spetzler et al., 2016).

We have grouped our reflections into three main categories regarding lessons learned: a) how modelers need to engage in PM, b) how to adapt to the social and political context of collaboration, and c) how to set up the PM process itself. Our reflections may be of most interest to modelers and researchers but we also share these lessons learned for facilitators and communities who may be interested in working in this arena.
## Lessons learned about the role of participatory modelers

"Participatory modelers," in the context of this paper, are individuals who facilitate, organize, and develop models for PM projects. A diverse set of skills, including prior content knowledge, facilitation skills, and technical modeling skills are needed to ensure that the PM process and resulting models are useful and helpful to the participating communities. It is a rare individual who possesses all of these skills; it is often better to consider a collaborative team as "the" participatory modeler (Prell et al., 2007). Here we report the skills that we have found to be most important for the team:

1. **Content knowledge coupled with openness to learn from local experts:** A participatory modeler needs to be able to speak intelligently with stakeholders about the problem being modeled, and thus needs to have or develop background knowledge regarding the subject matter(s) related to the site, problem, or issue to be modeled. In addition to academic knowledge, important content knowledge comes from learning directly from the stakeholders about their unique experience with the complex problem to be addressed, and about their concerns, needs, and aspirations. When modelers come in as "outside experts," all too often the result is alienation rather than collaboration (Hoch et al., 2015; Zellner et al., In Press). Recognizing the value and importance of local content knowledge means that modelers focus less on their own expertise and more on learning from local stakeholders. As an example, in the context of a collaborative stormwater management park project in a low-income community (Zellner et al., 2018), project leaders developed an ice-breaker activity to explain how to use causal loop diagramming (CLD) to lay out the complex interactions between flooding and economic development. The issue of traffic congestion was chosen to illustrate how one could draw the links among factors that reinforce congestion or alleviate it. As project leaders developed the exercise with key stakeholders in preparation for a larger community workshop, they learned from participants that traffic congestion was not an issue in their community, so they could not relate to the topic or to the CLD activity. The team then switched to a housekeeping and chores problem that most people could relate to, and which proved to be very effective in illustrating how community members could use CLD to link the problem they were facing with their collective aspirations. This unlocked and completely transformed the planning process for the project. The community's CLD was built on local knowledge, beyond what the modelers could contribute.

2. **Skills to liaise effectively and build relationships with stakeholders:** Participatory modelers need skills in identifying and selecting stakeholders (Reed et al., 2009, 2013a), and in

building relationships and, consequently, credibility with stakeholders. These include strong listening skills accompanied by humility and patience. A good listener learns more than an expounder, and deep listening (not interrupting, allowing for people to express their ideas in their preferred way) and patience are critical in developing trusting relationships. Furthermore, the recognition that stakeholders' expertise is equally valid to the modelers' provides a solid foundation for mutual trust and respect, with the understanding that modelers are, themselves, another type of stakeholder (Barreteau et al., 2003; Zellner, 2008).

3. **Facilitation skills to elicit effective stakeholder engagement:** Effective stakeholder engagement includes ensuring stakeholders communicate with each other. Often it means managing conflict. The facilitation process (Kaner, 2014) elicits participants' needs, priorities, and understanding of the system, and fosters empathy among people who may hold different worldviews (Zellner, 2008; Hovmand, 2014). This may require opportunities for participants to "step into other people's shoes". For example, we saw increased understanding of a complex system and empathy with other stakeholders when we coached stakeholder groups to discuss mental model representations by other groups and to brainstorm solutions that would be acceptable to these groups (Singer et al., 2017). Professional facilitation can be useful in such cases, but just as with "outside experts" with content knowledge, it may not always benefit the engagement process. Community facilitators can be effective partners in the participatory modeling team (Hovmand, 2014).

4. **Ability to recognize, extract, and communicate a conceptual model from discussions with and among stakeholder groups:** It is not a simple task to identify common themes from group discussions and translate them into a conceptual model of the system. It is equally difficult to make that conceptual model transparent and tractable in a way in which stakeholders recognize their inputs and gain intellectual ownership of the model. The modeling techniques need to fit the problem characteristics, and the model needs to be understandable and usable for the people who contributed to building it (Borshchev and Filippov, 2004; Martin and Schlüter, 2015). Modelers must strive to ensure their own views and favored methods do not drive the model development. As one example, an initial effort to develop an extension to a National Park in Thailand failed when modelers focused on using a spatial distribution model to demarcate boundaries. Applying different modeling techniques (including role-playing games and agent-based modeling) helped move negotiations to a more successful, innovative, and integrative discussion of solutions based on resource access and multi-functional space (Barnaud et al., 2013).
5. **Ability to recognize how one's own intuitions can serve as biases in the PM process:** Reflective thinking and self-awareness can help us assess the limits of our expertise and recognize the subjectivity in our own knowledge and judgements (Glynn, 2017). Such awareness generally improves the outcomes of our decisions (Kruger and Dunning, 1999; Wu and Dunning, 2017). Reflective thinking can help us recognize our biases, beliefs, heuristics, and values that often drive our efforts and the decisions that we make (Voinov et al., 2014; Glynn et al., 2017). For example, one of the authors collaboratively developed a flooding model to help communities design green infrastructure scenarios, where the modelers had represented flooding as a specific depth of accumulated water (Zellner et al., In Press). In collaborative design workshops with stakeholders, participants noted that there were, in fact, different definitions of flooding, depending on who and what areas were affected. The modeling team then included in the PM interfaces an ability for users to define the depth of water that constituted flooding for them (Zellner, pers. obs.). Another author realized that her own bias towards an older, male participant in a leadership role caused her to interpret his emphasis on responsibility and single-point decision making as authoritarian and power hungry. However, in the context of the stakeholder group—a group of wildland firefighters—it was a sign of caution and a risk management approach. Without this realization, important insights would have been lacking from the PM process.

6. **Ability to measure the success of the PM process:** Building a model collectively is not the final goal of a PM process. Rather, the social learning that model development and use supports, and the actions that learning informs are most important. PM efforts must be evaluated, both formatively and summatively, and results shared (Hoch et al., 2015; Milz et al., 2017; Radinsky et al., 2017). Also, it is critical that modelers participate in measuring and communicating how PM helped communities to make better decisions.

## Lessons learned about the social and political context of collaboration

Stakeholder groups are not monolithic; they have varying organizational structures, multiple and competing objectives, and they often evolve over time. Individual stakeholders cannot represent an entire group; stakeholders and stakeholder groups are dynamic and cross-scalar. Multiple manifestations of a “community” exist within any particular geographic region. Communities can be defined, for instance, by political, religious, genealogical/familial/clan, or by resource management oversight characteristics. Setting manageable boundaries for selection of groups to participate in a modeling initiative, and identifying in which stages different stakeholders should participate, is crucial (Prell et al., 2007; Reed

et al., 2013a, 2013b, 2017). Some stakeholders need to be engaged from the very beginning (Reed et al., 2009; Cormier-Salem, 2014; Sterling et al., 2017). Yet, trying to engage every stakeholder may mean spending a lot of time reaching consensus but not creating a lot of clarity on action (Büscher and de Beer, 2011; Sterling et al., 2017). Experience, context, and clear stakeholder engagement strategies help to overcome these challenges (Sterling et al., 2017).

Navigating power asymmetries is a tricky ethical and logistical issue. On the one hand, when significant power differentials exist, inviting all stakeholders (a “neutral posture”) may simply exacerbate initial power asymmetries if less powerful participants are less able to contribute (Kritek, 2002). On the other hand, excluding or separating the more powerful participants (a “non-neutral posture”) to empower the weaker actors engenders questions regarding how people are chosen (Barnaud and Van Paassen, 2013) and potentially weakens the usefulness of the results of the PM process.

Understanding the social and political context within which one is operating is key. For instance, some stakeholders may have specific expectations about roles and relationships based on previous experiences. Others may have little or no experience in working with modelers on planning and decision-making, and may need extra assistance in learning how to participate effectively. Still others may have engagement fatigue. Governance structures differ around the world, and those structures and norms may change the real and perceived benefits of participating in a PM process. Some groups may be uncomfortable with group decision-making, preferring to defer to figures of authority. And, at times, groups are set on the decisions that they want to make, and, while they may be open to a collaborative engagement, they are resistant to new ideas. Our lessons from and about operating within a variety of social and political contexts include:

1. **In some cultures, knowledge is not shared freely:** Researchers from a western science perspective often assume that people share knowledge, and share it in the same way across cultures. In some cultures, knowledge is considered to be freely available to all. In other cultures, knowledge is held more closely, and may not be shared with others until they have earned the right to learn something (van Kerkhoff, 2013; Matsui, 2015). Knowledge ownership has implications for who participates in various aspects of the process, for how information circulates, what information is available for use in the modeling processes, and if and how that information is used. Knowledge that is not represented in the PM process has little to no chance of informing the resulting decisions. Explicit criteria for a culturally appropriate process of knowledge sharing and use can help ensure these concerns are considered (Conley and Moote, 2003), even if not represented directly in models. As an example, when working in the Pacific, researchers often ask participants to specify who should have access to knowledge used in the model.
This could include restricting to those who provide
the information and the modelers only, adding other
knowledge holders in a particular domain (for in-
stance other fisherpeople), adding other community
members but no one beyond the community, and so
on (Sterling pers. obs.).

2. **Community co-leads are crucial partners in navigating social and political context:** Frequently, it helps to identify a small group of advisors/co-leads from the focal community who can help you navigate the local social and political setting (Hovmand, 2014; Zellner et al., In Press). Selecting appropriate co-leads is a challenge of its own, and it is important to explore systematically who would best fill these roles, and who has appropriate knowledge and trust of the community. As an example, while working in Solomon Islands, one of the authors found that some people who have knowledge that might be of most interest to researchers may be reluctant to engage in a leadership role. In contrast, some of those who were interested in assuming that role in the process lacked the trust of the community (Sterling pers. obs.). It took deep knowledge of the community members and their relationships to determine whom to engage.

3. **Economic or policy gains may not be the (only) benefits participants care about:** Participatory modelers and conveners may assume that their goals for the process are shared by those who engage with them. More specifically, they may assume that reaching a policy decision, implementing an action, and delivering economic gains to the community are important to everyone. This is not universally the case. In our work, we have found that some participants may be guided more by social norms, such as wanting to fit in or enhance their stature in a group or a community. Others may be motivated by a desire to contribute in a positive way to another community or to the next generation or to leave behind a meaningful legacy. Some may desire to preserve the present or the past, to act against the potential disappearance of a culture or way of life, or against the potential extinction of a species, or against the degradation of an environment or ecosystem (Sterling et al., 2017). Some might participate to be at the table, gain first-hand information, or try to influence participants towards their point of view. Some participants may be motivated by all of these factors. It is important for participatory modelers to recognize that the PM process is embedded in this social and political context, and that focusing too narrowly on economic or policy questions may shortchange the stakeholder community.

4. **Many people care deeply about their social standing, which can affect their participation:** PM is a process that naturally engenders group dynamics. Group dynamics (such as “groupthink,” pos- turing and differentiation, attention seeking, and following a confident leader) can sometimes impede critical thinking and independent perspectives that

are essential to improving knowledge and manage-
ment of an issue (Forsyth, 2014). In situations where
group dynamics are very strong and or are poorly
managed, the social standing of individuals who
think differently from the group may be threatened
by speaking counter to the group, and they may re-
duce their participation. Alternatively, people might
only share what they think the researcher or leader
wants to know. For other individuals, social stand-
ing may be increased by being a strong voice in the
group. Unmanaged concerns about social standing
and power imbalances within a group may lead to a
less representative process or model than intended
(Hoch et al., 2015; Zellner et al., In Press).

5. **Methods used, and the way in which they are used, can significantly influence how engaged stakeholders feel:** Not everyone likes structure, or “being taught.” An over-reliance on capturing input in writing (e.g., on flip charts) and on categorization can hamper data collection in societies where storytelling, narrative, and oral information exchange is the norm. Similarly, sharing information using graphs and pie charts can be off-putting for some stakeholders while being very helpful for others. Visual depictions of future scenarios accompanied by vignettes or narratives can be a powerful tool for eliciting discussion, but may also lead people to think of the particular scenarios described as the most likely or only possible futures (Spiegelhalter et al., 2011). Open and active engagement in collaborative design is crucial to tailoring the PM into a process that the community owns, masters, legitimizes, and ultimately uses to guide their actions (Zellner et al., In Press). In Solomon Islands, for instance, use of art-based facilitation elicited responses from community members who might not otherwise have shared perspectives (McCarter et al., 2018).

6. **Conscious and unconscious biases can affect processes, interactions, and models:** PM processes can threaten existing social and political order, and the existing social and political order can threaten the integrity and usefulness of PM. Not everyone is open to learning that their preferred decision may fail. The reluctance can affect the models themselves: people may select data to fit their pre-existing mental model (Tversky and Kahne-man, 1974) or cultural model or decision, or to seek confirmation of their hypotheses (Anderson et al., 1980; Nickerson, 1998; Sébastien and Bauler, 2013; Glynn et al., 2017). This reluctance can affect the acceptance and implementation of PM results: peo-ple may show more distrust when outputs do not match their expectations (Hoch et al., 2015). Many innate prioritizations unconsciously affect the way we make decisions, conduct science, and manage ourselves, our communities, and our environments and resources. Recognizing these prioritizations improves PM (Hämäläinen and Alaja, 2008; Glynn, 2014, 2017; Hämäläinen, 2015; Morewedge et al., 2015; Voinov et al., 2016; Glynn et al., 2017).
## Lessons learned about the PM process

Managing the participation process in PM is as important as managing the model building process. Models within a collaborative process can function as “boundary objects” (Star and Griesemer, 1989; Harvey and Chrisman, 1998), providing a means to bridge ideas across disciplines and participants’ perspectives, and thus promoting learning through collaboration (Akkerman and Bakker, 2011; Zellner et al., 2012). PM processes help stakeholders bring to light assumptions, causes, solutions, and values of which they may have only subconscious awareness. This provides opportunities for users to make ideas visible and open for discussion, negotiation, and revision, and supports constructive discourse. PM models also allow individual and collective cognition to be externalized and made explicit, by mentally offloading difficult tasks into an environment (e.g., computer screens and notebooks) where thinking can be organized and discussed (Bart, 1995; Zellner, 2008). Furthermore, because modeling forces us to explicitly formalize diverse knowledges, ideally individuals coming from different backgrounds should be able to communicate in this shared workspace (van der Leeuw, 2004).

Robust, meaningful, and impactful PM is a huge effort, especially if compared to a more traditional top-down or authoritative decision-making process. It takes time, funding, and iterative engagement to build the relationships and trust among the participants, tools, and process (particularly how to navigate conflict and tradeoffs), in addition to building the models themselves. That said, once the iterative process is established, further collective innovation and action may come more easily and quickly. Participants become used to thinking collaboratively with modeling tools. Slowing down to engage in such processes paradoxically gets communities to where they want to be, faster (Zellner and Campbell, In Press).

Previous experiences with collaborative processes shape participant expectations of PM, however. Community trust in such processes can be eroded if conveners of a past collaborative process did not take their input seriously, or if members are afraid to voice their opinions, perspectives, and knowledge in public. Some processes become driven more by the modelers and by the needs of a quantitative model than the interests and needs of the community. This can establish an expectation that the modelers are “in charge” and lead to less innovation and participation from the community in model development and use.

Lessons we have learned about how to manage these issues include:

1. **PM is not always the right approach:** Evaluating the effectiveness of PM is inherently complex, and there is no universal agreement about its benefits. Perhaps especially because we are participatory modelers, it is crucial to consider explicitly whether the benefits of PM are relevant and sufficiently valuable for the problems being considered to be worth the time and effort PM requires. For example, in cases of emergency one might want to follow a top-down approach rather than having extensive discussions. If the social and political context will not allow true decision-making authority to go to a participatory

group, then it might not be a wise use of time to engage in PM.

2. **Participants' needs, rights, and relationships must be respected and protected:** Free, prior, and informed consent (FPIC) is a principle mandated by international human rights standards that is fundamental to ethical engagement (FAO, 2016). This principle ensures that there is no coercion to stakeholder engagement, that stakeholders have the opportunity to consent or deny engagement prior to commencement of activities, and that they are informed regarding the details of the engagement, including but not limited to scale, scope, purpose, implications, and potential impacts including risks. However, obtaining FPIC can be challenging. Sometimes the process of informed consent raises suspicion, making participants wary of signing documents and prompting questions about risks involved in participation. In the initial stages of collaboration, it is necessary to build trust and explain how informed consent means participants should have the freedom to act and speak without concern about what others outside of the process might think of them. It is important to be aware of power dynamics that favor consent and to allow for critical inquiry and dissent (Forsyth, 2014).

3. **The most informative results arise from fearless and playful collaborative modeling:** The first model is rarely the best model, and one of the underlying motivations for PM is the belief that we get better models, and better and more useful results by engaging the full knowledge of the community. It takes time to build the collaboration and confidence in model conclusions and model limitations (Zellner and Campbell, 2015). Confidence increases as understanding increases, and open, iterative modeling where all questions are explored with an open mind increases that understanding. Processes that create an environment where mistakes are acceptable and even desirable for learning and innovation help build confidence. Art and improvisation exercises can be especially useful to build and connect different pieces, overcome the fear of trial-and-error, create relationships among people who do not know each other well, or strengthen relationships among those who do not trust each other (Pink, 2006).

4. **Stakeholder expectations may exceed the scope of the PM process:** Communication is key among researchers and between researchers and other stakeholders and participants. It is particularly important to understand and communicate clearly about the process, what the outcomes will be, what products will result from the process, and how they will be shared or used. When stakeholders have expectations that cannot be met, they may disengage from the process, and may be more reluctant to engage in future collaborative processes. For instance, when working with herders in Kenya, Reid et al. (2016) suggested that external organizers be clear regarding their ability to meet a request. Especially important is addressing those situations where researchers do
not have expertise and therefore need to reach out to others for relevant resources or to co-generate new knowledge. It is useful to iteratively state, revise, and refine expectations throughout the process, to ensure everyone knows what they are doing and why. This can help maintain a clear definition of the kind of outcomes that are expected, and increases the likelihood of acceptance of the outcomes.

5. **Conflict management may be necessary:** Processes that focus on positions in negotiations often mask the underlying needs people have but cannot articulate (Susskind et al., 1999). A good process will help people identify their motivations, which in some conflict situations might be related to feelings of being disrespected or undervalued, or to disparities in social power among participants (Sturm and Antonakis, 2015; Simpson et al., 2015). Listening to how stakeholders talk about an issue or learning about their relationships with each other can help identify other factors that may be driving disagreements, such as history, identity conflicts, or deep value differences (Madden and McQuinn, 2014). Respectful listening to, and acknowledgment of, opposing points of view can be helpful in resolving differences and conflicts and in fostering innovation, especially with power dynamics in mind (Barnaud and Van Paassen, 2013; Minter et al., 2014). Learning and trust-building, for both scientists and communities, is a critical outcome of collaborative processes, including PM (Susskind et al., 1999; Conley and Wondolleck and Yaffee, 2000; Moote, 2003; Gray et al., 2017).

6. **Exploration and compromise may be more valuable than consensus:** PM generally involves multiple sources of knowledge stemming from diverse knowledge systems, where differing opinions and proposed strategies are common. One knowledge system should not be used to validate or invalidate information from another system. Rather, credible, salient, and legitimate knowledge (as defined by Cash et al., 2003) can be brought to the table and the congruences and differences explored (Tengö et al., 2014). This often involves balancing rigor and flexibility in what constitutes "evidence" (Sterling et al., 2017). Rather than attempting to minimize differences when they arise, it is important to recognize how differences shed light on diverse parts of a system or on the varying trade-offs across solutions (Zellner et al., In Press). Attempts to reach general agreement/consensus across differences can, at times, completely invalidate a set of values or knowledge. True differences can be overcome by focusing on mutually valued goals, or by working with these differences to assess what participants give up and how to compensate for it.

## Conclusions

PM is a powerful approach for addressing complex social and environmental problems. While it holds great promise, it can also come at a high cost: it is difficult

to gather adequate skills, funds, and participants' time, and it takes time to gradually build strong relationships between scientists, community partners, and public agencies. It may require that actors with decision-making responsibilities cede some of their authority to a group and process over which they have little control. Based on the lessons presented above, we conclude our reflection with some cross-cutting recommendations.

Members of a project team should reflect on their abilities with regards to our lessons learned on the role of participatory modelers. All participatory modelers need to be aware of how to support relationship building among participants, and design and use tools accordingly. If critical skills are missing, the project team should first build its own capacity and may consider recruiting project partners with the needed expertise. Successful PM efforts create an inclusive environment that supports participants with differing values, ideas, and priorities. Sharing (time, viewpoints, stories, common challenges, food, small slices of life, etc.) helps to develop these relationships and understanding. Stakeholders know more about the problem, obstacles and opportunities, and the community than do modelers from outside the community. Provide time for people to understand the purpose of the modeling, but more importantly, for participants to share their wants and needs. Modelers are a support to them, to help them achieve their goals and aspirations. Find ways to "check in" regularly and in different ways to see how learning is progressing in individuals and across the collaboration, and how stakeholders are perceiving the collaboration and the issues in general (Bennett, 2016). This includes structured observations of the dialogue and deliberations occurring around the PM tools, and what actions participants have collectively agreed to support. Power dynamics frequently influence how people from different backgrounds interact. Try to understand the motivations underlying participants' behavior, and explicitly acknowledge and legitimize those motivations. Note that researchers are all part of the process, not outsiders taking objective notes from afar, so modelers should also expose their motivations and reflect on their biases.

Whatever tools and techniques are used, make sure to work with a subset of the stakeholders to design the PM with them. Not only will the PM process (tools, location and time, facilitation and deliberation setups) thus be comfortable to users, but also the non-modeler participants become advocates and facilitators of these tools, building trust in the process, and easing the transition towards self-reliance and appropriate tool use to inform decision-making. Provide feedback to stakeholders on model development and, to groups who were not involved in its development, on prior model use and outcomes. Recruit and, if necessary, train different kinds of facilitators (technical/modeling, community) to help manage the social interactions among community participants, between modelers and participants, and between users and the tools developed and used (Hovmand, 2014).

Consider how to deal with dissonance between what participants initially expect and what the model suggests; if the gap is too wide, it can be difficult for participants to
embrace the results and make effective decisions. This discomfort may be compounded by the fact that uncertainty is inherent in any complex problem that is the focus of the PM activity, and is often reflected in the modeling process and outcomes. Participants may equate such uncertainty to ignorance, and thus distrust the modeling effort and its insights, instead of incorporating the uncertainty into the design of robust policies that can be effective in an uncertain world (Zellner 2008). Scaffolding between expectations and results needs to be built into the PM process, in order to address confirmatory bias (Hoch et al., 2015; Zellner et al., In Press).

Keep models simple, relevant, and tractable, distilled to key decision points of importance to the participants (Zellner et al., In Press). The purpose of PM is for stakeholders to participate in the analysis of the complex issues they face, to gain insights relative to their roles in problems and in their solutions, to harness their collective creativity in designing solutions, and to examine and deliberate about the various tradeoffs with appropriate information. If models become intractable or irrelevant “black boxes” that do not adequately represent and make it easy for stakeholders to examine assumptions, goals, and values, then that possibility is lost.

Finally, PM involves the community in understanding and modeling what are typically complex human, social, and natural systems. Although a project may “wrap up,” achieving its decision-specific goals, the community and those complex systems continue to exist and evolve. A valuable result of PM would be to foster ongoing dialogue and collaborative analysis that is adaptive to the inevitable surprises brought about by complex problems and systems. Theoretical frameworks, case studies, and individual experiences with PM and other forms of collaborative governance are signaling what some have identified as a “new governance era” (e.g., Leong et al., 2011). Yet, these approaches are not fully integrated into agency policies and practices (Leong et al., 2011; Zellner and Campbell, 2015) despite notable international efforts and commitments such as the *Local Agenda 21* coming out of the UN Rio Declaration on Environment and Development, the European Union's (EU) *Aarhus Convention on Access to Information, Public Participation in Decision-Making and Access to Justice in Environmental Matters*, and the EU's *Water Framework Directive*. Having overcome the initial hurdles of setting up a PM process, it is worth investing in collaborative stakeholder networks to facilitate ongoing learning and exchanges from peer to peer, outside of PM efforts. Long-term relationships with stakeholders can carry across many complex projects, transferring the ways of thinking across projects and building the capacity to effectively deal with such complexity in different cases.

Our hope is that these lessons are useful to new practitioners who venture on this path, with a realistic view of what PM entails, recommendations of steps to take to ensure a productive process, and the enthusiasm and confidence in this approach as a support for a new form of collaborative governance to grapple with difficult and persistent problems.

## Acknowledgements

Co-authors of this paper are U.S. Government employees, therefore: Any use of trade, product, or firm names in this publication is for descriptive purposes only and does not imply endorsement by the U.S. Government. We thank Nadav Gazit for comments, technical edits, and graphic design, Don Rosenberry and Collin Lawrence for U.S. Geological Survey technical reviews, and Amanda Sigouin for editing support. We appreciate the helpful comments from two outside reviewers and the thoughtful guidance from the editors.

## Funding information

This work was supported by the National Socio-Environmental Synthesis Center (SESYNC) under funding received from the National Science Foundation DBI-1052875. The material is also based upon work supported by the National Science Foundation (NSF) under Grants No. EF-1427091, 1427453, and 1444184. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. In addition, this research was conducted by the Assessing Biocultural Indicators Working Group supported in part by SNAPP: Science for Nature and People Partnership, a collaboration of The Nature Conservancy, the Wildlife Conservation Society and the National Center for Ecological Analysis and Synthesis (NCEAS) at the University of California, Santa Barbara.

## Competing interests

The authors have no competing interests to declare.

## Author contributions

Contributed to conception and design: EJS, MZ, KEJ, KL; Contributed to acquisition of data: All authors; Contributed to analysis and interpretation of data: EJS, MZ, KEJ, KL, PDG; Drafted and/or revised the article: All authors; Approved the submitted version for publication: EJS, MZ.

## References

* Akkerman, SF and Bakker, A. 2011. Boundary crossing and boundary objects. *Rev Educ Res* **81**(2): 132–169. DOI: https://doi.org/10.3102/0034654311404435

* Anderson, CA, Lepper, MR and Ross, L. 1980. Perseverance of social theories: The role of explanation in the persistence of discredited information. *J Pers Soc Psychol* **39**(6): 1037. DOI: https://doi.org/10.1037/h0077720

* Barnaud, C, Le Page, C, Dumrongrojwatthana, P and Trébuil, G. 2013. Spatial representations are not neutral: Lessons from a participatory agent-based modelling process in a land-use conflict. *Environmental Modelling & Software* **45**: 150–159. DOI: https://doi.org/10.1016/j.envsoft.2011.11.016

* Barnaud, C and Van Paassen, A. 2013. Equity, power games, and legitimacy: Dilemmas of participatory natural resource management. *Ecology and*
*Society* **18**(2): 21. DOI: https://doi.org/10.5751/ES-05459-180221

Barreteau, O, Antona, M, D'Aquino, P, Aubert, S, Boissau, S, Bousquet, F, Daré, W, Etienne, M, Le Page, C and Mathevet, R. 2003. Our companion modelling approach. *JASSS-J Artif Soc S* **6**(1): n.p.

Bart, J. 1995. Acceptance criteria for using individual-based models to make management decisions. *Ecol Appl* **5**(2): 411–420. DOI: https://doi.org/10.2307/1942032

Becu, N, Neef, A, Schreinemachers, P and Sangkapitux, C. 2007. Participatory computer simulation to support collective decision-making: Potential and limits of stakeholder involvement. *Land Use Policy* **25**(4): 498–509. DOI: https://doi.org/10.1016/j.landusepol.2007.11.002

Bennett, NJ. 2016. Using perceptions as evidence to improve conservation and environmental management. *Conserv Biol* **30**(3): 582–592. DOI: https://doi.org/10.1111/cobi.12681

Borshchev, A and Filippov, A. 2004. From system dynamics and discrete event to practical agent based modeling: Reasons, techniques, tools. *The 22nd International Conference of the System Dynamics Society, July 25–29*. Oxford, England.

Büscher, B and de Beer, E. 2011. The contemporary paradox of long-term planning for social-ecological change and its effects on the discourse-practice divide: evidence from Southern Africa. *J Environ Plann Man* **54**(3): 301–318. DOI: https://doi.org/10.1080/09640568.2010.506075

Cash, DW, Clark, WC, Alcock, F, Dickson, NM, Eckley, N, Guston, DH, Jäger, J and Mitchell, RB. 2003. Knowledge systems for sustainable development. *Proceedings of the national academy of sciences* **100**(14): 8086–8091. DOI: https://doi.org/10.1073/pnas.1231332100

Charmaz, K and Belgrave, LL. 2007. Grounded Theory. In: Ritzer, G (ed.), *The blackwell encyclopedia of sociology* 4: 25. Oxford, UK: John Wiley & Sons, Ltd. DOI: https://doi.org/10.1002/9781405165518.wbeosg070.pub2

Conley, A and Moote, MA. 2003. Evaluating collaborative natural resource management. *Society and Natural Resources* **16**(5): 371–386. DOI: https://doi.org/10.1080/08941920309181

Cormier-Salem, M-C. 2014. Participatory governance of Marine Protected Areas: A political challenge, an ethical imperative, different trajectories. Senegal case studies. *SAPIENS* **7**(2).

Douglas, EM, Wheeler, SA, Smith, DJ, Overton, IC, Gray, SA, Doody, TM and Crossman, ND. 2016. Using mental-modelling to explore how irrigators in the Murray-Darling Basin make water-use decisions. *J Hydrol: Reg Stud* **6**: 1–12. DOI: https://doi.org/10.1016/j.ejrh.2016.01.035

FAO. 2016. Free Prior and Informed Consent: An indigenous peoples' right and a good practice for local communities. *Manual for Project Practitioners*. FAO. Available at: http://www.fao.org/3/a-i6190e.pdf.

Fischhoff, B. 1995. Risk perception and communication unplugged: Twenty years of process. *Risk Analysis* **15**(2): 137–145. DOI: https://doi.org/10.1111/j.1539-6924.1995.tb00308.x

Forsyth, DR. 2014. *Group Dynamics*. 6th ed. Belmont, CA: Wadsworth Cengage Learning. 34.

Glynn, PD. 2014. W(h)ither the oracle? Cognitive biases and other human challenges of integrated environmental modeling. In: Ames, DP, Quinn, NWT and Rizzoli, AE (eds.), *7th Intl Congress on Env Modeling and Software*. San Diego, USA: International Environmental Modelling and Software Society.

Glynn, PD. 2017. Integrated environmental modelling: Human decisions, human challenges. *Geol Soc London Spec Publ* **408**(1): 161–182. DOI: https://doi.org/10.1144/SP408.9

Glynn, PD, Voinov, AA, Shapiro, CD and White, PA. 2017. From data to decisions: Processing information, biases, and beliefs for improved management of natural resources and environments. *Earth's Future* **5**(4): 356–378. DOI: https://doi.org/10.1002/2016EF000487

Gray, S, Gagnon, A, Gray, S, O'Dwyer, B, O'Mahony, C, Muir, D, Devoy, RJN, Falaleeva, M and Gault, J. 2014. Are coastal managers detecting the problem? Assessing stakeholder perception of climate vulnerability using Fuzzy Cognitive Mapping. *Ocean Coast Manage* **94**: 74–89. DOI: https://doi.org/10.1016/j.ocecoaman.2013.11.008

Gray, S, Jordan, R, Crall, A, Newman, G, Hmelo-Silver, C, Huang, J, Novak, W, Mellor, D, Frensley, T, Prysby, M and Singer, A. 2017. Combining participatory modelling and citizen science to support volunteer conservation action. *Biol Conserv* **208**: 76–86. DOI: https://doi.org/10.1016/j.biocon.2016.07.037

Gray, S, Voinov, A, Paolisso, M, Jordan, R, BenDor, T, Bommel, P, Glynn, P, Hedelin, B, Hubacek, K, Introne, J, Kolagani, N, Laursen, B, Prell, C, Schmitt Olabisi, L, Singer, A, Sterling, E and Zellner, M. 2018. Purpose, processes, partnerships, and products: Four Ps to advance participatory socio-environmental modeling. *Ecol Appl* **28**(1): 46–61. DOI: https://doi.org/10.1002/eap.1627

Hämäläinen, RP. 2015. Behavioural issues in environmental modelling–The missing perspective. *Environ Modell Softw* **73**: 244–253. DOI: https://doi.org/10.1016/j.envsoft.2015.08.019

Hämäläinen, RP and Alaja, S. 2008. The threat of weighting biases in environmental decision analysis. *Ecol Econ* **68**(1–2): 556–569. DOI: https://doi.org/10.1016/j.ecolecon.2008.05.025

Harvey, F and Chrisman, N. 1998. Boundary objects and the social construction of GIS technology. *Environ Plann A* **30**(9): 1683–1694. DOI: https://doi.org/10.1068/a301683

Hoch, C, Zellner, M, Milz, D, Radinsky, J and Lyons, L. 2015. Seeing is not believing: Cognitive bias and modelling in collaborative planning. *Plan Theor Pract* **16**(3): 319–335. DOI: https://doi.org/10.1080/14649357.2015.1045015
Hovmand, PS. 2014. *Community based system dynamics*. 1st ed. New York, USA: Springer-Verlag. DOI: https://doi.org/10.1007/978-1-4614-8763-0

Htun, H, Gray, SA, Lepczyk, CA, Titmus, A and Adams, K. 2016. Combining watershed models and knowledge-based models to predict local-scale impacts of climate change on endangered wildlife. *Environ Modell Softw* **84**: 440–457. DOI: https://doi.org/10.1016/j.envsoft.2016.07.009

Hubacek, K, Beharry, N, Bonn, A, Burt, T, Holden, J, Ravera, F, Reed, M, Stringer, L and Tarrasón, D. 2009. Ecosystem services in dynamic and contested landscapes: The case of UK uplands. In: Winter, M, Lobley, M (eds.), *What is land for? The Food, Fuel and Climate Change Debate*, 167–188. London, UK: Earthscan.

Kaner, S. 2014. *Facilitator's Guide to Participatory Decision-Making*, 3rd ed. Hoboken, New Jersey: Jossey-Bass. 363. (Jossey-Bass Business & Management Series).

Kritek, PB. 2002. *Negotiating at an uneven table: Developing moral courage in resolving our conflicts*. 2nd ed. Hoboken, New Jersey: Jossey-Bass. (Jossey-Bass Health Series).

Kruger, J and Dunning, D. 1999. Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. *J Pers Soc Psychol* **77**(6): 1121. DOI: https://doi.org/10.1037/0022-3514.77.6.1121

Leong, KM, Decker, DJ, Lauber, TB and Chase, L. 2012. Stakeholders as beneficiaries of wildlife management. In: Decker, DJ, Riley, SJ and Siemer, WF (eds.), *Human dimensions of wildlife management*, 2nd ed., 26–41. Baltimore, USA: The Johns Hopkins University Press.

Leong, KM, Decker, DJ, Lauber, TB, Raik, DB and Siemer, WF. 2009. Overcoming jurisdictional boundaries through stakeholder engagement and collaborative governance: Lessons learned from white-tailed deer management in the U.S. In: Andersson, K, Eklund, E, Lehtola, M and Salmi, P (eds.), *Beyond the rural-urban divide: Cross-continental perspectives on the differentiated countryside and its regulation* **360**: 221–247. Bingley, UK: Emerald Publishing Group.

Leong, KM, Emmerson, DP and Byron, R. 2011. The new governance era: Implications for collaborative conservation and adaptive management in department of the interior agencies. *Hum Dimens Wildl* **16**(4): 236–243. DOI: https://doi.org/10.1080/10871209.2011.585436

Madden, F and McQuinn, B. 2014. Conservation's blind spot: The case for conflict transformation in wildlife conservation. *Biol Conserv* **178**: 97–106. DOI: https://doi.org/10.1016/j.biocon.2014.07.015

Martin, R and Schlüter, M. 2015. Combining system dynamics and agent-based modeling to analyze social-ecological interactions—An example from modeling restoration of a shallow lake. *Front Environ Sci* **3**(66): 1–15. DOI: https://doi.org/10.3389/fenvs.2015.00066

Matsui, K. 2015. Problems of defining and validating traditional knowledge: A historical approach. *The International Indigenous Policy Journal* **6**(2). Available at: https://ir.lib.uwo.ca/iipj/vol6/iss2/2. DOI: https://doi.org/10.18584/iipj.2015.6.2.2

McCarter, J, Sterling, E, Jupiter, S, Cullman, G, Albert, S, Basi, M, Betley, E, Boseto, D, Bulehite, E, Harron, R, Holland, P, Horning, N, Hughes, A, Jino, N, Malone, C, Mauli, S, Pae, B, Papae, R, Rence, F, Revo, O, Taqala, E, Taqu, M, Woltz, H and Filardi, CE. 2018. Biocultural approaches to developing well-being indicators in Solomon Islands. *Ecology and Society* **23**(1). DOI: https://doi.org/10.5751/ES-09867-230132

Milz, D, Zellner, M, Hoch, C, Radinsky, J, Pudlock, K and Lyons, L. 2017. Reconsidering scale: Using GIS to inform spatial planning talk. *Planning Practice & Research*, 1–18.

Minter, T, van der Ploeg, J, Pedrablanca, M, Sunderland, T and Persoon, GA. 2014. Limits to indigenous participation: The Agta and the northern sierra Madre Natural Park, the Philippines. *Hum Ecol* **42**(5): 769–778. DOI: https://doi.org/10.1007/s10745-014-9673-5

Morewedge, CK, Yoon, H, Scopelliti, I, Symborski, CW, Korris, JH and Kassam, K. 2015. Debiasing decisions: Improved decision making with a single training intervention. *Policy Insights from the Behavioral and Brain Sciences* **2**(1): 129–140. DOI: https://doi.org/10.1177/2372732215600886

Nayaki, A, Gray, S, Lepczyk, C, Skibins, J and Rentsch, D. 2014. Local-scale dynamics and local drivers of bushmeat trade. *Conserv Biol* **28**(5): 1403–1414. DOI: https://doi.org/10.1111/cobi.12316

Nickerson, RS. 1998. Confirmation bias: A ubiquitous phenomenon in many guises. *Rev Gen Psychol* **2**(2): 175. DOI: https://doi.org/10.1037/1089-2680.2.2.175

Oteros-Rozas, E, Martín-López, B, Daw, TM, Bohensky, EL, Butler, JR, Hill, R, Martin-Ortega, J, Quinlan, A, Ravera, F, Ruiz-Mallén, I, Thyresson, M, Mistry, J, Palomo, I, Peterson, GD, Plieninger, T, Waylen, KA, Beach, D, Bohnet, IC, Hamann, M, Hanspach, J, Hubacek, K, Lavorel, S and Vilardy, S. 2015. Participatory scenario planning in place-based social-ecological research: Insights and experiences from 23 case studies. *Ecol Soc* **20**(4): 32. DOI: https://doi.org/10.5751/ES-07985-200432

Pink, D. 2006. A whole new mind: Why right-brainers will rule the future. New York, USA: Penguin Group.

Prell, C, Hubacek, K, Reed, M, Quinn, C, Jin, N, Holden, J, Burt, T, Kirby, M and Sendzimir, J. 2007. If you have a hammer everything looks like a nail: Traditional versus participatory model building. *Interdiscipl Sci Rev* **32**(3): 263–282. DOI: https://doi.org/10.1179/030801807X211720

Radinsky, J, Milz, D, Zellner, M, Pudlock, K, Witek, C and Lyons, L. 2017. How planners and stakeholders learn with visualization tools: Using learning sciences methods to examine planning processes. *J*
Environ Plann Man **60**(7): 1296-1323. DOI: https://doi.org/10.1080/09640568.2016.1221795

Railsback, SF and Grimm, V. 2012. Agent-based and individual-based modeling: A practical introduction. Princeton, USA: Princeton University Press.

Reed, MS, Graves, A, Dandy, N, Posthumus, H, Hubacek, K, Morris, J, Prell, C, Quinn, CH and Stringer, LC. 2009. Who's in and why? A typology of stakeholder analysis methods for natural resource management. *J Environ Manage* **90**(5): 1933-1949. DOI: https://doi.org/10.1016/j.jenvman.2009.01.001

Reed, MS, Hubacek, K, Bonn, A, Burt, TP, Holden, J, Stringer, LC, Beharry-Borg, N, Buckmaster, S, Chapman, D, Chapman, PJ, Clay, GD, Cornell, SJ, Dougill, AJ, Evely, AC, Fraser, EDG, Jin, N, Irvine, BJ, Kirkby, MJ, Kunin, WE, Prell, C, Quinn, CH, Slee, B, Stagl, S, Termansen, M, Thorp, S and Worrall, F. 2013a. Anticipating and managing future trade-offs and complementarities between ecosystem services. *Ecol Soc* **18**(1): 5. DOI: https://doi.org/10.5751/ES-04924-180105

Reed, MS, Kenter, J, Bonn, A, Broad, K, Burt, T, Fazey, IR, Fraser, EDG, Hubacek, K, Nainggolan, D, Quinn, CH, Stringer, LC and Ravera, F. 2013b. Participatory scenario development for environmental management: A methodological framework illustrated with experience from the UK uplands. *J Environ Manage* **128**: 345-362. DOI: https://doi.org/10.1016/j.jenvman.2013.05.016

Reed, MS, Vella, S, Challies, E, de Vente, J, Frewer, L, Hohenwallner-Ries, D, Huber, T, Neumann, RK, Oughton, EA, del Ceno, JS and van Delden, H. 2017. A theory of participation: What makes stakeholder and public engagement in environmental management work? *Restor Ecol*. DOI: https://doi.org/10.1111/rec.12541

Reid, R, Nkedianye, D, Said, M, Kaelo, D, Neselle, M, Makui, O, Onetu, L, Kiruswa, S, Ole Kamuaro, N, Kristjanson, P, Ogutu, J, BurnSilver, SB, Goldman, MJ, Boone, RB, Galvin, KA, Dickson, NM and Clark, WC. 2016. Evolution of models to support community and policy action with science: Balancing pastoral livelihoods and wildlife conservation in savannas of East Africa. *P Natl Acad Sci USA* **113**(17): 4579-4584. DOI: https://doi.org/10.1073/pnas.0900313106

Schmitt Olabisi, L, Liverpool-Tasie, S, Rivers, L, Ligmann-Zielinska, A, Du, J, Denny, R, Marquart-Pyatt, S and Sidibé, A. 2017. Using participatory modeling processes to identify sources of climate risk in West Africa. *Environment Systems and Decisions* **32**: 23. DOI: https://doi.org/10.1007/s10669-017-9653-6

Sébastien, L and Bauler, T. 2013. Use and influence of composite indicators for sustainable development at the EU-level. *Ecol Indic* **35**: 3-12. DOI: https://doi.org/10.1016/j.ecolind.2013.04.014

Simpson, JA, Farrell, AK, Orina, MM and Rothman, AJ. 2015. Power and social influence in relationships. In: Mikulincer, M and Shaver, PR (eds.), *APA Handbook of Personality and Social Psychology: Interpersonal Relations 3*. Washington, DC, USA: American Psychological Association. DOI: https://doi.org/10.1037/14344-015

Singer, A, Jetter, A, Ellsworth, L, Gray, S, Zhang, P and Oussama Laraichi, M. 2017. Policy scenarios for fire-adapted communities: Understanding stakeholder risk perceptions in Ashland, Oregon. *Project Report*. Portland, USA: Portland State University. Available at: www.mentalmodeling.org.

Spetzler, C, Winter, H and Meyer, J. 2016. Decision quality: Value creation from better business decisions. Hoboken, USA: John Wiley & Sons. DOI: https://doi.org/10.1002/9781119176657

Spiegelhalter, D, Pearson, M and Short, I. 2011. Visualizing uncertainty about the future. *Science* **333**(6048): 1393-1400. DOI: https://doi.org/10.1126/science.1191181

Star, SL and Griesemer, JR. 1989. Institutional ecology, translations' and boundary objects: Amateurs and professionals in Berkeley's Museum of Vertebrate Zoology, 1907-39. *Soc Stud Sci* **19**(3): 387-420. DOI: https://doi.org/10.1177/030631289019003001

Sterling, EJ, Betley, E, Sigouin, A, Gomez, A, Toomey, A, Cullman, G, Malone, C, Pekor, A, Arengo, F, Blair, M, Filardi, C, Landrigan, K and Porzecanski, AL. 2017. Assessing the evidence for stakeholder engagement in biodiversity conservation. *Biol Conserv* **209**: 159-171. DOI: https://doi.org/10.1016/j.biocon.2017.02.008

Sturm, RE and Antonakis, J. 2015. Interpersonal power: A review, critique, and research agenda. *J Manag* **41**(1): 136-163. DOI: https://doi.org/10.1177/0149206314555769

Susskind, L, McKearnan, S and Thomas-Larmer, J. 1999. The Consensus Building Handbook: A Comprehensive Guide to Reaching Agreement. Thousand Oaks, USA: SAGE Publication Ltd.

Tengö, M, Brondizio, ES, Elmqvist, T, Malmer, P and Spierenburg, M. 2014. Connecting diverse knowledge systems for enhanced ecosystem governance: The multiple evidence base approach. *Ambio* **43**(5): 579-591. DOI: https://doi.org/10.1007/s13280-014-0501-3

Tversky, A and Kahneman, D. 1974. Judgment under uncertainty: Heuristics and biases. *Science* **185**(4157): 1124-1131. DOI: https://doi.org/10.1126/science.185.4157.1124

Van Der Leeuw, SE. 2004. Why model? *Cybernet Syst* **35**(2-3): 117-128. DOI: https://doi.org/10.1080/01969720490426803

van Kerkhoff, L. 2013. Knowledge governance for sustainable development: A review. *Challenges in Sustainability* **1**(2): 82-93. DOI: https://doi.org/10.12924/cis2013.01020082

Voinov, A and Bousquet, F. 2010. Modelling with stakeholders. *Environ Modell Softw* **25**(11): 1268-1281. DOI: https://doi.org/10.1016/j.envsoft.2010.03.007
**Voinov, A** and **Brown Gaddis, EJ.** 2008. Lessons for successful participatory watershed modeling: A perspective from modeling practitioners. *Ecol Model* **216**(2): 197–207. DOI: https://doi.org/10.1016/j.ecolmodel.2008.03.010

**Voinov, A**, **Kolagani, N**, **McCall, MK**, **Glynn, PD**, **Kragt, ME**, **Ostermann, FO**, **Pierce, SA** and **Ramu, P.** 2016. Modelling with stakeholders–next generation. *Environ Modell Softw* **77**: 196–220. DOI: https://doi.org/10.1016/j.envsoft.2015.11.016

**Voinov, A**, **Seppelt, R**, **Reis, S**, **Nabel, JE** and **Shokravi, S.** 2014. Values in socio-environmental modeling: Persuasion for action or excuse for inaction. *Environ Modell Softw* **53**: 207–212. DOI: https://doi.org/10.1016/j.envsoft.2013.12.005

**Wu, K** and **Dunning, D.** 2017. Hypocognition: Making sense of the landscape beyond one's conceptual reach. *Rev Gen Psychol* **1**(2): 25–35. DOI: https://doi.org/10.1037/gpr0000126

**Wondolleck, JM** and **Yaffee, SL.** 2000. Making collaboration work: Lessons from innovation in natural resource management. Washington, DC, USA: Island Press.

**Zellner, ML.** 2008. Embracing complexity and uncertainty: The potential of agent-based modeling for environmental planning and policy. *Plan Theor Pract 9*(4): 437–457. DOI: https://doi.org/10.1080/14649350802481470

**Zellner, ML** and **Campbell, SD.** 2015. Planning for deep-rooted problems: What can we learn from aligning complex systems and wicked problems? *Plan Theor Pract 16*(4): 457–478. DOI: https://doi.org/10.1080/14649357.2015.1084360

**Zellner, ML** and **Campbell, S.** n.d. Planning with(in) complexity: Pathways to extend collaborative planning, incremental planning, and Big Data with complex systems modelling. In: de Roo, G, Yamu, C and Zuidem, C (eds.), *Research handbook on planning and complexity*. Cheltenham, UK: Edward Elgar Publishing. In Press.

**Zellner, ML**, **Keller, J** and **Massey, D.** 2018. Robbins Renewal and Resiliency Project Management. Report to the Chicago Community Trust, Chicago, IL: University of Illinois at Chicago.

**Zellner, ML**, **Lyons, LB**, **Hoch, CJ**, **Weizeorick, J**, **Kunda, C** and **Milz, DC.** 2012. Modeling, learning, and planning together: An application of participatory agent-based modeling to environmental planning. *URISA Journal* **24**(1): 77–93.

**Zellner, ML**, **Lyons, L**, **Milz, D**, **Shelley, J**, **Hoch, C**, **Massey, D** and **Radinsky, J.** n.d. Participatory complex systems modeling for environmental planning: Opportunities and barriers to learning and policy innovation. In: Porter, WF, Zhao, J, Schmitt Olabisi, L and McNall, M (eds.), *Innovations in collaborative modeling*. East Lansing, USA: Michigan State University Press. In Press.

**How to cite this article:** Sterling, EJ, Zellner, M, Jenni, KE, Leong, K, Glynn, PD, BenDor, TK, Bommel, P, Hubacek, K, Jetter, AJ, Jordan, R, Olabisi, LS, Paolisso, M and Gray, S. 2019. Try, try again: Lessons learned from success and failure in participatory modeling. *Elem Sci Anth*, **7**: 9. DOI: https://doi.org/10.1525/elementa.347

**Domain Editor-in-Chief:** Anne R. Kapuscinski, University of California, US

**Associate Editors:** Kim A. Locke, Geographical Sciences, University of Maryland, US; Mark Borsuk, Duke University, US

**Knowledge Domain:** Sustainability Transitions

**Submitted:** 14 March 2018

**Accepted:** 27 January 2019

**Published:** 14 February 2019

**Copyright:** © 2019 The Author(s). This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See http://creativecommons.org/licenses/by/4.0/.