# Serious games and citizen science; from parallel pathways to greater synergies

Erika Speelman, Elena Escano, Diego Marcos, Nicolas Becu

► To cite this version:

Erika Speelman, Elena Escano, Diego Marcos, Nicolas Becu. Serious games and citizen science; from parallel pathways to greater synergies. Current Opinion in Environmental Sustainability, 2023, 64, pp.101320. 10.1016/j.cosust.2023.101320. halshs-04185518

HAL Id: halshs-04185518

https://shs.hal.science/halshs-04185518

Submitted on 22 Aug 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
# Serious games and citizen science; from parallel pathway synergies

Erika N. Speelman<fnref n="1" />*, Elena Escano<fnref n="2" />, Diego Marcos<fnref n="3" />, Nicolas Becu<fnref n="4" />

*corresponding author: erika.speelman@wur.nl

## ORCID:

Erika N. Speelman: 0000-0003-1842-4939

Elena Escano: 0000-0001-7412-5912

Diego Marcos: 0000-0001-5607-4445

Nicolas Becu: 0000-0002-7463-2719

## Abstract

Citizen science and serious games are participatory methods that gained traction in recent years especially in the environmental sciences in the context of natural resource management. The two approaches interact with both instrumental and relational values of decision-making and appear to have similar objectives in supporting the development of a shared understanding of the environmental impacts of human land-use decisions with a larger and more active role for stakeholders. One can argue that these two methods have been on seemingly parallel pathways of development. In this paper, we present a brief overview of these developments and explore the complementarity and potential synergies between the two approaches to further strengthen the engagement and empowerment of the public in science.

**Keywords:** Stakeholder participation, Public engagement, Natural resource management, Gamification, Social change

<footnote n="1">¹ Laboratory of Geo-Information Science and Remote Sensing, Environmental Sciences, Wageningen
University & Research, 6708 PB Wageningen, The Netherlands</footnote>

<footnote n="2">² Wageningen Food Safety Research, Wageningen University and Research, P.O. Box 230, 6700 AE
Wageningen, The Netherlands</footnote>

<footnote n="3">³ Inria, University of Montpellier, 34090, Montpellier, France</footnote>

<footnote n="4">⁴ Laboratory Littoral, Environment and Societies (UMR LIENSs 7266), French National Centre for
Scientific Research (CNRS) & La Rochelle University, 2 rue Olympe de Gouges, 17000, La
Rochelle, France</footnote>
# 1 Introduction

In today's ever tighter-knit world, the governance of landscapes and the management of the natural resources within are increasingly complex with a host of multi-level competing interests and claims [1-3]. As a result, steering resource and landscape management towards increased sustainability is recognized as one of the biggest and most important challenges of our times [4,5]. Decision-making in these highly complex social-ecological landscapes is not only challenging due to difficult trade-offs between social, economic, and ecological interests and their consequences (the so-called instrumental values), but also due to the multitude of stakeholders with often a matching number of (conflicting) individual and group perspectives on the value of nature for humans and the impacts on human relations as a result (the so-called relational values) [3,6]. Both the instrumental perspectives as well as the relational perspectives on values play an important role in decision-making [3]. However, more often than not relational values and relational rationality are not taken into account or only in a rudimentary form while instrumental values and rationality are viewed as superior [7]. However, new exciting insights from the field of cognitive science show the role and importance of engaging the social mind, recognizing the power of values, experience and emotions when aiming to change minds and ultimately changing behavior and decisions [8].

Against this background, and in light of current increasing demands towards more science for impact, a trend towards more public engagement in research is seen [9-12]. This increased engagement empowers communities to become more vocal and contribute to e.g. landscape-level planning and decision-making processes [13]. In the domains of natural resource management and landscape governance, two methods, in particular, have become increasingly popular, namely citizen science and serious games. While these two approaches often have similar overall goals, they have different starting points. Both aim for a deeper and stronger shared understanding of conditions and trends, awareness raising, a greater role for citizens and local stakeholders in decision-making and the emergence of innovative, 'out-of-the-box' solutions, not constrained by current rules and land-use concepts but supported by concepts such as 'co-production of knowledge' and 'co-management'. However, the citizen science and serious games communities of practice have developed separately, with seemingly distinct geographical foci and different levels of public participation. While citizen science projects seem strongly focused on citizens' involvement as sensors and data collectors with the objective to enlarge scientific knowledge, serious gaming projects often aim to contribute to (local) processes of change through in-depth interactions with stakeholders using co-developed games as boundary objects in interactive game sessions. Facilitating social learning among stakeholders with the ultimate goal to impact real life decisions and move systems towards increased sustainability is a common objective in serious gaming applications [14-17]. In addition, citizen science seems to be more frequently used in the global north, with serious gaming approaches being more often developed and applied in the global south with the two rarely being used in conjuncture.
In this paper, we provide an overview of the status and development of the fields of citizen science and serious games by exploring the seemingly parallel pathways of development and application of these two approaches. Based on the overview, we aim to explore the current lack of synergy between citizen science and serious gaming approaches. We identify and propose opportunities for greater complementarity and synergy between citizen science and serious games as different ways of linking specific, place-based pattern information to generic, process-based concepts, in support of locally desirable system change.

## 2 Citizen science and serious games: parallel developed pathways

### Citizen science

The first research projects in which the public actively contributed by collecting data date back to the first bird counting activities in the late 1800 and early 1900 in the USA and in the UK. However, the concept of ‘citizen science’ was not introduced until the late 1980s. Citizen science can be defined as the active participation and contribution of the general public in research activities to increase scientific knowledge [12]. While this definition leaves space for a variety of citizen contributions, the majority of citizen science projects reported on, confine the role and contribution of citizens to data collection and monitoring of natural phenomena or of the status of specific natural resources. A distinction can be made between often large-scale resource monitoring mainly led by scientists and community monitoring often led by concerned citizens in their own communities [13]. The method’s strong origin in natural science still results in many of the recent publications falling within the realm of natural sciences focusing on data collection and data monitoring of natural phenomena [18]. Some recent examples include the monitoring of: North American butterflies [19], South African birds [20], mosquitoes in Germany [21], pollinating insects in France [22,23], ticks in the Netherlands [24] nitrogen dioxide [25,26], and natural resource systems and common pool resources [13,27]. However, with the many success stories from the natural sciences, the method’s popularity increased also outside the natural science domain with applications in other domains mushrooming, from cases related to food safety, such as studies of urban gardens identifying trace metal contamination risks [28], to health equity promotion [29] or highlight sources of pollution in urban rivers [30].

Advantages of involving citizens in scientific endeavors include a larger spatial and temporal reach in data collection resulting in a larger sample providing a more solid evidence base with increased awareness of particular scientific issues [13,27], and strengthened scientific empowerment of citizens often mentioned as secondary benefits. Also improving the data collection techniques is often seen as a large advantage [13,27], especially with the development of online interactive tools and mobile phone applications such as eBird [31] or the Zooniverse platform [32,33]. The progress made has allowed for the involvement of a considerable number of individuals in data collection activities, significantly
enhancing accessibility for the general public to participate. Even though getting large numbers of people to contribute became much easier with the onset of these tools, keeping 'citizen scientists' engaged to establish continuous data collection remains challenging resulting in oscillating data quantity and quality posing new challenges for data analysis and interpretation [34]. Wavering or oscillating levels of engagement and commitment of the public in citizen science projects seem to be particularly an issue for projects targeting large audiences that will experience little to no direct impact of their efforts in their daily lives. Relatively new in the field of citizen science is the use of machine learning approaches [35], particularly in projects related to ecology and biodiversity such as iNaturalist [36] and Pl@ntNet [37]. Machine learning opens new avenues for analyzing these large citizen science data sets.

With the expansion and development of the field, the concept of citizen science evolved, similar to that of participatory research the role or citizens grew, allowing the potential expansion of the method's objectives beyond mere data collection [38,39]. Shirk et al. proposed five distinct models of citizen science on the basis of the level of citizen participation and decision-making power, in which three models of collaboration hold a larger and more equal role for citizens, namely collaborative, co-created, and collegial citizen science [39]. These newer forms models of citizen science, though still a minority, allow the public to participate e.g. in the interpretation and exploitation of the collected data, but in some cases also in the design of the study and even the determination of the study. As a result of increased public engagement through citizen science, the general public becomes more aware and more informed about developments in specific scientific topics scientific fields and scientific research in general facilitating the scientific empowerment of the general public.

## Serious gaming

Serious games are often described as games for serious purposes beyond pure entertainment values [40]. These games are models of reality commonly depicting a real-life system or elements thereof including the associated dynamics in a condensed, and often stylized manner, thereby providing simplified worlds from which learners can step back and understand the structure of the real world [41]. They cut off the bewildering complexity of real life by immersing participants in a demonstration of its possibilities, allowing participants to experience first-hand concepts that may sometimes seem abstract, such as organization, power, negotiation, etc. As such, serious games are increasingly recognized as universal learning tools developed to function as boundary objects to facilitate learning [42] by allowing participants to openly interact and by triggering engagement while stimulating discussion among participants. Through game play, participants can explore the effects and consequences of their decisions in the safety of the game setting. Two virtues are generally attributed to the use of serious games: engagement and learning. Classically, it is said that participation in serious games increases interest in the simulated subjects. From the perspective of psychologists and pedagogues, two elements structure player engagement: on the one hand, the subjective enjoyment of the game experience, and on the other
hand, the motivations to play the game [43]. By involving cognitive skills, and multiple senses in game play with others, games contribute to learning processes in which substantive, experiential and social learning all play a role [14,44].

The first steps in using games in academic settings are commonly attributed to Richard Duke who pioneered the field in the 1960sand 1970s [45]. While the games as a (scientific) research method has been around for about half a century, the term “serious games” became only commonly used in the last decade (Figure 1). Duke focused his research on games on developing games and exploring game use in educational settings. While educational games still form a large part of the current body of literature on serious games, many other fields have also discovered the engaging power of games. Over the past decade, the development and application of serious games have spread across many fields of academic research, as well as to its preferred partners, whether institutional partners or climate transition actors [9,11,46-50]. As an example, Gerber and his co-authors [51] identified 115 games developed in recent years that deal with climate change, most of them being used for educational purposes. In the field of natural sciences and natural resource management, games often represent real life social-ecological systems and are used for educational or training purposes, as well as interventions aiming to steer towards more informed decision-making processes and to facilitate discussions on a variety of often difficult, complex and sensitive topics [52-54]. Thanks to the increased interest in game-based research and the growing community, practices evolved in recent years towards more integration of stakeholders in the design phases and especially for the selection of data and dilemmas to be represented in serious games. Driven by participatory modeling [55,56] and co-design [57,58], game practitioners find in this upstream integration of stakeholders a way to improve game design and usability [59] and, ultimately, the richness of the exchanges between players about the issues at stake [60]. Thus, serious games form an excellent tool for transdisciplinary research due to the ability of games to integrate diverse forms of knowledge and create and preserve an environment in which relational values in decision-making processes are fostered and can be explored [61,62].

## Parallel pathways

In recent years the interest in citizen science and serious games has grown dramatically [9,10]. We performed some simple bibliographic searches that confirmed this trend and showed a steady increase in the number of publications in both fields with a particular sharp increase in the number of publications on citizen science since 2018 (Figure 1a) (for more information see Appendix 1). When looking at the origin of the papers, we see a clear division between research originating from the Global North and from the Global South [63]. Where around 60% of the publications originating from the Global North focused on citizen science, we see the opposite for the Global South, where roughly 60% of the papers used a serious gaming approach (Figure 1b, 1c). Zooming into literature in the domain of natural resource management, identified also content-level differences between the two bodies of literature by
looking at the title, abstract, and keywords of the papers (Figure 2). This clearly shows that the literature on citizen science is still very much focused on data collection and monitoring of natural resources, while the literature on serious games focuses more on learning.

<figure><img src="image_1.png" /><figcaption>**Figure 1:** Results of two bibliographic searches into: 'citizen science' and 'serious games' showing: a) cumulative number of papers in the last decade (from 2010 to 2021); b) proportion of published articles on 'citizen science' and 'serious games' with the first author from the global south; c) proportion of published articles on 'citizen science' and 'serious games' with the first author from the global north. Searches were executed in Scopus (searches last executed 02-01-2023), for more information, see Appendix I).</figcaption></figure>

<figure><img src="image_2.png" /><figcaption>**Figure 2:** Word cloud visualization results of two bibliographic searches in the domain of natural resources management into: a) 'citizen science' and, b) 'serious games'. The font size of the words is positively related to the frequency of the keywords used. Searches were executed in Scopus and analyzed using python "wordcloud" package (executed 15-01-2023).</figcaption></figure>
3 Serious games and citizen science: potential synergies

Citizen science and serious gaming have proven capable of engaging non-scientists in scientific research, be it stakeholders in specific study areas and contexts or the general public. While the overall objectives of these methods are much alike, citizen science generally has a stronger emphasis on instrumental perspectives through data collection and data monitoring of human impacts on natural phenomena with the main goal of improving scientific research especially where there is a great interest in discovering and understanding spatial and temporal trends [30]. Creating awareness among the public about the issues at hand though significant, is commonly of second order importance in these studies [64], while, the improvement of community awareness and positive attitudes towards joint action are crucial factors for the successful implementation of large-scale monitoring efforts and ultimately for the future of the approach [25,26]. Whereas serious gaming approaches in the context of natural resource management focus strongly on relational perspectives by creating space to experience, discuss, and (collectively) explore the complex multi-faceted decision-making processes in natural resource management contexts. One of the common objectives of serious gaming applications is to stimulate (social) learning to contribute to better informed decision-making processes, while scientific contributions of many of these studies often remain anecdotal as objective assessment methods remain largely lacking [10]. With the evolution of both methods over time, the conceptual basis of citizen science broadened up allowing for more and different forms and levels of citizen participation [38,39] while in the case of serious gaming, the call for evidence or proof of the impacts of games on learning and decision-making processes became louder [9,10,65]. As a result, the conceptual gap between the two methods seems to have narrowed and interesting synergies between citizen science and serious gaming can be envisioned allowing for a fuller understanding including both instrumental and relational perspectives. In some cases, using the two methods in conjunction by developing, for instance, more data-driven games based on data collected and monitored through citizen science could tackle the skepticism against the use of “simple” games [66], while using games or game elements could strentgthen the commitment and willingness for long term participation in citizen science projects.

Where citizen science approaches have the capacity to easily reach out and engage with large numbers of people with often minimal required efforts and commitment from participants, these projects often struggle to keep the momentum and keep the contributors engaged in the long-term [67]. It is said to be a challenge to develop strategies for ensuring project continuity and sustaining community engagement [68]. In serious gaming projects, the opposite is often true, the initial phase of getting people to come to the table to play has often proven to be challenging [54,69,70], as participation requires openness and active involvement of its players. In addition, people are often skeptical at first sight of the potential lack of seriousness of playing games [62,66]. Moving in the direction of more data driven games by
making use of data collected using citizen science could take away some of that skepticism. Barreteau et al. [71] explain that among the different ways of involving stakeholders in the model design process with a game being a model of reality, two types of co-development grant control over the use of the model to the stakeholders: "Dialogue with modelers and control over model use" and "Co-building of a model and control over model use". This control allows stakeholders to evaluate and become aware of the domain of validity of the model. This is what interests us here, as the joint use of citizen science and serious games could allow actors to contribute to data collection and evaluate the suitability of the data, in relation to the intention of the game they are co-developing. We see this as a potential way to improve the reliability of data-driven games and to improve the quality of exchanges during the games via the production of refutable arguments.

We believe that next to the two methods being used as standalone approaches, the combination of the two approaches can stimulate participation and reinforce the engagement and (long term) commitment of participants throughout the pursued research agenda while strengthening and benefitting from the strong aspects of each of the approaches (Figure 3). Gamification, as the concept of using game elements outside the context of games to establish and maintain engagement [72], holds a potential key role in developing a patchwork of citizen science and serious gaming approaches (Figure 3). Gamified elements have been proposed as a way to enhance user engagement in citizen science efforts and has successfully been applied with varying levels of gamification such as in Seek by iNaturalist (https://www.inaturalist.org/) and Foldit (http://fold.it/). It has been recognized that gamification holds promise to make behavioral science better facilitating more structured and objective ways to collect information from larger samples [73]. A gamified citizen science approach could be especially useful in projects concerning issues that are at the beginning of the issue-attention cycle when data discovery is needed to gain insights, as well as in the monitoring and evaluation phase when data monitoring efforts are key and often tricky to keep up (Figure 3). Especially, with the enlarged complexity of the world increased citizen contribution through transdisciplinary multi-method approaches [74] like our proposal is required.

To illustrate how such potential integration could work, we sketch the case of an endangered animal species that forages in a rural landscape where local farmers grow crops and heard their cattle, like for instance the tiger in Nepal. Several organizations work towards protecting the tiger in these landscapes, but the presence of the tiger threatens the lives and livelihoods of people living in the area, while it provides a new source of income for others through tourism. To address this issue a thorough understanding of the movements and damages of the tiger population needs to be gained. Citizen science could prove to be useful in this phase to for instance map tiger and damage sighting. This information would be input to the development of a serious games in which stakeholders can discuss the current conflict of interests and explore possible alternatives. The issues discussed and scenarios played out in
the game sessions, can in turn be used again to set out potential new measures for monitoring of
disturbance and threat by the tigers.

<figure><img src="image_3.png" /></figure>

**Figure 3:** Conceptualization of a multi-method approach of citizen science, serious gaming and gamification efforts within transdisciplinary research in natural resource management contexts, matching the desired level of engagement and (long term) commitment along the issue-attention cycle (adaptation of [10]). In the early (stage 1 and 2) and later stages (stage 4 and 5) citizen science can be gamified to gain and maintain attention and interest of citizen scientists. The collected information in stage 1 and 2 can input for the development of serious games (stage 3 and 4), while the output of the game sessions can be used to lead the data collection for monitoring in stage 4 and 5.

**Conclusions**

In light of the many daunting challenges the world faces today, steering natural resource management
and land-use systems towards increased sustainability is crucial, yet ever more difficult due to the
increased complexity of the world resulting in ever more competing claims and interests and the associated perspectives and values. Developing and using methods that allow for both instrumental and relational perspectives in land-use decision-making processes to be included through the participation of stakeholders and the general public, such as citizen science and serious gaming approaches, is therefore key. These two methods have developed in parallel in separate communities of practice largely in different parts of the world. Over time both approaches evolved seemingly narrowing the conceptual gap between them. However, so far the methods remain rarely used in conjunction. We propose that next to the independent use of citizen science and serious games approaches, creating a patchwork of citizen science and serious gaming efforts with gamification as a connector would combine the strengths of each of the two methods. We believe that combining citizen science and serious gaming could facilitate and satisfy the need for specific, place-based pattern information and strong engagement with (local) stakeholders to co-produce knowledge for locally desirable system change.

**Funding**

This work was (partially) funded by the Wageningen University INREF-SESAM research programme "Scenario Evaluation for Sustainable Agro-forestry Management" (for more information: sesam.wur.nl & www.wur.eu/INREF).

**Competing Interests**

There are no conflicting interests.

**Author contributions:**

**ES** conceptualized the paper, wrote the original draft, contributed to bibliographic searches and the visualizations; **EE** executed and analyzed the bibliographic searches, reviewed and edited the final draft; **DM** executed and analyzed the bibliographic searches, developed the data visualizations, reviewed and edited the final draft; **NB** contributed to the draft contributed to bibliographic searches and reviewed and edited the final draft.

**References**

1. Cash DW, Adger WN, Berkes F, Garden P, Lebel L, Olsson P, Pritchard L, Young O: **Scale and cross-scale dynamics: governance and information in a multilevel world.** *Ecology and society* 2006, **11**.

2. Folke C, Polasky S, Rockström J, Galaz V, Westley F, Lamont M, Scheffer M, Österblom H, Carpenter SR, Chapin FS: **Our future in the Anthropocene biosphere.** *Ambio* 2021, **50**:834-869.
3. Meine van Noordwijk GJH, Grace B Villamor, Erika N Speelman: **Relational versus instrumental perspectives on values and resource management decisions.** *Current Opinion in Environmental Sustainability* under review.

4. Löbmann MT, Maring L, Prokop G, Brils J, Bender J, Bispo A, Helming K: **Systems knowledge for sustainable soil and land management.** *Science of the Total Environment* 2022, **822**:153389.

5. Seppelt R, Verburg PH, Norström A, Cramer W, Václavík T: **Focus on cross-scale feedbacks in global sustainable land management.** *Environmental Research Letters* 2018, **13**.

6. West S, Haider LJ, Stålhammar S, Woroniecki S: **A relational turn for sustainability science? Relational thinking, leverage points and transformations.** *Ecosystems and people* 2020, **16**:304-325.

7. Githinji M, van Noordwijk, M., Muthuri, C., Speelman, E.N., Hofstede, G.J. : **Farmer land-use decision-making from an instrumental and relational perspective.** *Current Opinion in Environmental Sustainability* under review.

8. Toomey AH: **Why facts don't change minds: Insights from cognitive science for the improved communication of conservation research.** *Biological Conservation* 2023, **278**:109886.

** This paper discusses the limited impact science and science communication has had so far on changing behaviour by examining four common myths about science communication science. The paper provides four alternative insights to improve science communication for impact from new behavioral science research.

9. Den Haan R-J, Van der Voort MC: **On evaluating social learning outcomes of serious games to collaboratively address sustainability problems: A literature review.** *Sustainability* 2018, **10**:4529.

10. Rodela R, Speelman, E.N.: **Impact assessment of serious games; a blind spot in an upcoming field?** *Current Opinion in Environmental Sustainability* under review.

11. Speelman EN, Rodela R, Doddema M, Ligtenberg A: **Serious gaming as a tool to facilitate inclusive business; a review of untapped potential.** *Current opinion in environmental sustainability* 2019, **41**:31-37.

12. Vohland K, Land-Zandstra A, Ceccaroni L, Lemmens R, Perelló J, Ponti M, Samson R, Wagenknecht K: **The science of citizen science**: Springer Nature; 2021.

13. Danielsen F, Eicken H, Funder M, Johnson N, Lee O, Theilade I, Argyriou D, Burgess ND: **Community monitoring of natural resource systems and the environment.** *Annual Review of Environment and Resources* 2022, **47**:637-670.

* This paper debates community monitoring of natural resources. Nowadays, community monitoring that addresses the environmental crisis is growing, while positive impacts are recognized, current challenges are also seen and discussed.

14. Bandura A, Walters RH: *Social learning theory*, vol 1: Englewood cliffs Prentice Hall; 1977.

15. Best L, Fung-Loy K, Ilahibaks N, Ramirez-Gomez SO, Speelman EN: **Toward inclusive landscape governance in contested landscapes: exploring the contribution of participatory tools in the Upper Suriname River Basin.** *Environmental Management* 2021, **68**:683-700.

16. Hossard L, Schneider C, Voltz M: **A role-playing game to stimulate thinking about vineyard management practices to limit pesticide use and impacts.** *Journal of Cleaner Production* 2022, **380**:134913.

17. Moojen FG, de Faccio Carvalho PC, dos Santos DT, Neto AB, Vieira PC, Ryschawy J: **A serious game to design integrated crop-livestock system and facilitate change in mindset toward system thinking.** *Agronomy for Sustainable Development* 2022, **42**:1-12.

18. Lemmens R, Falquet G, Tsinaraki C, Klan F, Schade S, Bastin L, Piera J, Antoniou V, Trojan J, Ostermann F: **A conceptual model for participants and activities in citizen science projects.** *The Science of Citizen Science* 2021:159-182.
19. Crossley MS, Smith OM, Berry LL, Phillips-Cosio R, Glassberg J, Holman KM, Holmquest JG, Meier AR, Varriano SA, McClung MR: **Recent climate change is creating hotspots of butterfly increase and decline across North America.** *Global Change Biology* 2021, **27**:2702-2714.

20. McPherson SC, Sumasgutner P, Downs CT: **South African raptors in urban landscapes: a review.** *Ostrich* 2021, **92**:41-57.

21. Pernat N, Kampen H, Jeschke JM, Werner D: **Citizen science versus professional data collection: Comparison of approaches to mosquito monitoring in Germany.** *Journal of Applied Ecology* 2021, **58**:214-223.

*This paper explores the differences between two methods for data collection on mosquito's in Germany, namely professional data collection and citizen science. Significant differences in species composition were found between data collected through the twp methods

22. Bedessem B, Julliard R, Montuschi E: **Measuring epistemic success of a biodiversity citizen science program: A citation study.** *Plos one* 2021, **16**:e0258350.

23. Bedessem B, Torres A-C, Fontaine C, Deguines N: **Science learning in biodiversity citizen science: Inputs from the analysis of online social interactions within a contributory project for pollinators' monitoring.** *Biological Conservation* 2022, **276**:109807.

24. Hartemink N, van Vliet AJ, Gort G, Gassner F, Jacobs F, Fonville M, Takken W, Sprong H: **Seasonal patterns and spatial variation of Borrelia burgdorferi (sensu lato) infections in Ixodes ricinus in the Netherlands.** *Parasites & Vectors* 2021, **14**:1-16.

25. Perelló J, Cigarini A, Vicens J, Bonhoure I, Rojas-Rueda D, Nieuwenhuijsen MJ, Cirach M, Daher C, Targa J, Ripoll A: **Large-scale citizen science provides high-resolution nitrogen dioxide values and health impact while enhancing community knowledge and collective action.** *Science of the Total Environment* 2021, **789**:147750.

26. Perelló J, Targa J, Daher C, Alonso M, Francis L, Masha-Caminals A, Bonhoure I, Cigarini A, Vicens J, Carrasco-Turigas G: **Large-scale citizen science protocol provides high-resolution nitrogen dioxide values while enhancing community knowledge and collective action.** *MethodsX* 2021, **8**:101475.

27. Ferraro PJ, Agrawal A: **Synthesizing evidence in sustainability science through harmonized experiments: Community monitoring in common pool resources.** *Proceedings of the National Academy of Sciences* 2021, **118**:e2106489118.

28. Taylor MP, Isley CF, Fry KL, Liu X, Gillings MM, Rouillon M, Soltani NS, Gore DB, Filippelli GM: **A citizen science approach to identifying trace metal contamination risks in urban gardens.** *Environment International* 2021, **155**:106582.

29. Rosas LG, Rodriguez Espinosa P, Montes Jimenez F, King AC: **The role of citizen science in promoting health equity.** *Annual review of public health* 2022, **43**:215-234.

30. Hegarty S, Hayes A, Regan F, Bishop I, Clinton R: **Using citizen science to understand river water quality while filling data gaps to meet United Nations Sustainable Development Goal 6 objectives.** *Science of The Total Environment* 2021, **783**:146953.

31. Sullivan BL, Wood CL, Iliff MJ, Bonney RE, Fink D, Kelling S: **eBird: A citizen-based bird observation network in the biological sciences.** *Biological conservation* 2009, **142**:2282-2292.

32. Simpson R, Page KR, De Roure D: **Zooniverse: observing the world's largest citizen science platform.** In *Proceedings of the 23rd international conference on world wide web: 2014:1049-1054*.

33. Smith A, Lynn S, Lintott C: **An introduction to the zooniverse.** In *Proceedings of the AAAI Conference on Human Computation and Crowdsourcing: 2013:103-103*.

34. Franzoni C, Sauermann H: **Crowd science: The organization of scientific research in open collaborative projects.** *Research policy* 2014, **43**:1-20.

35. Ponti M, Seredko A: **Human-machine-learning integration and task allocation in citizen science.** *Humanities and Social Sciences Communications* 2022, **9**:1-15.
36. Van Horn G, Mac Aodha O, Song Y, Cui Y, Sun C, Shepard A, Adam H, Perona P, Belongie S: **The inaturalist species classification and detection dataset**. In *Proceedings of the IEEE conference on computer vision and pattern recognition*: 2018:8769-8778.

37. Affouard A, Goëau H, Bonnet P, Lombardo J-C, Joly A: **PI@ ntnet app in the era of deep learning**. In *ICLR: International Conference on Learning Representations*: 2017.

38. Phillips TB, Ballard HL, Lewenstein BV, Bonney R: **Engagement in science through citizen science: Moving beyond data collection**. *Science Education* 2019, **103**:665-690.

*This paper attempts to qulaitatively characterise citizen scicen engagement acreoss projects from a participant's perspective. Building on pertinent literature, the paper presents an analysis of interviews from multiple projects exploring "engagement". A dimensions of enaggement framework is presented.

39. Shirk JL, Ballard HL, Wilderman CC, Phillips T, Wiggins A, Jordan R, McCallie E, Minarchek M, Lewenstein BV, Krasny ME: **Public participation in scientific research: a framework for deliberate design**. *Ecology and society* 2012, **17**.

40. Rodela R, Ligtenberg A, Bosma R: **Conceptualizing serious games as a learning-based intervention in the context of natural resources and environmental governance**. *Water* 2019, **11**:245.

41. Greenblat CS: **Group dynamics and game design: Some reflections**. *Simulation & Games* 1980, **11**:35-58.

42. De Caluwé L, Hofstede GJ, Peters V: **Why do games work**. *search of the active substance* 2008:826.

43. Connolly TM, Boyle EA, MacArthur E, Hainey T, Boyle JM: **A systematic literature review of empirical evidence on computer games and serious games**. *Computers & education* 2012, **59**:661-686.

44. Ducrot R: **Gaming across scale in peri-urban water management: contribution from two experiences in Bolivia and Brazil**. *International Journal of Sustainable Development & World Ecology* 2009, **16**:240-252.

45. Duke RD. *Gaming: The Future's Language*. Sage publications, New York; 1974

46. Rumore D, Schenk T, Susskind L: **Role-play simulations for climate change adaptation education and engagement**. *Nature Climate Change* 2016, **6**:745-750.

47. Vervoort J, Mangnus A, McGreevy S, Ota K, Thompson K, Rupprecht C, Tamura N, Moossdorff C, Spiegelberg M, Kobayashi M: **Unlocking the potential of gaming for anticipatory governance**. *Earth System Governance* 2022, **11**:1001-1030.

48. de Vlieger NM, Sainsbury L, Smith SP, Riley N, Miller A, Collins CE, Bucher T: **Feasibility and Acceptability of ‘VitaVillage’: A Serious Game for Nutrition Education**. *Nutrients* 2021, **14**:189.

49. Hammady R, Arnab S: **Serious gaming for behaviour change: A systematic review**. *Information* 2022, **13**:142.

* This paper seeks to identify game design mechanics and features that are reported to commonly influence behaviour change during and/or after the interventions. The authors identify key theories of behaviour change to inform the game design process.

50. Morganti L, Pallavicini F, Cadel E, Candelieri A, Archetti F, Mantovani F: **Gaming for Earth: Serious games and gamification to engage consumers in pro-environmental behaviours for energy efficiency**. *Energy Research & Social Science* 2017, **29**:95-102.

51. Gerber A, Ulrich M, Wäger FX, Roca-Puigròs M, Gonçalves JS, Wäger P: **Games on climate change: Identifying development potentials through advanced classification and game characteristics mapping**. *Sustainability* 2021, **13**:1997.

52. Garcia CA, Savilaakso S, Verburg RW, Stoudmann N, Fernbach P, Sloman SA, Peterson GD, Araújo MB, Bastin J-F, Blaser J: **Strategy games to improve environmental policymaking**. *Nature Sustainability* 2022:1-8.

53. García-Barrios L, Rivera-Núñez T, Cruz-Morales J, Urdapilleta-Carrasco J, Castro-Salcido E, Peña-Azcona I, Martínez-López O, López-Cruz A, Morales M, Espinoza J: **The Flow of Peasant Lives:**
a board game to simulate livelihood strategies and trajectories resulting from complex rural household decisions. *Ecology and Society* 2020, **25**.

54. Speelman E, García-Barrios L, Groot J, Tittonell P: Gaming for smallholder participation in the design of more sustainable agricultural landscapes. *Agricultural Systems* 2014, **126**:62-75.

55. Abrami G, Daré Ws, Ducrot R, Salliou N, Bommel P: Participatory modelling. In *The Routledge Handbook of Research Methods for Social-Ecological Systems*. Edited by: Routledge; 2021:189-204.

56. Voinov A, Bousquet F: Modelling with stakeholders. *Environmental modelling & software* 2010, **25**:1268-1281.

57. Khaled R, Vasalou A: Bridging serious games and participatory design. *International Journal of Child-Computer Interaction* 2014, **2**:93-100.

58. Steen M: Co-design as a process of joint inquiry and imagination. *Design issues* 2013, **29**:16-28.

59. Becu N, Amblard F, Brax N, Gaudou B, Marilleau N: How to involve stakeholders in the modeling process. In *Agent-Based Spatial Simulation with NetLogo*. Edited by: Elsevier; 2015:223-252.

60. Di Mascio D, Dalton R: Using Serious Games to Establish a Dialogue Between Designers and Citizens in Participatory Design: The Architectural Portal of People's Narratives. *Serious Games and Edutainment Applications: Volume II* 2017:433-454.

61. Salliou N, Bruley E, Moreau C, Luthe T, Blanco V, Lavorel S, Grêt-Regamey A: Correction to: Game of Cruxes: co-designing a game for scientists and stakeholders for identifying joint problems. *Sustainability Science* 2021, **16**:1763-1763.

62. Speelman EN, van Noordwijk M, Garcia C: Gaming to better manage complex natural resource landscapes. In *Co-investment in ecosystem services: global lessons from payment and incentive schemes*. Edited by: World Agroforestry Centre; 2018:1-11.

63. Pagel H, Ranke K, Hempel F, Köhler J: The use of the concept „global south “in social science & humanities. *University of California, Berkeley* 2014, **125**.

64. Fan Y, Wang X, Funk T, Rashid I, Herman B, Bompoti N, Mahmud MS, Chrysochoou M, Yang M, Vadas TM: A Critical Review for Real-Time Continuous Soil Monitoring: Advantages, Challenges, and Perspectives. *Environmental Science & Technology* 2022, **56**:13546-13564.

65. Mayer I, Bekebrede G, Harteveld C, Warmelink H, Zhou Q, Van Ruijven T, Lo J, Kortmann R, Wenzler I: The research and evaluation of serious games: Toward a comprehensive methodology. *British journal of educational technology* 2014, **45**:502-527.

66. Speelman EN, Rodela R, van Noordwijk M, Hofstede GJ: Games, sustainable developmental goals, local communities and natural resource management systems: an expert event: SESAM Research Program. Edited by: Wageningen University and Research Centre; 2021.

67. Asingizwe D, Poortvliet PM, Koenraadt CJ, van Vliet AJ, Ingabire CM, Mutesa L, Leeuwis C: Why (not) participate in citizen science? Motivational factors and barriers to participate in a citizen science program for malaria control in Rwanda. *PLoS One* 2020, **15**:e0237396.

* This paper explores the motivational factors and barriers to participate in a citizen science program for malaria control in Rwanda. It assesses the changes in motivational factors over time and compares these factors among age and gender groups.

68. Albagli S, Iwama AY: Citizen science and the right to research: building local knowledge of climate change impacts. *Humanities and Social Sciences Communications* 2022, **9**.

*This paper aims to provide a theoretical and empirical contributions on participatory approaches and methods of citizen science for risk mapping and adaptation to climate change by providing a literature review combined with insights of an action research project in Brazil.

69. Gourmelon F, Chlous-Ducharme F, Kerbiriou C, Rouan M, Bioret F: Role-playing game developed from a modelling process: A relevant participatory tool for sustainable development? A co-construction experiment in an insular biosphere reserve. *Land use policy* 2013, **32**:96-107.

70. van Beek L, Milkoreit M, Prokopy L, Reed JB, Vervoort J, Wardekker A, Weiner R: The effects of serious gaming on risk perceptions of climate tipping points. *Climatic Change* 2022, **170**:31.
71. Barreteau O, Bots P, Daniell K, Etienne M, Perez P, Barnaud C, Bazile D, Becu N, Castella J-C, Daré Ws: **Participatory approaches.** *Simulating social complexity: A handbook* 2013:197-234.

72. De la Torre R, Onggo BS, Corlu CG, Nogal M, Juan AA: **The role of simulation and serious games in teaching concepts on circular economy and sustainable energy.** *Energies* 2021, **14**:1138.

73. Long B, Simson J, Buxó-Lugo A, Watson DG, Mehr SA: **How games can make behavioural science better.** *Nature* 2023, **613**:433-436.

74. Senabre Hidalgo E, Perelló J, Becker F, Bonhoure I, Legris M, Cigarini A: **Participation and co-creation in citizen science.** Chapter 11. In: Vohland K. et al. (Eds). 2021. *The Science of Citizen Science*. Springer. https://doi.org/10.1007/978-3-030-58278-4. pp: 199-218 2021.