# How Networks of Citizen Observatories Can Increase the Quality and Quantity of Citizen-Science-Generated Data Used to Monitor SDG Indicators

Sasha Woods, Maria Daskolia, Alexis Joly, Pierre Bonnet, Karen Soacha, Sonia Liñan, Tim Woods, Jaume Piera, Luigi Ceccaroni

► To cite this version:

Sasha Woods, Maria Daskolia, Alexis Joly, Pierre Bonnet, Karen Soacha, et al.. How Networks of Citizen Observatories Can Increase the Quality and Quantity of Citizen-Science-Generated Data Used to Monitor SDG Indicators. Sustainability, 2022, 14 (7), 10.3390/su14074078. [DOI: 10.36588/su.hal-03658842](https://doi.org/10.36588/su.hal-03658842)

HAL Id: hal-03658842

https://hal.inrae.fr/hal-03658842

Submitted on 4 May 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Article

# How Networks of Citizen Observatories Can Increase the Quality and Quantity of Citizen-Science-Generated Data Used to Monitor SDG Indicators

Sasha Marie Woods 1,*, Maria Daskolia 2 ID, Alexis Joly 3, Pierre Bonnet 4 ID, Karen Soacha 5,6, Sonia Liñan 5 ID, Tim Woods 7 ID, Jaume Piera 5 ID and Luigi Ceccaroni 1 ID

1. Earthwatch Europe, Oxford OX2 7DE, UK; lceccaroni@earthwatch.org.uk

2. Environmental Education Lab, Department of Educational Studies, National and Kapodistrian University of Athens, University Campus, 15784 Athens, Greece; mdaskol@eds.uoa.gr

3. INRIA, Université de Montpellier, 34090 Montpellier, France; alexis.joly@inria.fr

4. Botany and Modeling of Plant Architecture and Vegetation (AMAP), French Agricultural Research Centre for International Development (CIRAD), French National Centre for Scientific Research (CNRS), French National Institute for Agriculture, Food and Environment (INRAE), Research Institute for Development (IRD), University of Montpellier, 34398 Montpellier, France; pierre.bonnet@cirad.fr

5. EMBIMOS Group, Institute of Marine Sciences-Spanish Research Council (ICM-CSIC), 08003 Barcelona, Spain; soacha@icm.csic.es (K.S.); slinan@icm.csic.es (S.L.); jpiera@icm.csic.es (J.P.)

6. Information and Knowledge, Doctoral School, Open University of Catalonia (UOC), 08035 Barcelona, Spain

7. European Citizen Science Association (ECSA), Invalidenstraße 43, 10115 Berlin, Germany; t.woods@drk.de

* Correspondence: swoods@earthwatch.org.uk

<figure><img src="image_2.png" /></figure>

check for
updates

**Citation:** Woods, S.M.; Daskolia, M.; Joly, A.; Bonnet, P.; Soacha, K.; Liñan, S.; Woods, T.; Piera, J.; Ceccaroni, L. How Networks of Citizen Observatories Can Increase the Quality and Quantity of Citizen-Science-Generated Data Used to Monitor SDG Indicators. *Sustainability* **2022**, *14*, 4078. https://doi.org/10.3390/su14074078

Academic Editor: Pedro Guilherme Rocha dos Reis

Received: 28 February 2022

Accepted: 25 March 2022

Published: 30 March 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_3.png" /></figure>

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

**Abstract:** There is a growing acknowledgement that citizen observatories, and other forms of citizen-generated data, have a significant role in tracking progress towards the Sustainable Development Goals. This is evident in the increasing number of Sustainable Development Goals' indicators for which such data are already being used and in the high-level recognition of the potential role that citizen science can play. In this article, we argue that networks of citizen observatories will help realise this potential. Drawing on the Cos4Cloud project as an example, we highlight how such networks can make citizen-generated data more interoperable and accessible (among other qualities), increasing their impact and usefulness. Furthermore, we highlight other, perhaps overlooked, advantages of citizen observatories and citizen-generated data: educating and informing citizen scientists about the Sustainable Development Goals and co-creating solutions to the global challenges they address.

**Keywords:** citizen science; citizen observatory; Sustainable Development Goals; interoperability; co-design; education

## 1. Introduction

Central to the 2030 Agenda for Sustainable Development [1] are the United Nations' 17 Sustainable Development Goals (SDGs) and the associated indicators defined and set as quality criteria for promoting action at the national, regional and international levels. The SDGs are also used to evaluate progress in a range of areas contributing to sustainable development, such as poverty reduction, improvements in health, education and sustainable growth and the preservation of natural environments, peace, justice and international cooperation. Reliable, timely, comprehensive and consistent data are essential to inform processes aimed at promoting and assessing the fulfilment of the 2030 Agenda [2]. There is, however, growing recognition that traditional data sources, such as government surveys, are not sufficient for providing such information [3].

Citizen science provides new means and opportunities for collecting various kinds of data to monitor the progress towards sustainability. Defined as work undertaken by, or within, citizen communities with the support and moderation of scientists, educators
and other community stakeholders, citizen science aims to advance science, foster scientific literacy and research competence, encourage democratic engagement and participation, and empower people to join the debate about complex current problems [4]. A growing body of evidence demonstrates that citizen science is particularly effective at contributing data to promoting and assessing specific SDGs and indicators. Some recent theoretical reviews [5] and empirical studies of active citizen science projects [2,6,7], studying citizen science's contribution towards the achievement of the SDGs, have identified several positive correlations.

A significant number of environmental citizen science initiatives and projects worldwide are “already contributing” or “could contribute” to one-third of the SDG indicators, e.g., by monitoring environmental quality (SDG 6, 13, 14 and 15), or health and wellbeing (SDG 3) [2]. Indeed, citizen science project coordinators show greater confidence in assessing the impacts of their projects by relating them to specific SDGs, or by attuning their methodology to the measurement of specific SDG indicators [8]. For example, the FreshWater Watch [9] project, run by Earthwatch, contributes to the specific SDG indicator 6.3.2 “Proportion of bodies of water with good ambient water quality”, with citizen scientists involved in the project measuring key indicators of their local water bodies’ quality and uploading the data collected to a global platform. Governments are known to use this information to understand river water quality, while it also helps to fill data gaps to meet SDG 6 objectives [10].

Besides engagement in data collection and policy-making, there are further collaboration channels through which citizen science can support the SDGs, including education and multi-stakeholder partnerships [5]. Indeed, education is an important field identified in Schleicher and Schmidt (2020)'s analysis of 130 German citizen science projects as to how they connect to the SDGs [7]. In the study by Moczek et al. [6], “Quality education” (SDG 4) was among the top SDGs reported to be currently supported through the projects surveyed.

Nevertheless, citizen science has yet to reach its full potential as a set of non-traditional scientific procedures and sources of collective data and as a means for societal participation and empowerment to monitor global sustainability challenges [11]; cited as having “the potential to provide credible data to bridge the data gaps [...] and [...] enable the monitoring of SDG environmental indicators” [12]. Moreover, beyond collecting and analysing data through social participation, citizen science is perfectly positioned to boost education, partnership, and action for sustainability, through its collaborative nature and by enriching mainstream school-based activities and approaches. However, most of these goals have not yet been fully realised.

Supporting citizen science to increase the scope, quality and quantity of citizen-generated data available for SDG monitoring and reporting and to boost education, partnership, and collaborative action for sustainability can be achieved in several ways. One way is to demonstrate its impact on society, the environment and the economy, as the MICS (Measuring the Impact of Citizen Science) project [13] does, through an online tool for impact assessment. A second way, which is the focus of this article, is by creating and supporting a network of citizen observatories through the development of services and associated measures, which is the main aim of Cos4Cloud (Co-designed citizen observatories for the EOS-Cloud), a European Horizon 2020 project [14].

Citizen observatories are community-based environmental monitoring and information systems that invite individuals to collect and share observations mainly of an environmental content, often by mobile phones connected to the internet [15,16]. They are characterised as including participation of citizens in environmental monitoring and governance, a bi-directional flow of data and information, “in situ” citizen-generated observations and the use of mobile and web technologies. However, citizen observatories face many challenges, such as the sustainability of citizen participation, the quality of the data generated with respect to the research questions addressed, the management of participation scalability and platform maintenance costs.
The Cos4Cloud project is developing a range of technological services that will enable the networking of citizen observatories and help address many of the challenges they face. The Cos4Cloud vision is to boost citizen science through the support of citizen observatories by improving:

* data quality, by simplifying expert verification, or using AI to improve identification;

* data interoperability, by facilitating integration between data sets;

* data accessibility, by processing data so they can be more easily uploaded to the European Open Science Cloud [17] ensuring open access;

* the sustainability and maintenance of citizen observatories, for example, through low-cost tools that can be used, adapted and replicated.

As Cos4Cloud partners, we argue here that networks of citizen observatories, meaning citizen observatories interacting, sharing and learning from and with each other and their communities of practice around common goals or interests, can aid citizen science in realising its full potential and contributing to SDG monitoring in three main ways:

* by capitalising on the interoperability of data, which will lead to larger, integrated and more useful data sets;

* by building on co-design approaches, involving and combining the citizens' and researchers' experience and expertise throughout the project's life cycle;

* by feeding and supporting the education and learning experience of building on, participating in and contributing to sustainable development and global citizenship (which is the focus of SDG 4.7, as well as a cross-cutting theme across all SDGs).

We do this, firstly, by explaining the methodology that underpins citizen observatories and how standard or new procedures can contribute to monitoring progress towards the SDGs. We discuss how networks of citizen observatories build on and enhance the progress made to date by individual citizen observatories and the limitations of this approach. Secondly, we demonstrate how projects such as Cos4Cloud can improve and support these efforts through interoperability, co-design and education, via three case studies: Pl@ntNet as a service to improve interoperability; co-design as a method to develop services, such as Cos4Bio, with the users (citizen observatories) in mind; and integration of citizen observatory technologies into environmental education. Finally, we discuss how these supported networks of citizen observatories, enhanced by the actions of Cos4Cloud, can be more effective at contributing to SDG monitoring, at a societal and political level.

## 2. Materials and Methods

### 2.1. Citizen Observatories: Opportunities to Support the SDGs

Citizen observatories can contribute to monitoring progress towards many SDGs. As mentioned in Section 1, this focuses on their main function, namely engaging citizens in contributing to the generation and analysis of reliable, timely, comprehensive and consistent data, based on which progress towards SDGs can be measured. This is enabled through the direct involvement of citizen scientists in data collection and analysis in a range of scientific fields. Citizen observatory data involve environmental and nature observations on land use (e.g., LandSense) and biodiversity (e.g., iSpot, Pl@ntnet or Artportalen), up to astronomy observations leading to the classification of galaxies (e.g., Galaxy Zoo). Another method central to how citizen observatories' function is by building on citizens' collective intelligence, including the "information, experience and knowledge embodied within individuals and communities" [18].

Despite worldwide political support for the development of citizen observatories, including by the European Commission, several reservations have been expressed about the reliability of data obtained via citizen-based observations compared to those obtained by professional researchers, and the lack of quality standards needed for informed decision-making and environmental governance. Indeed, there are studies demonstrating limitations in the scalability and reliability of data collected through citizen observatories [19]. Nevertheless, there is also evidence that the citizen-centred approach used by citizen observatories
appears to translate into sustained participation by participants, leading to the generation of better-quality data [20]. By providing data across vast geographical and temporal scales, citizen observatories can accumulate new, timely and comprehensive data.

An illustration of the ability and efficiency of citizen science to collect high-throughput and large-scale, research-grade data is the Global Biodiversity Information Facility (GBIF), a biodiversity network and data infrastructure. More than 87% of the data entries published on the GBIF platform since 2019 consist of citizen science data, compared to an average of 51% of occurrences coming from citizens before 2019. This source has great potential for addressing some of the data gaps on biodiversity and ecosystem health [11].

## 2.2. Limitations of the Citizen Observatory Method

Although some citizen observatories can disseminate their data widely through international platforms such as GBIF, these represent only a small fraction of all the citizen observatories worldwide. Most encounter a range of technical, financial and administrative difficulties that prevent them from contributing their citizen-generated data to such high-level aggregation platforms. These challenges include: (i) a lack of pathways to make their data and information available; (ii) a lack of connections, mutualisation and synchronisation between different citizen observatories, meaning that data sets remain largely fragmented or overlapping; (iii) greater difficulty in identifying gaps in data coverage, because data sets are not connected; and (iv) a lack of expertise or resources to deploy new technologies efficiently, such as artificial intelligence (AI) and big data analytics.

These barriers limit the use of citizen observatory data, for example, when establishing global regulations. As a consequence, the available data for high-level monitoring and decision-making are strongly biased towards particular regions, taxonomic groups and acquisition protocols; the latter being a methodology for structured data collection, designed to enable robust statistical analysis in relation to the research question being addressed. This is even worse for scientific fields other than biodiversity (e.g., air quality, water quality, noise pollution), most of which lack a single platform or infrastructure recognised globally as the central repository for environmental data.

## 2.3. The Cos4Cloud Methodology for Improving Citizen Observatory Networks

More significant interaction among citizen observatories could improve the usefulness of their data, and increase the scale and impact of their contribution to monitoring the SDGs. Cos4Cloud is one of the first initiatives to address the challenges faced by citizen observatories by facilitating citizen observatory networking through interoperability, co-design and education.

Many citizen observatories collect data to feed the monitoring of the SDGs directly or indirectly. Cos4Cloud works with nine such observatories, some of which feed into SDG monitoring directly (such as the previously mentioned FreshWater Watch, which contributes to SDG indicator 6.3.2 "Proportion of bodies of water with good ambient water quality") and some of which feed into SDG monitoring indirectly (such as Odour Collect, a citizen observatory developed to empower citizens to tackle odour pollution and thus contribute to SDG 3.9.1 "Mortality rate attributed to household and ambient air pollution").

The Cos4Cloud project is developing a suite of technological tools and services to support and strengthen networking between these different citizen observatories. A full description of each service is available on the project's website [14]. Once ready (in 2023), these will be uploaded to the European Open Science Cloud (EOSC) as modules, so that any citizen science observatory can select and install the services it needs to improve its functionalities. In this way, Cos4Cloud reflects Europe's need for better connections between the disparate infrastructures developed for different citizen observatories, such as their data storage and data-sharing mechanisms.

Cos4Cloud will act as a minimum viable ecosystem (MVE) for citizen observatories. The MVE, a methodology that aims to co-design and co-develop an ecosystem of actors for citizen observatories proposed by Cos4Cloud, consists of co-designed and prototyped ser-
vices to face citizen observatories' challenges. A key aspect is to ensure that the interactions among the different components have a synergic benefit for all participating stakeholders. As shown in Figure 1, there are three main components of the MVE for COs: CO-portals, experts' portals, and AI-based automatic-validation tools. The interaction and feedback among them (coloured arrows) improve the associated services and data quality.

<figure><img src="image_4.png" /><figcaption>**Figure 1.** The Cos4Cloud MVE. Starting from a raw observation (1), citizen observatory portals generate a collaborative validation from the inputs of other participants and suggestions provided by AI-based automatic-validation tools (2). Community-validated observations (3) are integrated into different experts' portals (4), providing a higher level of validation (5). The expert validations provide feedback and can be grouped into different reference sets (6) that can be used as complementary high-quality training sets (7) to improve the classification capabilities of the AI-based automatic-validation tools (which in turn will provide better recommendations for future collaborative validations in each CO) generating high-quality data in global repositories (8).</figcaption></figure>

Thus, the main contribution of Cos4Cloud to SDG monitoring is via networking: strengthening, enriching and multiplying the work offered by each citizen observatory alone. This networking will be enabled by: (1) improved interoperability; (2) the use of co-design; and (3) education and learning for sustainable development. These three elements are discussed as case studies in the following section, where we explore how individual citizen observatories are being supported by the technologies developed in the Cos4Cloud project.

## 3. Results

### 3.1. *How Cos4Cloud Improves Interoperability: AI-Based Plant Identification through Pl@ntNet*

Pl@ntNet [21] is a citizen observatory that relies on AI technologies to help people identify plants with their smartphones. It is one of the largest biodiversity observatories in the world, with several million contributors in over 200 countries.

The general workflow of Pl@ntNet is shown in Figure 2 and consists of three main components and processes: the Pl@ntNet app, enriching the database and exploiting the database.
<figure><img src="image_5.png" /><figcaption>**Figure 2.** General pipeline of the Pl@ntNet citizen observatory and its three main components and processes [22].</figcaption></figure>

**Pl@ntNet app:** The primary component consists of a plant-recognition application for smartphones that allows anyone to take a picture of a plant and send it to a server for recognition, performed by a convolutional neural network [23]. Pictures should be centred on individual specimens, avoiding strong contrast, as well as the presence of visual noise (such as fingers or any manufactured objects). The user is returned with a list of candidate species associated with a confidence score and an adapted illustrative picture enabling the user to validate a species and share the observation.

**Enriching the database:** This process aims to improve the observations collected by the app users. The annotation of the vast volume of data (over four million observations were shared in 2020) cannot be carried out by expert botanists alone and, therefore, various crowdsourcing mechanisms are used: the main one integrated within the app itself. In all views of the application, the users can enrich an observation, either by proposing a new species name or by voting on the names submitted by other contributors.

**Exploitation of the database:** In this step, the enriched dataset is (i) used to enhance the model by re-training it on a larger and more complete dataset, and (ii) made available for the needs of stakeholders, including both personal (88%) and professional (12%) users.

Pl@ntNet directly contributes to monitoring progress towards several SDG indicators. Broadly, Pl@ntNet contributes to SDG 15 "Life on Land", supporting the identification of new species. When an image of a species which is not yet illustrated in Pl@ntNet is submitted, the AI plant identification model usually provides a "no result" message to the citizen scientist. The citizen scientist can then share their plant observation without any species name. In this instance, more experienced botanists would have to suggest the most probable plant species name visible on the image, which would then need to be validated by the users' network community. This mechanism has thus allowed for the expansion of 800 plant species illustrated in the first version of the Pl@ntNet mobile app (launched in February 2013), to more than 38,000 plant species in the actual version and more specifically, within SDG 2 "Zero Hunger", Pl@ntNet contributes to the indicators 2.3 and 2.4, which are related to improving agricultural productivity and sustainable food production systems, respectively, through its use by professional agrarian bodies.

The primary added value of the Cos4Cloud project over the base Pl@ntNet workflow is to enable its interoperability with (i) other citizen observatories and networks, (ii) third-party applications developed by industrial, academic or associative actors, and (iii) networks of experts and scientists. In this regard, we highlight hereafter two main achievements.
### 3.1.1. Pl@ntNet Identification as a Service

In the context of Cos4Cloud, Pl@ntNet is developing a RESTful JSON-based application programming interface (API), a software interface dedicated to offering a service to other software, allowing the integration of Pl@ntNet's visual identification engine in third-party applications. The web service [24] is co-designed and developed through the collaborative effort of the Pl@ntNet team and the end users of the service, while the API allows interoperability among the systems which use it. In this way, an application for monitoring the biodiversity of a region can, therefore, use this image-based plant identification service, complete the taxonomical/ecological information disseminated to its users thanks to the data provided by GBIF or Encyclopedia of Life (EOL) on the species determined, and enrich the knowledge of the species concerned by sharing these occurrences on the country's national biodiversity data aggregation portal.

As illustrated in Table 1, the service offers a diversity of key features facilitating its integration in a wide variety of contexts. The regularly updated and enriched identification model ensures continuous development and improvement of the Pl@ntNet API. If the service does not exhaustively cover the plant diversity of interest to a user at the time of the integration, the user can use the service knowing that it might be able to cover the species of interest in a future update. The automated rejection of inappropriate content also facilitates interoperability, normalising and standardising the quality of shared data and providing assurance to application developers that their users will not be able to share inappropriate content within their systems or with other platforms.

<table><caption>**Table 1.** Pl@ntNet features facilitating interoperability and integration.</caption><thead><tr><td>Functionality</td><td>Description</td><td>Added Value</td><td>Example</td></tr></thead><tbody><tr><td>Set-valued species classification</td><td>The service returns a list of species corresponding to the query, each associated with a probability output by the deep learning model.</td><td>The probability output allows for better quality control in the downstream applications provided to end users.</td><td>An application may suggest to its end users to re-submit a better-quality photo if the confidence score returned for identification is too low. If the confidence score is very high, it can automatically record the observation for the end user.</td></tr><tr><td>Flora selection (optional)</td><td>This option allows the returned species to be filtered according to a given flora (among 41 possible floras at the time of writing).</td><td>The identification quality is improved, and the service can be more easily customised for specific use cases.</td><td>An application will not display a visually similar species for a given identification request (of the same genus, for example) if the species is not within the selected flora.</td></tr><tr><td>Similarity search</td><td>For each recognised species, the service returns the closest pictures to the query in the Pl@ntNet database.</td><td>This allows better quality control of the identification by the end user. It also provides access to Pl@ntNet images to illustrate the species.</td><td>Application developers do not have to manage visual data to illustrate the identified species and can benefit from the data shared by Pl@ntNet.</td></tr><tr><td>Multi-images and multi-organ observations</td><td>Several pictures of the same individual plant can be provided as input of the service (particularly photos of the different organs, including the flower, the fruit or the leaf).</td><td>The identification quality is improved, and the service can be more easily customised for specific use cases.</td><td>A biodiversity monitoring app can be used whatever the season or the phenological stage of the plant.</td></tr></tbody></table>
<table><caption>**Table 1.** *Cont.*</caption><thead><tr><th>Key Features</th><th>Description</th><th>Added Value</th><th>Example</th></tr></thead><tbody><tr><td>Regularly updated and enriched identification model</td><td>Pl@ntNet's AI model used for the identification service is regularly updated based on the collaboratively revised Pl@ntNet database and research progress.</td><td>The identification quality is regularly improved, and the number of recognised species increases.</td><td>Newly identified species are rapidly recognisable by other users</td></tr><tr><td>Automated rejection of inappropriate content</td><td>Any picture of non-plant entities is automatically tagged as "Reject".</td><td>Inappropriate content can be filtered before it enters the data flow of the downstream applications (database, app screens, etc.).</td><td>This feature facilitates the integration of the service into educational applications, ensuring the identification of only visual plant data.</td></tr><tr><td>API dashboard and documentation</td><td>The website my.plantnet.org provides rich documentation of the service as well as a dashboard allowing the users to follow its usage of the service.</td><td>Developers can easily integrate the service into downstream applications and monitor their usage.</td><td>An application using this service can monitor its consumption of the service in terms of number of queries per day, week, month, etc.</td></tr><tr><th>Quality of Service</th><th>Description</th><th>Added Value</th><th>Example</th></tr><tr><td>Non-regression tests of the identification quality</td><td>The quality of the identification is measured on several benchmarks before each model's update.</td><td>The user has the guarantee that the identification quality is not degrading.</td><td>An application using this service can enrich the amount of information on each new identifiable species, these species being always identifiable in future versions of the recognition model.</td></tr><tr><td>Quality of service monitoring</td><td>The quality of service is monitored continuously based on several criteria (status, response time and scalability) and an alert system allows the developers to react quickly.</td><td>Service downtime is limited.</td><td>An application using this service can use it at any time.</td></tr><tr><td>Secured access with an API key</td><td>The use of the service requires a token provided at account creation (with email validation).</td><td>This highly limits the risk of attacks.</td><td>This limits the risk of denial of service attacks.</td></tr><tr><td>Scalability</td><td>Pl@ntNet's infrastructure and data management software are highly scalable.</td><td>Developers of downstream applications do not have to manage the workload themselves.</td><td>A third-party application can have a peak in usage or a large increase in users without causing the service to slow down.</td></tr></tbody></table>

At the time of publication, 5000 end users have created an account to use the Pl@ntNet identification API, with an average of 500 identification queries per account. As indicated, thanks to the interoperability of the API, the service can be used in a variety of contexts, many of which support the work of citizen observatories.

Bineo Consulting, a partner in the Cos4Cloud project, is one such user. Bineo is a developer of web technologies for biodiversity monitoring, including Natusfera: a citizen observatory adapted from iNaturalist. Within the framework of biodiversity portals, Bineo Consulting uses the Pl@ntNet identification API to facilitate a review of the quality of observations shared on these portals. Users of these portals can, therefore, validate the data produced more efficiently, accelerating the use of these data in different contexts, both regulatory and educational. During 2020–2022, Bineo Consulting has requested 50,000 identifications on this service.

Tela Botanica, a network of 50,000 French-speaking botanists, aims to promote the exchange of information, and produce open data for the benefit of all botanists, from amateurs to professionals. This non-governmental organisation has been developing an information portal on French flora for over 20 years. It offers a personal observation
manager, “le Carnet en Ligne” (the Online Book), which allows its members to produce and share botanical observations. Through the interoperability of this web software with the Pl@ntNet identification API, users can benefit from identification assistance, which enables them to compare their initial identification with the one proposed by Pl@ntNet.

### 3.1.2. Pl@ntNet Research-Grade Data

As part of Cos4Cloud, Pl@ntNet has developed a pipeline for filtering and sharing research-grade Pl@ntNet data. These research-grade observations are shared in two main ways: (i) through the GBIF platform [25], a platform of free and open access biodiversity data, and (ii) through a web service within Pl@ntNet API. The pipeline to filter the data consists of five key steps.

Firstly, AI-based rejection of inappropriate content is carried out, where any picture of non-plant entities is automatically rejected. Secondly, AI-based rejection of house plants and captive plants of the gardens removes species that are not part of the GBIF spectrum of interest, which focuses on the spontaneous biodiversity of natural or semi-natural environments. Thirdly, there is AI-based quality control of species names, where only observations with a confidence score above 90% according to the most recent Pl@ntNet's identification model are shared, except for observations revised by the community, which is the fourth step. Here, community-based quality control of species names ensures that observations revised by the community are shared only if their identification reaches a sufficient degree of confidence. The confidence score is computed according to a weighted majority voting algorithm, where the weight of each member mostly depends on the number of species he has observed in the Pl@ntNet database. In this way, the validation of a plant observation is based on both the evaluation of the image quality and the species identification. The scientific name of a species can only be validated if (i) the name suggested by the citizen scientist is registered in one of the taxonomic reference lists provided by the Pl@ntNet data providers (for example, Kew Gardens); and (ii) scientists with a high confidence score on their user profile validate the initial determination proposal of the citizen scientist. Finally, location-based quality control of species names ensures that observations are only shared when the species name matches the most trusted checklist for the country/region where the observation was recorded.

The monitoring of Pl@ntNet observations shared with GBIF allows experts who analyse these data on the GBIF platform to directly access all the information related to each observation on the Pl@ntNet website. This continuous back and forth, made possible by the interoperability of the two platforms, encourages case-by-case exchanges between data producers and analysts. Thus, thanks to this pipeline and improved interoperability between the Pl@ntNet citizen observatory and GBIF via Cos4Cloud technology, 11 million research-grade Pl@ntNet observations have been shared within GBIF. Through the GBIF literature tracking programme, it is possible to query the published research articles using Pl@ntNet research-grade data (154 articles at the time of publication). Thus, while Pl@ntNet itself does not provide interpretations on environmental pollution, sustainability or climate change based on the types of plants growing in an area, the data collected contribute to such predictions, as evidenced by the literature utilising these data impacting a number of SDGs, including: new approaches for biodiversity conservation and restoration (SDG 15), new approaches for invasive species prediction and risk assessment (SDG 15, 2.3 and 2.4), new approaches for sustainable agriculture and landscape management (SDGs 2.3 and 2.4), new modelling approaches of the impact of global change on vegetation (SDGs 13 and 15).

Pl@ntNet users are encouraged to produce and share observations of interest for biodiversity studies thanks to ongoing collaboration with many local and national NGOs and research organisations with which Pl@ntNet partners. These actors, who promote a better understanding and protection of biodiversity, encourage the networks of users they train to implement good practices for identification, and also to contribute to less known and more interesting species for scientific purposes. This encouragement is formulated both in the communication materials that they distribute and during face-to-face training
sessions. Indeed, face-to-face discussions are integral to the Cos4Cloud methodology for co-design, described in the following case study 3.2. How Cos4Cloud's co-design methodology supports citizen observatories.

Co-design is a collaborative strategy for developing solutions to problems that involve different types of actors [26]. Collaborative work and collective intelligence, together with co-creation, participatory design and design thinking, came together to create a set of tools that support the development of specialised services of Cos4Cloud.

These tools were implemented through a process that begins with the need to understand each observatory's technological and social conditions, that is, its context and the user community, in order for the challenges to be solved. It continued with the definition of the needs and expectations of the community involved, the ideation of possible solutions, the prototyping of the idea with its multiple adjustment cycles and, finally, testing that helped validate the prototype. This approach considers users as "experts" of their own experience, and their needs and concerns are central to the design process. In Cos4Cloud, users are understood from two perspectives: the citizen observatories themselves and the user communities. This does not mean that the perspectives are artificially separated; on the contrary, their combination allows for a greater understanding of the complexity of the technological ecosystem of the observatories.

Co-design, as a methodological process that guided the construction of services, in the context of Cos4Cloud, had two main roles: firstly, to take into account the diversity and level of involvement of the end users; and, secondly, to facilitate the networking among citizen observatory communities. The co-design of the observatories scaled or increased the capacity of the services to respond to the socio-technological challenges that the observatories faced. This co-design approach was based on three main axes: (1) the integration of the communities of each observatory; (2) the connection of challenges; and (3) the co-creation of solutions from the perspective of interoperability.

**Integration of the communities:** The diversity of geographic scales in which citizen science observatories operate, added to their possible thematic approaches or the particularities of the project they support, makes these platforms work in some cases as silos, both at the community and data management levels. Additionally, the governance schemes of the observatories are not regularly visible, and therefore it is not clear how decision-making and possible collaboration between observatories and their respective communities operate.

By integrating the nine observatories, Cos4Cloud took advantage of the power of networking, in addition to co-design, to create a community of co-design. This made it possible to interact with user communities from various regions of Europe and Colombia dedicated to environmental and biodiversity monitoring to gain a more comprehensive understanding of the context of the required technological services and their needs.

During the co-design process, collaborative workshops were held that collected user stories, identified needs, prioritised requirements and mapped the expectations of the observatory communities, promoting potential future collaboration between them.

**Connection of challenges:** Both citizen science and the technological platforms that support the management of their data are the subject of relatively recent studies. The communities that participate in this platform (end users, managers, developers, researchers and community managers) and those dedicated to studying them are in an initial process of understanding the socio-technological challenges they face. The governance of data and infrastructure, the diversity of technologies based on the platform needs, and the multiplicity of user profiles and thematic areas of application, from health to astronomy, are some of the variables to be considered.

To achieve a greater understanding of these challenges from the reality of multiple observatories, the Cos4Cloud co-design was approached by an interdisciplinary team that managed to connect the dimensions of education, technology, communication, engagement and governance.

**Co-creation of solutions:** Each observatory has been developed with its own set of technologies. This technological diversity, added to the lack of categorisation and data
transfer standards, made it even more challenging to design technological services useful for the global community of citizen science observatories. To address this challenge, the technical teams from each of the nine observatories joined in a single working group that, hand in hand with the co-design community, co-created and tested multiple technological solutions.

As a result, the services are co-designed considering interoperability principles and multiple technological scenarios; they integrate the principles of FAIR data, prioritise the openness and reproducibility of services through open licences, and are created with the user in mind. One such service is Cos4Bio [27], whose main mission is to create an ecosystem that experts in biodiversity related to citizen science can use to carry out searches and downloads quickly and in a standardised way, generating data sets from different sources of information, such as citizen observatories. Through co-design workshops, experts expressed a need for biodiversity data to be searchable by location as well as species—feedback that has subsequently been implemented into the service.

## 3.2. *How Cos4Cloud Enables Citizen Observatories to Support Education and Learning*

The inherent difficulty of addressing current environmental and sustainability challenges requires not only a range of interventions but also the combination and integration of many interventions at various levels. Citizen science and environmental education can both address and promote sustainability. In the case of citizen science, this is achieved by: (1) aiding environmental monitoring and research procedures, (2) raising scientific and environmental literacy and (3) promoting global citizenship through the citizens' active involvement in real issues. Environmental awareness and participation in policy-making are equally expected outcomes together with the development of a sense of place. This is in alignment with how environmental education views its role [28], by emphasising the need for (a) pedagogical practices that actively engage learners in individual and collective inquiry processes about current environmental and sustainability issues and (b) learning experiences that lead to the development of “action competence”, the ability to act based on the active and critical construction of knowledge, which leads to emancipation and democratic participation [29,30]. Thus, the concept of environmentally active and responsible citizenship is a central tenet for both citizen science and environmental education and a key tool for bringing forth sustainability.

In the Cos4Cloud project, we not only assert that citizen science and environmental education are compatible and complementary to each other, but also that there is a great potential for integrating them into school-based practice to support the realisation of the SDGs. Responsible and active citizenship and informed participation for attaining sustainability lie at the core of citizen science [31,32]. Likewise, environmental education departs from engaging with science in the traditional way. It opens up new ways to learn [33] by emphasising experience, intrinsic motivation, self-directed engagement and collaboration on a quest to explore and address local sustainability issues and concerns [34]. To this end, it puts forth a pedagogy that involves challenge-based, cross-disciplinary and action-oriented learning and participation in authentic learning experiences, which take place in either formal or informal education contexts [35] with the aim to empower young people to become active and responsible citizens [36].

The Cos4Cloud project builds on the integration of citizen science into formal school educational processes. Such integration opens up various learning opportunities along the lines of all SDGs, particularly SDG 4. Moreover, it carries significant benefits for the students and the school communities. Engaging students in collaborative inquiry processes with a focus on real environmental issues enhances the school community's openness to science and participation in local sustainable development efforts [37]. The educational potential of integrating citizen science into environmental education activities and projects within schools goes well beyond cognitive outcomes to fostering the students' scientific and environmental literacy and action competence. It aims to cultivate new ways of thinking and collaborating as a basis for developing active citizenship for sustainability [38].
vant research [39] shows that engaging students in appropriately designed environmental citizen science projects within school contexts increases their participation in similar future initiatives. Likewise, environmental education provides a rich pedagogical context, which structures and enhances the learning experiences gained from citizen science activities far beyond contributing to data collection procedures alone.

Alongside the Cos4Cloud project's initiatives to develop technological services to support the networking of citizen observatories, there have been sustained efforts to activate networking within formal education to promote and expand citizen science practice for sustainability beyond the traditional adult audiences and engage youths, school students in particular, and as a mode to enrich the quality of education for sustainability. This strategy was designed to take place across three interconnected actions, through: (1) the creation of an educational network of teachers, educational stakeholders and schools supportive of integrating citizen science into school-based environmental education practices; (2) the provision of teacher training programmes and the design of supportive educational materials to assist teachers in their work; and (3) the implementation of school-based educational activities, and the evaluation of their learning potential and educational impact in contributing to achieving SDGs.

In this context, environmental educators were empowered to engage students in projects focusing on current environmental challenges through citizen observatories such as Pl@ntnet and Odour Collect. Similarly, students were able to use citizen science-based technology including the Pl@ntNet-API to not only monitor biodiversity but also support inquiry-based learning about the sustainability challenges associated with and affecting it. By facilitating access to species identification, participants quickly discover that not only are there a large number of species around them, but that they make multiple contributions to the natural and societal environment. Indeed, by training students to use Pl@ntNet, they become autonomous in exploring the biodiversity of environments outside of school activities, allowing them to better understand the challenges surrounding it. Cos4Cloud plans to develop these methods across other citizen science observatories, integrating citizen science technologies to strengthen education for sustainable development across Europe.

Thus, while individual citizen observatories can provide a context for citizens' education and learning on local sustainability issues and participation, directly contributing to SDG 4.7 (Education for sustainable development and global citizenship) and transversally benefitting multiple SDGs by educating citizens on global challenges and the SDGs themselves, by creating a network of citizen observatories and supporting them with interoperable technology through projects such as Cos4Cloud, students are better educated and engaged, improving the quality and quantity of data collected.

To explore the educational potential of the integration of citizen observatory technologies into school environmental education, Cos4Cloud coordinated the implementation and evaluation of five case studies within Greek schools, during the spring term of 2020–2021. Of these case studies, four were carried out in secondary schools and one in a primary school, and four were based on the use of Pl@ntnet, while one focused on Odour Collect.

To support the integration of citizen science into school environmental practice, Cos4Cloud facilitated the creation of educational scenarios by different groups of Greek educational designers, including teachers. Each scenario focused on a different theme and environmental issue, and offered a detailed description of a proposed way of implementing citizen science in schools, using the project's citizen observatory platforms. All scenarios were accompanied by creative and pedagogically informed activities and employed a range of integrated and interdisciplinary approaches to learning, encouraging students to engage in action learning and to use the outcomes of their learning to improve their school and local environments. For example, one scenario, "The magical world of plants—transforming the schoolyard and protecting life with Pl@ntnet", focused on the theme of biodiversity, the issue of climate change as a biodiversity problem, and the designing of a "green fence" to absorb noise and chemical pollution, improving their local environment.
The five case studies considered and evaluated a number of these educational scenarios. For each case study, teachers completed a case study report and evaluation form to describe how they carried out their educational activities and to share their personal estimations of what the students had learned from participating in them. The completed forms were subjected to qualitative content analysis. Although the analysis is still in progress, and will be published in full at a later stage, there are strong indications that the integration of Cos4Cloud's technologies into environmental education had a distinct impact. For instance, teachers described a range of learning outcomes from the students' engagement in the project's citizen observatory platforms and tools, including the development of new skills related to their scientific and digital literacies, or the awakening of students' senses and emotions; the nurturing of a sense of connectedness with the environments the students studied; as well as a new appreciation for nature and the recognition that citizen science opens up a way to participate and contribute to some broader, global and worthwhile efforts.

# 4. Discussion

Citizen observatories can contribute to monitoring progress towards many SDGs. As we have detailed, the main contribution of Cos4Cloud to SDG monitoring is via the enhanced networking of citizen observatories through improved interoperability, co-design and education and learning for sustainable development. These enhancements should enable citizen observatories to have greater positive impacts at both the societal and political level.

## 4.1. *Citizen Observatory Networks for Society*

The strength of citizen observatories is the potential mobilisation of a large number of people to make progress towards a common goal. As networks, citizen observatories provide an enhanced experience: participants feel “part of something bigger” without losing their connection with local issues. One potential barrier to participation, which Cos4Cloud addresses through the co-design approach, is the identification of typical participation profiles, so that potential participants can easily see whether the proposed citizen observatory corresponds to their expectations in terms of objective, level of involvement and expertise in particular. This helps to limit the possible disappointment of participants who would not have the means (technical or in terms of availability) to participate, and maintains active engagement. Furthermore, sustained communication between active members, and during the training of new members, is facilitated if each actor knows the capacities and expectations of the other participants, via a functional network of participants and observatories.

Through the enhancement of citizen observatory networks with Cos4Cloud services, we aim to reduce the cost of data gathering, which is a significant barrier in SDG reporting [3] and in citizen science efforts more generally. With the availability of free and widely accessible web or mobile technologies, citizen observatories encourage the involvement of new participants, who benefit from new tools to progress in their activities.

Indeed, Cos4Cloud can also support efforts to include low-income groups, especially those in the Global South. Thanks to the pooling of resources, whether it is data, methodological support, or capacity to maintain IT infrastructure or software, the costs of adapting citizen observatories to new geographical, thematic or linguistic contexts have been reduced. This favours their deployment in contexts where we would not have been able to set them up due to high initial costs. For example, we can cite the Kenyan Lewa Conservancy, which has set up a citizen observatory for monitoring its flora [40], or the South African environmental education association iScanTree, based on Pl@ntNet services. Thanks to previous work to adapt Pl@ntNet services to the European and North African contexts, the cost of implementing these two new citizen observatories has been greatly reduced. The web and mobile interfaces for participation were already in place, a number of locally occurring species were already described and illustrated and training materials for new
members were already produced. Because all these services and documents are already available, the designers of these new citizen observatories have been able to focus only on the most specific parts of their contexts and objectives, thus reducing the time needed to implement them. The drastic reduction in implementation costs partially explains why actors usually far removed from scientific activities can become involved in setting up and running new citizen observatories. Eco-tourism structures, training centres and organisations in charge of managing territorial areas have thus appropriated the services offered within the framework of Cos4Cloud's citizen observatories, adapting them to their needs in South America, Sub-Saharan or Southern Africa, India or other regions of Southeast Asia.

Finally, the creation and support of networks between teachers and school communities with citizen observatories are among the activities reinforced by Cos4Cloud. The development of networking in formal education and informal learning is a key component of expanding citizen science and increasing its educational impacts; it is also a key condition for promoting sustainability and the SDGs. This is most readily sought through the creation of synergies with school environmental education for sustainability and can be significantly fostered through networking. Empowering citizens, youths in particular, to actively participate in documenting and reclaiming quality environmental conditions in their communities involves a great learning potential and stands as a key indicator of sustainability. The co-design of appropriate educational materials and the provision of in-service teacher training can significantly boost educational networking and all of the above have been defined as important objectives of the Cos4Cloud project.

## 4.2. Citizen Observatory Networks for Policy

SDG 17, “Partnerships for the goals: Strengthen the means of implementation and revitalise the global partnership for sustainable development”, provides a policy framework that supports public engagement in scientific research, inclusive of citizen science both as a research object of policy, and as an instrument of policy [41]. That said, inconsistency in data quality across citizen science projects and bias among scientists for certain data sources [42] render citizen science data underused in environmental policy.

Cos4Cloud services aim to improve data quality and thus build trust in citizen science data, particularly through improved data interoperability. The aforementioned Cos4Bio, which integrates biodiversity observations from multiple citizen observatories in one place, and the similarly functioning Cos4Env, which integrates environmental observations, allow experts in the fields of citizen science, biodiversity and environmental science to view all observations in one place. While individual citizen observatories can validate these observations according to their own specific algorithms, experts using the service will be able to search observations from multiple citizen observatories including ArtPortalent, iSpot, Natusfera and PlantNet and utilise the data for their research. For example, researchers studying the distribution of a specific species at the national level could download all the observations of that species from the service, regardless of the citizen science observatory or country they come from, breaking down the barriers of validity and accessibility that prevent citizen science data from feeding into environmental policy.

## 5. Conclusions

Citizen science is evolving from a field dominated by isolated, small-scale, environmental monitoring projects (i.e., local groups focusing on local issues) to one in which citizen-generated data are more integrated and interoperable and make a significant contribution to monitoring progress towards overcoming current great global challenges. Like scientific progress in general, citizen science also addresses human needs and concerns. In addition, there are established projects that already integrate data at a global scale, such as some of those found on the EU-citizen.science and SciStarter platforms, which enable participation from anywhere in the world. There is a growing focus, among citizen-science projects, on the most significant challenges of our time.
The UN 17 SDGs encompass these global challenges but, with just eight years to go until 2030 (at the time of writing), there is an urgent need to step up efforts towards achieving them. Networks of citizen observatories, such as those supported and facilitated by Cos4Cloud, are essential to unlock the potential of citizen science. Such networks amplify the impact of citizen-generated data and help overcome some of the barriers to their use (e.g., lack of interoperability and accessibility, and undefined quality). As discussed in this article, these networks also create further opportunities in measuring progress towards the SDGs and in achieving them through data interoperability, through co-design of tools and activities and through education and learning.

This opening up of science, and of the processes through which the global challenges we face are tracked and measured, is, we argue, even more important than citizen science's potential to contribute data to measure this progress. It will not be possible to achieve the SDGs, or to address the environmental challenges we face, without far more, and more significant, actions from citizens. To put it another way: we do not just need citizens to assist in monitoring biodiversity loss; we need them to take drastic actions, at the population scale, to reverse this trend. Similarly, the environmental challenges will not be solved simply by citizens helping to measure how hot our world is getting; we need those same citizens, along with all the others, to take the steps needed to reverse current negative trends. By raising awareness of the challenges we face and the remedial measures that individuals can take, citizen science is a pathway to motivating society to act towards a more sustainable future.

**Author Contributions:** Conceptualisation, J.P.; methodology, J.P., A.J., P.B., K.S., S.L. and M.D.; software, J.P., A.J. and P.B.; validation, J.P., A.J. and P.B.; formal analysis, K.S., S.L. and M.D.; investigation, J.P., A.J., P.B., K.S., S.L. and M.D.; writing—original draft preparation, T.W., S.M.W., M.D. and L.C.; writing—review and editing, S.M.W., M.D., A.J., P.B., K.S., S.L., L.C. and J.P.; supervision, J.P.; project administration, K.S. and S.L.; funding acquisition, J.P. All authors have read and agreed to the published version of the manuscript.

**Funding:** The research described in this paper was funded by the European Commission via the Cos4Cloud and MICS projects, which have received funding from the European Union's Horizon 2020 research and innovation programme under grant agreements 863463 and 824711. ICM-CSIC authors acknowledge the institutional support of the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000928-S). The opinions expressed in this work are those of the authors and are not necessarily those of the Cos4Cloud or MICS partners or the European Commission.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest. Funders had no role in the design of the study; in the collection, analysis, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

## References

1. United Nations. Resolution A/RES/70/1. Transforming Our World: The 2030 Agenda for Sustainable Development. In *Seventieth United Nations General Assembly*; United Nations: New York, NY, USA, 2015. Available online: www.un.org/ga/search/view_doc.asp?symbol=A/RES/70/1&Lang=E* (accessed on 27 February 2022).

2. Fraisl, D.; Campbell, J.; See, L.; Wehn, U.; Wardlaw, J.; Gold, M.; Moorthy, I.; Arias, R.; Piera, J.; Oliver, J.L.; et al. Mapping citizen science contributions to the UN sustainable development goals. *Sustain. Sci.* **2020**, *15*, 1735–1751. [CrossRef]

3. Fritz, S.; See, L.; Carlson, T.; Haklay, M.M.; Oliver, J.L.; Fraisl, D.; Mondardini, R.; Brocklehurst, M.; Shanley, L.A.; Schade, S.; et al. Citizen science and the United Nations Sustainable Development Goals. *Nat. Sustain.* **2019**, *2*, 922–930. [CrossRef]

4. Ceccaroni, L.; Bowser, A.; Brenton, P. Civic education and citizen science: Definitions, categories, knowledge representation. In *Analyzing the Role of Citizen Science in Modern Research*; Ceccaroni, L., Piera, J., Eds.; IGI Global: Hershey, PA, USA, 2017; pp. 1–23. ISBN 9781522509622.
5. Shulla, K.; Filho, W.L.; Sommer, J.H.; Salvia, A.L.; Borgemeister, C. Channels of collaboration for citizen science and the sustainable development goals. *J. Clean. Prod.* **2020**, *264*, 121735. [CrossRef]

6. Moczek, N.; Voigt-Heucke, S.; Mortega, K.; Cartas, C.F.; Knobloch, J. A Self-Assessment of European Citizen Science Projects on Their Contribution to the UN Sustainable Development Goals (SDGs). *Sustainability* **2021**, *13*, 1774. [CrossRef]

7. Schleicher, K.; Schmidt, C. Citizen Science in Germany as Research and Sustainability Education: Analysis of the Main Forms and Foci and Its Relation to the Sustainable Development Goals. *Sustainability* **2020**, *12*, 6044. [CrossRef]

8. Sprinks, J.; Woods, S.; Parkinson, S.; Wehn, U.; Joyce, H.; Ceccaroni, L.; Gharesifard, M. Coordinator Perceptions When Assessing the Impact of Citizen Science towards Sustainable Development Goals. *Sustainability* **2021**, *13*, 2377. [CrossRef]

9. FreshWater Watch. Available online: https://freshwaterwatch.thewaterhub.org/ (accessed on 27 February 2022).

10. Hegarty, S.; Hayes, A.; Regan, F.; Bishop, I.; Clinton, R. Using citizen science to understand river water quality while filling data gaps to meet United Nations Sustainable Development Goal 6 objectives. *Sci. Total Environ.* **2021**, *783*, 146953. [CrossRef]

11. UNEP. *Measuring Progress: Environment and the SDGs*; United Nations Environment Programme: Nairobi, Kenya, 2021.

12. UNEP. *Global Environment Outlook—GEO-6: Healthy Planet, Healthy People*; United Nations Environment Programme: Nairobi, Kenya, 2019; ISBN 978-1-108-70766-4.

13. MICS. Available online: https://mics.tools/ (accessed on 27 February 2022).

14. Cos4Cloud. Available online: https://cos4cloud-eosc.eu/ (accessed on 27 February 2022).

15. WeObserve. *Citizen Observatories*; International Institute for Applied Systems Analysis (IIASA): Laxembourg, 2018. Available online: www.weobserve.eu/about/citizen-observatories/ (accessed on 27 February 2022).

16. Ceccaroni, L.; Piera, J.; Wernand, M.R.; Zielinski, O.; Busch, J.A.; Van der Woerd, H.J.; Bardaji, R.; Friedrichs, A.; Novoa, S.; Thijsse, P.; et al. Citclops: A next-generation sensor system for the monitoring of natural waters and a citizens’ observatory for the assessment of ecosystems’ status. *PLoS ONE* **2020**, *15*, e0230084. [CrossRef]

17. ESOC Portal. Available online: https://eosc-portal.eu/ (accessed on 27 February 2022).

18. Liu, H.-Y.; Kobernus, M.; Broday, D.; Bartonova, A. A conceptual approach to a citizens’ observatory—Supporting community-based environmental governance. *Environ. Health* **2014**, *13*, 107. [CrossRef]

19. Rathnayake, C.; Joshi, S.; Cerratto-Pargman, T. Mapping the current landscape of citizen-driven environmental monitoring: A systematic literature review. *Sustain. Sci. Pr. Policy* **2020**, *16*, 326–334. [CrossRef]

20. Ajates, R. Reducing the Risk of Co-Optation in Alternative Food Networks: Multi-Stakeholder Cooperatives, Social Capital, and Third Spaces of Cooperation. *Sustainability* **2021**, *13*, 11219. [CrossRef]

21. Joly, A.; Bonnet, P.; Goëau, H.; Barbe, J.; Selmi, S.; Champ, J.; Dufour-Kowalski, S.; Affouard, A.; Carré, J.; Molino, J.-F.; et al. A look inside the Pl@ntNet experience. *Multimed. Syst.* **2015**, *22*, 751–766. [CrossRef]

22. Lorieul, T. Uncertainty in Predictions of Deep Learning Models for Fine-Grained Classification. Ph.D. Thesis, Université de Montpellier, Montpellier, France, 2020.

23. Affouard, A.; Lombardo, J.-C.; Goeau, H.; Bonnet, P.; Joly, A. Pl@ntNet App in the Era of Deep Learning. In Proceedings of the 5th International Conference on Learning Representations, ICLR 2017, Workshop Track Proceedings, Toulon, France, 24–26 April 2017. Available online: OpenReview.net (accessed on 27 February 2022).

24. Pl@ntNet. Available online: https://my.plantnet.org/ (accessed on 27 February 2022).

25. Affouard, A.; Joly, A.; Lombardo, J.-C.; Champ, J.; Goeau, H.; Bonnet, P. Pl@ntNet Observations; Version 1.2. Pl@ntNet. Occurrence Dataset. 2020. Available online: https://www.gbif.org/dataset/7a3679ef-5582-4aaa-81f0-8c2545cafc81 (accessed on 1 January 2021).

26. Sanders, E.B.-N.; Stappers, P.J. Co-creation and the new landscapes of design. *CoDesign* **2008**, *4*, 5–18. [CrossRef]

27. Cos4Bio. Available online: https://cos4bio.eu/ (accessed on 27 February 2022).

28. Short, P.C. Responsible Environmental Action: Its Role and Status in Environmental Education and Environmental Quality. *J. Environ. Educ.* **2009**, *41*, 7–21. [CrossRef]

29. Jensen, B.B.; Schnack, K. The action competence approach in environmental education. *Environ. Educ. Res.* **2006**, *12*, 471–486. [CrossRef]

30. Breiting, S.; Sorensen, F. Action competence and environmental education. *Camb. J. Educ.* **1999**, *29*, 349–353. [CrossRef]

31. Citizen Science: Innovation in Open Science, Society and Policy. Available online: https://discovery.ucl.ac.uk/id/eprint/10058422/1/Citizen-Science.pdf (accessed on 27 February 2022).

32. Tauginienė, L.; Butkevičienė, E.; Vohland, K.; Heinisch, B.; Daskolia, M.; Suškevičs, M.; Portela, M.; Balázs, B.; Prūse, B. Citizen science in the social sciences and humanities: The power of interdisciplinarity. *Palgrave Commun.* **2020**, *6*, 89. [CrossRef]

33. Mominó, J.M.; Piera, J.; Jurado, E. Citizen Observatories as Advanced Learning Environments. In *Analyzing the Role of Citizen Science in Modern Research*; IGI Global: Hershey, PA, USA, 2017; pp. 192–212.

34. UNESCO. Education for Sustainable Development: A Roadmap. 2020. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000374802 (accessed on 27 February 2022).

35. UNESCO. Framework for the Implementation of Education for Sustainable Development for the Period of 2020–2030. 2009. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000370215 (accessed on 27 February 2022).

36. UNESCO. Education for Sustainable Development Goals: Learning Objectives. *Paris: UNESCO*. 2017. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000247444?posInSet=2&q (accessed on 27 February 2022).
37. Bonney, R.; Shirk, J.L.; Phillips, T.B.; Wiggins, A.; Ballard, H.L.; Miller-Rushing, A.J.; Parrish, J.K. Next Steps for Citizen Science. *Science* **2014**, *343*, 1436–1437. [CrossRef]

38. Jordan, R.C.; Gray, S.T.; Howe, D.V.; Brooks, W.R.; Ehrenfeld, J.G. Knowledge gain and behavioral change in citizen-science programs. *Conserv. Biol.* **2011**, *25*, 1148–1154. [CrossRef]

39. Jordan, R.C.; Ballard, H.L.; Phillips, T. Key issues and new approaches for evaluating citizen-science learning outcomes. *Front. Ecol. Environ.* **2012**, *10*, 307–309. [CrossRef]

40. Bonnet, P.; Joly, A.; Faton, J.; Brown, S.; Kimiti, D.; Deneu, B.; Servajean, M.; Affouard, A.; Lombardo, J.; Mary, L.; et al. How citizen scientists contribute to monitor protected areas thanks to automatic plant identification tools. *Ecol. Solut. Évid.* **2020**, *1*, e12023. [CrossRef]

41. Schade, S.; Pelacho, M.; van Noordwijk, T.G.E.; Vohland, K.; Hecker, S.; Manzoni, M. Citizen Science and Policy. In *The Science of Citizen Science*; Vohland, K., Ed.; Springer: Berlin/Heidelberg, Germany, 2020; pp. 351–371. [CrossRef]

42. Burgess, H.K.; DeBey, L.B.; Froehlich, H.E.; Schmidt, N.; Theobald, E.J.; Ettinger, A.K.; HilleRisLambers, J.; Tewksbury, J.; Parrish, J.K. The science of citizen science: Exploring barriers to use as a primary research tool. *Biol. Conserv.* **2017**, *208*, 113–120. [CrossRef]