# Towards value-creating and sustainable open data ecosystems: A comparative case study and a research agenda

Bastiaan van Loenen, Anneke Zuiderwijk, Glenn Vancau- Wenberghe, Francisco J Lopez-Pellicer, Ingrid Mulder, Charalampos Alexopoulos, Rikke Magnussen, Mubashrah Saddiqa, Melanie Dulong de Rosnay, Joep Crompvoets, et al.

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Bastiaan van Loenen, Anneke Zuiderwijk, Glenn Vancau- Wenberghe, Francisco J Lopez-Pellicer, Ingrid Mulder, et al. Towards value-creating and sustainable open data ecosystems: A comparative case study and a research agenda. JeDEM - eJournal of eDemocracy and Open Government, 2021, 13 (2), pp.1-27. 10.29379/jedem.v13i2.644. halshs-03500727

HAL Id: halshs-03500727

https://shs.hal.science/halshs-03500727

Submitted on 22 Dec 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Towards value-creating and sustainable open data ecosystems: A comparative case study and a research agenda

Bastiaan van Loenen,<fnref n="1" />* Anneke Zuiderwijk,<fnref n="2" /> Glenn Vancauwenberghe,<fnref n="3" /> Francisco J. Lopez-Pellicer,<fnref n="4" /> Ingrid Mulder,<fnref n="5" /> Charalampos Alexopoulos,<fnref n="6" /> Rikke Magnussen,<fnref n="7" /> Mubashrah Saddiqa,<fnref n="8" /> Melanie Dulong de Rosnay,<fnref n="9" /> Joep Crompvoets,<fnref n="10" /> Andrea Polini,¹¹ Barbara Re,¹² Cesar Casiano Flores¹³

<footnote n="1">¹* ORCID Nr: 0000-0001-8847-6334
Delft University of Technology, the Netherlands, b.vanloenen@tudelft.nl</footnote>

<footnote n="2">² ORCID Nr: 0000-0002-3552-7289
Delft University of Technology, the Netherlands, a.m.g.zuiderwijk-vaneijk@tudelft.nl</footnote>

<footnote n="3">³ ORCID Nr: 0000-0002-9915-9606
KU Leuven, Belgium, glenn.vancauwenberghe@kuleuven.be</footnote>

<footnote n="4">⁴ ORCID Nr: 0000-0001-6491-7430
Universidad de Zaragoza, Spain, fjlopez@unizar.es</footnote>

<footnote n="5">⁵ ORCID Nr: 0000-0003-1043-1341
Delft University of Technology, the Netherlands, i.j.mulder@tudelft.nl</footnote>

<footnote n="6">⁶ ORCID Nr: 0000-0002-6610-0675
University of the Aegean, Greece, alexop@aegean.gr</footnote>

<footnote n="7">⁷ ORCID Nr: 0000-0001-8058-9027
Aalborg University, Denmark, rikkem@hum.aau.dk</footnote>

<footnote n="8">⁸ ORCID Nr: 0000-0002-4816-2426
Aalborg University, Denmark, mus@es.aau.dk</footnote>

<footnote n="9">⁹ ORCID Nr: 0000-0002-0297-7603
CNRS Center for Internet and Society, France, melanie.dulong@cnrs.fr</footnote>

<footnote n="10">¹⁰ ORCID Nr: 0000-0003-1077-597X
Public Governance Institute (PGI), KU Leuven, Belgium, joep.crompvoets@kuleuven.be</footnote>
¹¹ ORCID Nr: 0000-0002-2840-7561
University of Camerino, Italy, andrea.polini@unicam.it

¹² ORCID Nr: 0000-0001-5374-2364
University of Camerino, Italy, barbara.re@unicam.it

¹³ ORCID Nr: 0000-0003-4707-6988
Public Governance Institute (PGI), KU Leuven, Belgium, cesar.casiano@kuleuven.be

**Abstract:** Current open data systems lag behind in their promised value creation and sustainability. The objective of the current study is twofold: 1) to investigate whether existing open data systems meet the requirements of open data ecosystems, and 2) to develop a research agenda that discusses the gaps between current open data systems on the one hand and participatory, value-creating, sustainable open data ecosystems on the other hand. The literature reveals that the main characteristics of value-creating, sustainable open data ecosystems are user-drivenness, inclusiveness, circularity, and skill-based. Our comparative case study of five open data systems in various application domains and countries highlighted that none of these systems are real open data ecosystems: they often do not balance open data supply and demand, exclude specific user groups and domains, are linear, and lack skill-training. We elaborate on a research agenda that discusses how research should address the challenge of making open data ecosystems more value-generating and sustainable.

**Keywords:** Open data, ecosystems, value creation, sustainability, research agenda

**Acknowledgement:** The authors acknowledge the financial support from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 955569.

# 1. Introduction

Data is key to the infrastructure and the success of information economies (European Commission, 2018). The European data economy's value is estimated to be €739 billion in 2020, representing over 2% of the EU28 GDP, and highly likely to grow significantly in the coming years (Barbero et al., 2018). When data is openly shared with the public, scholars expect stakeholders to create even more value with it (Vickery, 2011; European Commission 2020; Huyer & van Knippenberg, 2020). In addition to promoting economic and social value creation, open data should improve efficiency and effectiveness of public services, increase transparency, accountability of institutions, and increase citizen participation (Huijboom & van den Broek, 2011; Hossain et al., 2016; Zuiderwijk et al., 2018). Moreover, open data (OD) are essential for achieving the UN Sustainable Development Goals (e.g., Global Partnership for Sustainable Development Data, 2017), and the Berlin declaration on Digital Society and Value-Based Digital Government in the European Union (eu2020.de, 2020). Open data (OD) is data that is available free of charge, openly licensed, and in an open, machine-readable format (European Commission, 2011, 2013a; Geiger & von Lucke, 2012; Gurin, 2014; Open Knowledge Foundation, 2015; Charalabidis et al., 2018). In the past decade, OD initiatives have resulted in greater availability of data. Despite the enormous potential, OD still faces many challenges that
hinder its potential. The increased volumes of open government data (OGD) do not directly or au-
tomatically lead to increased transparency and trust in government Grimmelikhuijsen, 2012;
Gurstein, 2011). One reason is a mismatch between the openly shared datasets and those that the
public demands (Independent Reporting Mechanism, 2015). Other possible explanations include a
lack of cooperation by public agencies (Peled, 2011) or insufficient care when providing datasets
(Janssen, Charalabidis, & Zuiderwijk, 2012). Moreover, there is a mismatch between the benefits
delivered by OGD initiatives and the objectives set (Zuiderwijk et al., 2018).

The idea has gained momentum that to realize the benefits of OD fully, traditional “one-way street” OD systems should be replaced by OD ecosystems. Such ecosystems use an approach to OD that focuses not only on data accessibility but also on the larger environment for OD use—its “ecosystem” (Pollock, 2011). The added value of the ecosystem perspective on OD is its focus on the relationships and interdependencies between the social and technological factors that affect the performance of OD activities (Dawes et al., 2016; Zuiderwijk et al., 2014). So far, previous research shows that the status of research into OD ecosystems is mainly explorative. Comprehensive, empirical research into the research challenges and needed activities under the ecosystem approach and how they relate to impact is scarce.

This study aims to: 1) to investigate whether existing OD systems meet the requirements of OD ecosystems and 2) develop a research agenda that discusses the gaps between current OD systems on the one hand and value-creating, sustainable OD ecosystems on the other hand. This study's contributions are threefold. First, it provides insight into the characteristics of value-creating, sustainable OD ecosystems compared to OD systems in general. Second, this study reveals to what extent five existing European OD systems can be considered ecosystems. Third, it develops a research agenda for the development of value-creating, sustainable OD ecosystems.

This article structure is: First, we explore the concept of OD ecosystems and derive four lenses for understanding OD ecosystems from the literature (Section 2). We then reason the selection of the multiple case study research approach used in this study (Section 3). Thereafter, we examine the extent to which five OD systems adhere to the ecosystem characteristics, analysing them through the four lenses (Section 4). Finally, Section 5 provides a research agenda for developing OD systems into value-creating, sustainable OD ecosystems, followed by our conclusions in Section 6.

## 2. Research background and motivation: open data ecosystems

This section provides the study's background and elaborates on the value creation and sustainability
of the OD ecosystem concept.

### 2.1. The open data ecosystem concept

The European Commission foresees a critical role for OD ecosystems, which facilitate re-use of pub-
lic sector information as well as privately-held data (European Commission, 2018, p. 5) and "the
deployment of a fully-fledged open data infrastructure" (European Commission, 2018, p. 7). In gen-
eral terms, Nardi and O'Day (1999) define an ecosystem as "a system of people, practices, values,
and technologies in a particular local environment". Ecosystems comprise various intercommuni-
cating, relatively closely related, and strongly interdependent components. Nevertheless, ecosystems
differ regarding their specific features.

In 2011, Pollock (2011) coined the concept of the OD ecosystem. Pollock reasoned that traditional OD systems, where data flows from data providers to data users only, should be transformed into ecosystems. In such ecosystems, available data is cycled and recycled among data providers and users. Intermediaries, i.e., actors in-between data providers and users, are essential to OD ecosystems because they form a bridge between the actors and create additional value. OD ecosystems should stimulate collaboration around data value creation and provide reusable components.

Pollock’s work was followed by other approaches (Ubaldi, 2013), often in an OGD context (see, for example, Dawes et al., 2016; ; Ubaldi, 2013; Harrison et al., 2012). Several scholars framed OD ecosystems relatively technically by pointing at how OD supply can be optimized to support that data’s use (Zuiderwijk et al., 2014). Others reason that OD ecosystems consist of “an active network, facilitating interaction and communication amongst everybody interested and/or involved in OD and the re-use of information and data, internal, as well as external to the organisation” (Share-PSI 2.0, 2016, p. 1). OD ecosystems can be realized at diverging levels, such as the data providers and data user level, or between them at the intermediaries level (Jetzek, 2017; Ubaldi, 2013). Furthermore, the scale of OD ecosystems may vary: within institutions, countries, regions, worldwide, as well as within different disciplines and domains (see Zuiderwijk, 2015). Moreover, Kapoor et al. (2015) suggest that an OD ecosystem includes various activities and Mulder et al. (2019) expand the understanding of a data ecosystem to include its actors and the political and organizational infrastructures promoting or participating in those activities. They also provide a general and theoretical framing to qualify the creation of an ecosystem that could activate OD as a public resource, which is a foundation for more inclusive, and profitable use of OD.

Finally, the various stakeholders in OD ecosystems are mutually interdependent. This interde-
pendence creates a joint responsibility: only unitedly, they make the system flourish or tear down
(Harrison et al., 2012). What is critical “to the ecosystems metaphor is the recognition that users,
technology innovators, and government leaders, data managers, and policymakers are mutually in-
terdependent in developing this business efficiently, effectively, and in ways calculated to bring
value to all participants” (Harrison et al., 2012, p. 922). However, the question remains: what makes
an OD ecosystem sustainable, and how can the promised value creation and impacts be evidenced?

## 2.2. Value creating and sustainable open data ecosystems

Van Loenen et al. (2018, p. 5) define an OD ecosystem as “a cyclical, sustainable, demand-driven environment oriented around agents that are mutually interdependent in the creation and delivery of value from open data” (cf. Boley & Chang, 2007). In the current work, we elaborate on the latter definition and take the premise that a value-creating, sustainable OD ecosystem needs to be (1) user-driven, (2) circular, (3) inclusive, and (4) skill-based (see Figure 1).
<figure><img src="image_3.png" /><figcaption>Figure 1: Towards value-creating and sustainable open data ecosystems.</figcaption></figure>

These four lenses have been identified in two iterative steps. First, we observed group discussions concerning the barriers of current OD systems among a group of fifteen OD experts. These experts are part of a project consortium that carries out scientific research on OD in Europe. The OD experts were asked what essential OD barriers for value-creating, sustainable OD ecosystems they had identified in their research and in practice. Second, we conducted a literature review on OD barriers. The literature review included both scientific and grey literature, such as European Commission reports. The literature review included literature recommendations by the OD experts as a starting point. A snowballing approach was used to find additional relevant literature. In addition, we used the keywords from the identified literature to identify additional relevant literature through search engines, including Google Scholar and Scopus.

An iterative approach was used. The two described steps informed each other, i.e., the group discussions provided literature recommendations and the findings from the literature review were discussed during the group discussions. Feedback given during the group discussions was then processed while continuing the literature review. In the following sections, we elaborate on these four OD ecosystems lenses to understand how value creation and sustainability can be realised.

### 2.2.1. User-driven open data systems

Most existing OD systems are neither user-driven nor balance demand-supply matching. Although users' role is assumed to be critical for the success of OD ecosystems, suppliers drive current ecosystems (Gascó-Hernández et al., 2018; Sieber & Johnson, 2015; Van Loenen, 2018; Crusoe et al., 2019). This means that provisioning data from governments to end users is often made through a one-way data portal or platform (Evans & Campos, 2013) without differentiating between various types of users (including cross-border users) or providing data in the way that the data supplier perceives as satisfying (see European Commission, 2020c). Even more, the published data may have no value or are not directly usable (Rhind, 2014). OD providers do not close (enable) the feedback loop – a mechanism for demand-supply matching – and hence are unaware of the actual use and users of their data, and what data and in what form the users need the data to create value (Harrison et al., 2012;
Susha, 2015). That is, they only know that opening datasets increases “usage numbers”, i.e. number of downloads (van Loenen et al, 2018).

However, who uses OD, for which purposes, and for what benefits, and how to satisfy the needs of the wide variety of user types remains unknown in most instances (see Olausson, 2016; Susha et al., 2015). Some efforts have been done creating best practices towards OD publication to enable value creation (Share-PSI 2.0, 2016). Besides, there have been endeavours towards the update and transposition of the PSI directives (European Commission, 2019) to define a legal framework facilitating access and reuse (LAPSI (European Commission, 2010) and LAPSI2.0 (European Commission, 2013) research networks). A balanced demand-supply driven approach should overcome these gaps in the data strategy of the government (conform what is written in European Commission 2020). New OD governance and business models accompanied with technical solutions and tools that imply a more active, participatory, or responsive level of government involvement with OD users are required to overcome the current situation (Open Data Institute & World Wide Web Foundation, 2013; Peled, 2013; Sieber & Johnson, 2015; Welle Donker & Van Loenen, 2016).

### 2.2.2. Circular open data systems

Existing OD systems are often linear. In linear OD ecosystems, OD users typically capture value without adding value to the system themselves. At the moment, only very few stakeholders in the ecosystem profit from OD, and usually, the ones who profit are not the ones who invested most effort, time, and money (Pollock, 2011; Welle Donker & Van Loenen, 2018). As a result of this unfair distribution of value in the OD ecosystem, stakeholders may be reluctant to participate. An OD ecosystem is circular (BSI Group, no year) when data is opened by data providers and then processed so that intermediaries, value-adding resellers, and enrichers can keep processed data in use for as long as possible. Users extract the maximum value from derived data/services. Finally, closing the cycle, stakeholders provide additional value to the ecosystem (Alexopoulos et al., 2014). For example, they can deliver value-added products based on the OD or evaluate data usage processes with data providers (Charalabidis, 2016). The necessary type of ecosystem is circular in nature, building upon a complex network of value generated by different participants that are creating valuable information and products and services. However, note that a circle that is too small can harm circular OD systems because prosumers are then focused on serving data to themselves.

### 2.2.3. Inclusive open data systems

Many current OD systems are exclusive. Research and practice mainly focus on a minimal selection of OD providers and user groups, namely governments as data providers and companies as OD re-users. At the same time, many more stakeholders exist (Van Loenen et al., 2018). Consequently, government data and non-government data (e.g., from businesses, citizens, researchers, and others) are often not integrated and do not cross borders. Moreover, not all stakeholders (e.g., data providers, intermediaries, value-adding resellers, enrichers, facilitators, end-users) from any nature (commercial, government, scientific, citizen) participate in OD ecosystem processes (eu2020.de, 2020; European Commission 2020). As a result, the exploration of OD is currently, for the larger part, left to ‘solution owners’, e.g., civic hackers, developers, small businesses, and entrepreneurs. Although
these people are usually well-equipped to generate technical applications, they often lack an over-
view of the issues they are trying to address. The inclusion of problem owners, i.e., citizens, public
administrators, interest groups with a clear view of critical problems to solve, would instead call for
an open and broader process based on participation and co-creation. The interaction with industry
partners, civil society, and governments is essential to ensure that OD initiatives are inclusive. In-
clusive OD ecosystems address society's needs at large by ensuring individuals and disadvantaged
groups (including elderly, women, disabled) have access to and benefit from today's growing
knowledge and information society. Making OD ecosystems more inclusive becomes possible by (a)
creating the necessary skills to use Information and Communication Technologies (ICT) and (b) en-
abling access through specific easy to use applications, having public interest and the needs of the
whole society in mind.

2.2.4. Skill-based open data systems

Existing OD systems usually depend on best effort. There is a lack of skilled people to use OD. Research indicates that many businesses and governments do not have the knowledge and skills to decide on and implement effective strategies of making the best use of their data (Barbero et al., 2018; Davies et al., 2019). This is already reflected in the digital gap within the EU public administrations and the significant variations in the quality of digital public services and which is heavily influenced by the discrepancies in digital skills and competences (Chinn et al., 2020; European Union, 2020). There are currently not enough people who have the right data skills (European Commission, 2020b; IDC & Open Evidence, 2017). This problem is likely to get even worse in the future (Deloitte, 2017). Despite the relevance of soft skills (Laker and Powell, 2011) when considering technological changes (Snape, 2017), the skills required are a higher education degree in economics, mathematics, physics, or other relevant scientific disciplines, plus familiarity with the industry concerned (Economist Intelligence Unit, 2012). The combination of all these skills is difficult to obtain (Deloitte, 2017).

In other words, the availability of data is not a sufficient condition for OD to become a resource
for innovation and social value. Whoever can gain access to OD should also have the appropriate
skills and competencies to use it, understand what kind of perspectives it opens, and what potential
uses it represents (Morelli et al., 2017). This way a lot of low and medium or social science-only
educated people have been excluded from the equation and the lucrative jobs of data scientists. More
effort is needed to increase the readiness of citizens and businesses. This obstacle has been recog-
nised by the European Commission, which is funding €700M for advanced digital skills through
short-term and long-term training (European Commission, 2019). One example is the ISA² pro-
gramme of the European Commission that aims to support the development of advanced digital
skills in the public sector within the scope of the “Interoperability Academy” action.

3. Research approach

This section presents the research approach of the study, which aims to investigate the characteris-
tics of existing OD systems and assess whether these systems can be considered as OD ecosystems.
Following the explorative nature of our research, we used a qualitative multiple case study research
approach. Case studies are appropriate for investigating contemporary phenomena in their real-
world context (Benbasat et al., 1987), especially when the boundaries between the phenomenon and
the context are not obvious (Yin, 2018). Case studies' holistic perspective enables examining "com-
plex and ubiquitous interactions among organizations, technologies, and people" (Dubé & Paré,
2003, p. 598). A case study approach is relevant for studies in which context can make a difference
(Kaarbo & Beasley, 1999). In addition, case studies are appropriate for asking how or why-questions
(Yin, 2018). They are especially admissible for problems where research and theory are at an early
and developing stage (Benbasat et al., 1987; Roethlisberger, 1977). In other words, theory develop-
ment research seeks for explanatory knowledge and makes use of case studies to demonstrate the
process that originates the outcome (Gregor, 2006; Verschuren et al., 2010).

The case study research approach is appropriate for this study for the following reasons. First, the boundaries of OD systems and their contexts are not evident. Furthermore, many factors, such as stakeholders, values, technologies, and types and data sensitivity, impact OD systems. OD systems typically consist of various interactions, including between organizations, technologies, and people. Finally, this study looks into the ‘how’ and ‘why’ of OD systems’ challenges. This study wants to know the characteristics of a value-creating, sustainable OD ecosystem and the extent to which these characteristics can be identified in existing OD systems. Moreover, we aim to propose a research agenda to further processes toward OD ecosystems.

Describing clearly how the cases have been selected helps to determine the limits for generalizing
the research findings (Eisenhardt, 1989). Generally, two ways to choose cases are distinguished: sta-
tistical sampling or theoretical sampling (Eisenhardt, 1989). As there is no accurate statistical evi-
dence available on the distribution of OD ecosystems, statistical sampling is not possible. Further-
more, OD theory development is still at a relatively early stage (Zuiderwijk, 2015), and therefore this
study aims to contribute to theory building rather than theory testing. Consequently, theoretical
sampling is most appropriate for this study.

Considering that conducting a single case study can lead to a highly context-dependent construc-
tion (Chalhoub-Deville, 1997), a multiple case study design was preferred. This type of design typi-
cally provides more compelling evidence (Yin, 2018) as multiple cases within different contexts ex-
tends the external generalizability of the study's conclusions (Yin, 2018). We defined the following
criteria to select the cases:

1) The cases involve systems in which OD is provided and used.

2) The cases involve OD systems with diverse data types.

3) The cases involve OD systems in which a diversity of OD actors is represented.

4) Information about the cases is available and accessible.

Table 1 provides an overview of the selected cases. Each case study builds on the work and expe-
riences of the authors of this paper. It is complemented with an extensive primary and secondary
literature review, and discussions with key stakeholders on the status of the ecosystems. Our ex-
plorative research approach resulted in an assessment based on expert judgements for five cases.
<table><caption>Table 1: Case study overview</caption><thead><tr><td>Case</td><td>Country</td></tr></thead><tbody><tr><td>1. Open geographic data system</td><td>The Netherlands</td></tr><tr><td>2. Open education data system</td><td>Denmark</td></tr><tr><td>3. Open agricultural data system</td><td>Italy</td></tr><tr><td>4. Open financial data system</td><td>Greece</td></tr><tr><td>5. Open legal data system</td><td>France</td></tr></tbody></table>

# 4. Open data systems assessed

This section explores to which extent five OD systems can be considered ecosystems. We use the characteristics of a value-creating, sustainable ecosystem as a lens of analysis: user-driven, inclusive, circular, and skill-based.

## 4.1. Case 1: Open geographic data system in the Netherlands

Geographic data is any data with a link to the earth and it is one of the high value dataset categories in the Open Data Directive (see European Commission, 2019b).

* User involvement: Some non-government actors have an advisory role through the participation in the public sector (Van Loenen, 2018). Users are informed through news feeds, OD websites, LinkedIn sites for specific datasets/organisations, and actively involved on operational issues. The open geographic data portal (PDOK) and OS Geo (open-source Geo Netherlands community) initiated geoforum.nl providing a forum for open geographic data. The public can find open public geographic data via a central portal (PDOK, www.pdok.nl).

* Circularity: The Dutch geographic data system is linear. For fundamental datasets, government users are by law required to notify data providers on data quality (e.g., errors). In a rare case, private actors provide improved geographic datasets to the government (e.g., ESRI to PDOK) (Welle Donker et al., 2019). This is preventing the company from frequently downloading/ accessing erroneous government data. A government organization in transportation (NDW) has data for data arrangements with private companies.

* Inclusiveness: Almost all public sector geographic data of national agencies is provided as OD. Scientific geographic data is increasingly shared openly, but delivered through separate (university) channels. Private geographic data remains closed. There have been attempts to cooperate between sectors, e.g., National Mapping Agency and OpenStreetMap. Although there are over 14 billion hits per year in the PDOK service (Sanders, 2020, p. 8), little is known about these users and their characteristics (see Van Loenen et al., 2017). There are indications that the users are not only geo experts anymore (see Van Loenen et al., 2017). There is no information available about the use by or involvement of disadvantaged groups.
* Skills. Open geographic data education is available in at least five universities, and several colleges. These are mostly technology-oriented. Interdisciplinary programs on open geographic data are non-existent.<fnref n="1" /> No information is available on other education levels. There is minimal attention for geographic skills development at the strategic domain level.

Although the provision of public sector open geographic data is abundant, the ecosystem is rather provider-driven, somewhat inclusive, mostly linear, and not skill-based.

## 4.2. Case 2: Open data use in schools in Denmark

In recent years, OD in education has attracted attention in educational research and practice in schools (Coughlan, 2020; Saddiqa et al., 2019). The integration of open real-world datasets into school education can potentially support the development of digital and data literacy skills among future generations of learners in public schools (Atenas & Havemann, 2015).

* User involvement: Educational user involvement and OD use is limited in public schools (Drigas & Leliopoulos, 2014; United Nations Education, 2018). Copenhagen city developed large sets of open city data available to its citizens (DESI, 2018). Citizen access to OD and ability to actively contribute to the improvement of digital services, has become a strategically focus in local governments in Copenhagen and other large cities in Denmark (Magnussen & Elming, 2017). Although OD is available to teachers and students' new educational models as well as data analysis tools need to be developed for educating students in OD user involvement (Saddiqa et al., 2019).

* Circularity: In the larger context of studies urban development citizen science projects in larger cities in Denmark focus is on a circular approach where students apply data in developing citizen-generated models for the development of deprived city spaces that local governments can implement in the development and renewal of (Magnussen et al., 2019).

* Inclusiveness: The Danish studies of OD education and educational citizen science in urban planning have primarily been focused at studying citizen development in deprived areas in the cities of Aarhus and Copenhagen. The studies show that students in schools are highly motivated to be part of the development of their deprived neighbourhood (Magnussen & Elming, 2017). Students and teachers at schools however overall lack data analysis competences and tools and structures for bridging the gap between OD in local governance and educational practice.

* Skills: OD studies in Danish schools show both great potentials and challenges in including open datasets in school contexts (Saddiqa et al., 2019). Educational processes, including using sensors for various types of data collection, were shown to improve students' understanding of how data is generated and how citizen collected data can be compared and applied to the available open datasets of the city.

Although the government provides access to the public to OD through the national portal with up-to-date data, and focus is given to explore how OD from local governments can be implemented as

<footnote n="1">¹ See for an overview of higher education in geographic data: https://sdispider.eu/wp/sdi-education-map/</footnote>
a learning resource in schools, OD systems particularly in education are exclusive, mostly linear,
and require skills, knowledge and tools to implement OD into the educational system.

4.3. Case 3: Open agricultural data system in Italy

In Italy, all the data related to the agricultural domain and managed by the public sector are openly
available. The most relevant geographic datasets are collected and made available through the
dati.gov central portal (https://www.dati.gov.it/), using standard formats and divided according
to specific categories of interest.

*   **User involvement:** In the Italian agricultural data system, users are not completely in the driver's seat for deciding, possibly in collaboration with the public sector, the data that can be delivered as OD. Datasets can be uploaded via web by the various local and national administrations on a single portal, which also include censuses related data. The DatiOpen Portal stores data from the 2011 census and provides specific information about the user that uploaded the data set. The portal includes mechanisms to comment on the available dataset, but this feature does not seem to be used much. In the last years the relevance of such portal diminished and few uploads have been performed recently. Overall, the portal currently stores 3,483 datasets. The dati.gov.it portal is probably the current reference for OD in Italy, but it does not seem to include any mechanisms for user involvement. The portal currently stores 350 datasets related to the agricultural/fishing sectors. A community of researchers and the government are also developing new ways to convey available data to multiple users, even if not directly connected to public portals. An attempt has been made to standardize the use of OD across the entire population. Currently only 1% of the national agricultural area is managed using these innovative technologies (Consiglio Nazionale delle Ricerche, 2018).

* Circularity: The agricultural OD system in Italy is principally linear, and based on the possibility to retrieve data from the dati.gov.it portal. There is no official information about the data provision from government authorities to the central government.

* Inclusiveness: In Italy the most relevant geographic datasets are collected and made available through the dati.gov portal, using standard formats and divided according to specific categories of interest. The current management of OD related to agricultural aspects does not seem to include action to favour inclusiveness. No information is available about the use or involvement of disadvantaged groups and minorities. Different datasets are currently not totally integrated, but there is a movement towards this objective and specific API are made available. The Italian OpenData agriculture project tries to collect all the datasets to allow the consultation and reuse in an open and digital format of a large amount of information.

* Skills: There are available courses related to OD in public administration and universities and open communities like SpaghettiOpenData (Spaghetti Open Data, 2019), but they are not specifically focusing on the domain of agricultural OD. Some advanced training courses based on the management of OD in different sectors have been provided, focusing also on the technologies available in different contexts. The main barrier toward the improvement of the current status seems to relate to the considerable knowledge of technologies used in the specific domain that are not always in the competence set of the users.
To conclude, the provision of open agricultural data from the public sector is somehow abundant, but strongly provider-driven. There is a slight interest from smaller companies and individuals in sharing their data openly, and available solutions do not seem to have a focus on improving inclusiveness. Overall the ecosystem can be principally considered exclusive and linear. The lack of skills to access and manage open data also derives from insufficient knowledge on the tools and potential that an OD ecosystem approach can provide to private companies and citizens.

### 4.4. Case 4: Open financial data system in Greece

Financial data consists of all the government transactions from all governmental levels: national, regional and local.

*   **User involvement:** The Greek transparency program offers capabilities to search, access, and retrieve financial data for public spending information. In this way, citizens, but most importantly data journalists now play a key role as public interest watchdogs. In general, the users can only access information in .pdf format that makes it difficult to run any advanced queries on the data or even combine them with other datasets. Thus, users are employing outside services to produce informative queries and visualisations. The developers could be considered as another user group of the system, building their applications and services based on diavgeia.gov.gr API (Alexopoulos et al., 2018). The central users of the system are public officials that upload and search for specific acts. All user groups could make suggestions through an online request form.

*   **Circularity:** The Greek financial data system is linear in principle. For fundamental datasets (contracts and payments), government users are by law required to notify data providers on data quality (e.g., errors). On the other hand, no data are produced in terms of the connected financial decisions/acts when they are re-uploaded or updated. This is not preventing the journalists or citizens from frequently downloading/accessing erroneous government data. In general, public sector organisations are the providers and the users of the system.

*   **Inclusiveness:** Financial data of public organisations is increasingly shared openly in a central manner but delivered through separate public organisation officials channels using different styles of decision or act documents. Private financial data remains closed or dispersed in companies' websites. This way, the companies fulfil their legal obligation of offering their financial data but not in a structured and processable central manner. No information or special features are available about use by or involvement of disadvantaged groups. The strong producer-driven focus is the main barrier for inclusiveness.

*   **Skills:** To the best of our knowledge, there are no OD education programmes in journalists' departments that offer guidance in the OD domain. There are some specific Information Systems or Computer Science departments that offer training on OD through seminars and lectures. Data journalists in Greece have developed specific strategies to produce and assemble their own data, for instance, collaborations and participation in networks. Greece is a member of the European Data Journalism Network (EDJNet) (European Data Journalism Network, 2017). However, OD education is not well developed in Greece. There is one Master
Programme and one school for data journalism. The field of data journalism is still mainly determined by the availability and accessibility of public datasets.

The provision of public sector open financial data is one of the success stories in the OGD domain. The system in Greece provides some information but further development is needed, since the eco-system could be considered mostly linear, provider-driven, and effort based.

### 4.5. Case 5: Open legal data system in France

Legal data consists of the text of laws (including preliminary parliamentary work, questions, amendments, debates minutes, use of budget, attendance, and votes by Members of Parliament, lists of declared lobbyists, meetings with lobbyists) and case law, the judicial decisions produced by courts at all levels (national, appeal, regional, municipal, administrative agencies such as National Commission for Data Protection). Case law is a specific OD: for privacy and security purposes, part of the data must be hidden by design during the production stage to avoid the persons' re-identification. Legal data also encompasses public databases, such as official journal, associations and companies' registration and accounting data, and income tax calculators.

*   **User involvement:** Until now, the market has been driven by private publishers acting as intermediary brokers structuring the way open legal data could be accessed and reused. Recently, the field is being dominated by technological innovation from the legal tech industry that improves access, search, and interfaces. Non-profit and civil society interests have been represented by citizen transparency advocates also developing applications for the public interest. Public and private professionals are gathered in the Open Law* association (Jean, 2017).

*   **Circularity:** The field developed entirely linearly as a top-down system from government to users. The original Legifrance legal data public database provides access to a limited amount of case law through a hard-to-search interface. The most significant portion of the case law was for decades being sold under exclusivity by the government to private publishers, who were adding economic value and reselling better searchable packages to legal professionals, law firms, and universities. A 2014 decree enacted royalty-free legal OD (for a chronology and legal sources on open legal data, see Barthe, 2020; Maurel, 2017).

*   **Inclusiveness:** Open legal data policy is first targeted at helping legal professionals to focus on high-value tasks rather than sole information search, fostering the development of a national legal tech industry. More inclusive aims of improving transparency and democracy for all citizens through direct data reuse and data-based civic advocacy is a secondary objective of national policies.

*   **Skills:** There are legal tech modules and professional programmes in law schools (Village de la Justice, 2021), but no specific interdisciplinary training dedicated to open legal data in computer science, engineering, business, and public management.

To conclude, French open legal data is dominated by legal professionals, not very inclusive and very linear. Without further legal provisions, legal professional users do not have to further distribute their transformed data under OD conditions which would benefit civil society as a whole. Legal tech applications (APIs, search engines, algorithms) development is dependent of design choices solely
made by the producers, judges and governmental offices making data available after a complex production process (Denis & Goëta, 2017).

## 4.6. Summary of the assessed open data systems

Table 2 summarizes the five assessed OD systems. It depicts that none of the five examined OD systems fully correspond to the ideal situation for value creating and sustainable OD ecosystems. None of the researched OD systems meet any of the four requirements regarding being user-driven, circular, inclusive, and skill-based. Several OD systems are making small progress regarding training to improve OD skills, and one of the systems can be considered both provider-driven and user-driven.

<table><caption>Table 2: Summary of the five assessed OD systems</caption><thead><tr><th></th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th></tr></thead><tbody><tr><td>Case #</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Country</td><td>NL</td><td>DK</td><td>IT</td><td>GR</td><td>FR</td></tr><tr><td>Domain</td><td>Geography</td><td>Education</td><td>Agriculture</td><td>Finance</td><td>Law</td></tr><tr><td>Provider/ User driven</td><td>Provider</td><td>Provider</td><td>Provider/ user</td><td>Provider</td><td>Provider</td></tr><tr><td>Exclusive/ Inclusive</td><td>Exclusive</td><td>Exclusive</td><td>Exclusive</td><td>Exclusive</td><td>Exclusive</td></tr><tr><td>Linear/ Circular</td><td>Linear</td><td>Mostly linear</td><td>Linear</td><td>Linear</td><td>Linear</td></tr><tr><td>Skills: best-effort/ skill based</td><td>Best effort-> technology skills- based</td><td>Best effort</td><td>Best effort</td><td>Best effort</td><td>Best effort</td></tr><tr><td>Ecosystem (Yes/ No)</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td></tr></tbody></table>

## 5. Research agenda

This section describes the research agenda we developed based on our qualitative inquiry. It is the result of several discussion sessions we had in our academic research teams which were taken to the discussions among the authors of this paper. The result of these is presented here.
The research agenda should provide guidance to develop from provider-driven, linear, exclusive, and best effort systems towards user-driven, circular, inclusive, and skill-based ecosystems. The research agenda items are categorized per challenge: user-involvement, circularity, inclusiveness, and skills.

## 5.1. Addressing challenge 1: From a supplier-driven to a user-driven open data ecosystem

This part of the research agenda focuses on determining optimal strategies to better match the demand and provision of OD between users of different categories and domains. Towards this direction, the setup of specific research actions is as follows:

* Identify and classify the needs related to the OD provision of a wide variety of representative user types. Special attention has to be paid to the needs of disadvantaged groups to access and use OD and benefit from today's growing knowledge and information society. This research activity should be the basis for developing technological and governance avenues to meet user needs (conform what is written in eu2020.de, 2020; European Commission, 2020).

* Investigate the technological requirements for providing Findable, Accessible, Interoperable, and Reusable (FAIR) data for different user categories. This research activity includes identifying gaps between the needs of user groups and the current features of OD platforms. It also concerns defining and evaluating new approaches for designing user-driven user interfaces for finding data that fulfils different findability and accessibility requirements from different domains. Besides, research should examine the potential of 'data integrators', enabling the technical interoperability of OD stemming from different domains. Furthermore, scientific research should determine the minimum and optimal set of metadata descriptions to be adopted to allow semantic interoperability of OD across domains and disciplines.

* Identify ways to involve producers and users in the OD ecosystem sustainably. The governance and legal perspectives should be explored, developing new governance models directed at maintaining and distributing value in the entire ecosystem instead of only focusing on the private and professional side's financial advantages.

* Evaluate the governance models' feasibility for achieving the added value for their respective stakeholder group: non-specialist data users, journalists, students, researchers, NGOs, government, companies, and data intermediaries. A new governance model strategy framework is needed to engage different user groups in the OD ecosystem sustainably.

## 5.2. Addressing challenge 2: From a linear to a circular open data ecosystem

The second part of the research agenda aims at analysing the processes of value creation and value capturing in OD ecosystems by investigating the mutual relationships and interaction between different OD and the factors determining the creation and capturing of value. This research element should lay the foundation for closing the OD life cycle by improving the understanding of OGD users' contributions and identifying avenues of organizing this both in a technical and open governance-supportive manner. Fulfilling this gap, the setup of specific research actions is as follows:
* Investigate the motivations and needs of non-government actors that use OGD to become active contributors to the OD ecosystem by sharing the results of their usage (i.e., non-government data (NGD)) in the OD ecosystem. The motivations should be identified and analysed for various (non-government) data holders: citizens, journalists, students, researchers, NGOs, businesses, and intermediaries.

* Examine how non-government users of OGD can be stimulated to share their data openly using technical steering mechanisms. This activity should explore the impact of technological choices and strategies in influencing and determining NGD holders' behaviour. It should also design user interfaces that enable NGD holders to readily provide their OD to the ecosystem.

## 5.3. Addressing challenge 3: From an exclusive to an inclusive open data ecosystem

The third research agenda component focuses on defining the technological requirements and governance mechanisms to stimulate non-government actors to participate in the OD ecosystem and share their data as OD. An inclusive OD ecosystem incorporates both OGD and open NGD. Besides, it incentivizes all stakeholders from all backgrounds to contribute and share. Towards this direction, the setup of specific research actions is as follows:

* Investigate non-government actors' motivations to become active contributors to the OD ecosystem by sharing their (non-government) data in the OD ecosystem. The motivations should be identified and analysed for various (non-government) data holders: citizens, journalists, students, researchers, NGOs, businesses, and intermediaries.

* Study what stimulates non-government data holders to share OD using technical steering mechanisms. It should explore the impact of technological choices and strategies in influencing and determining non-government data holders' behaviour. It should design user interfaces and APIs enabling NGD providers to readily provide their OD to the ecosystem.

* Research how non-government data holders can be incentivized to share OD using steering mechanisms. This task should explore different steering mechanisms to influence and control non-government data holders' behaviour in the OD ecosystem. Research should investigate the use, impact, and limitation of network-based steering, based on cooperation and solidarity. It should explore the opportunities and limitations of market-type steering, through competition and financial incentives, to stimulate non-government data holders to share their data. Non-government perspectives should be provided by citizens, journalists, students, researchers, NGOs, companies, and intermediaries.

## 5.4. Addressing challenge 4: From best-effort-based to skill-based open data ecosystems

For skill-based training, we recommend the following research activities:

* Study what training program can make sustainable connections between research groups from different disciplinary fields. Existing OD training capacity is typically structured along traditional disciplinary, sectoral, and geographical lines. Therefore, it is too fragmented to train researchers in the diverse set of skills they need to become successful OD researchers
or practitioners. A training program should make sustainable connections between research groups from different disciplinary fields. Furthermore, it should pool expertise and resources across nations and disciplines, thereby restructuring European training capacity to address novel challenges in OD research and practice, which should be culminating in a new European curriculum on OD ecosystems.

* Examine how researchers can be provided with knowledge and skills to apply holistic, inter-disciplinary, and intersectoral thinking and practice to design user-driven, circular, and in-clusive OD ecosystems.

* Investigate how to equip researchers with the tools, models, structures, skills, and competencies to convert their knowledge, ideas, and research findings into new products and services for the EU market.

## 6. Conclusions

This study's objective was twofold: 1) to investigate whether existing OD systems meet the requirements of OD ecosystems, and 2) develop a research agenda that discusses the gaps between current OD systems on the one hand and value-creating, sustainable OD ecosystems on the other hand. To attain the first research objective, we combined a literature review with a multiple case study approach. The literature review showed that developing a value-creating, sustainable OD ecosystem requires it to match demand and supply, be circular and inclusive, and skill-based. The combination of these four characteristics is essential to unlocking the enormous potential of OD. Like in natural ecosystems, the lack of any identified elements will impact other aspects of the ecosystem. Hence, it is of utmost importance to integrate the findings and develop a holistic perspective on creating a sustainable OD ecosystem. An OD system can be referred to as an OD ecosystem if it is user-driven, circular, inclusive, and skill-based.

We used these four lenses to assess the extent to which five existing OD systems in Europe show these characteristics. Our assessment highlighted that currently none of the examined OD systems can be characterised as an OD ecosystem. In fact, none of these OD systems fulfilled the four requirements fully. Although some of the studied OD systems are considered rather advanced systems in current OD evaluation research, from an OD ecosystem perspective they are only in the very beginning stages of development.

We developed a four pathway research agenda to stimulate the development of the current provider-driven, linear, exclusive, and best effort systems towards user-driven, circular, inclusive and skill-based systems. Each pathway addresses one of the four OD ecosystem characteristics. The first pathway, from provider-driven to user-driven strategies focuses on determining optimal strategies to better match the demand and provision of OD between users of different categories and domains. The second pathway, from linearity to circularity, aims at analysing the processes of value creation and value capturing in OD ecosystems by investigating the mutual relationships and interaction between different OD and the factors determining the creation and capturing of value. The third pathway, from exclusiveness to inclusiveness, focuses on defining the technological requirements and governance mechanisms to stimulate non-government actors to participate in the OD ecosystem
and share their data as OD. Finally, the fourth pathway, from best effort to skill based OD ecosystems, should provide the foundation for OD knowledge sharing and training resulting in holistic, interdisciplinary, and intersectoral thinking and practice to design user-driven, circular, and inclusive OD ecosystems. The development of the research agenda could significantly strengthen the achievement of the Berlin Declaration goals and the 2019-2024 priorities on Green Deal and 'A Europe fit for the Digital Age' and Data Strategy of the European Commission. Moreover, it could also contribute to achieve the UN Sustainable Development Goals in 2030.

We applied an explorative research approach resulting in an assessment based on expert judgments for a limited number of cases. Although these are clear limitations of the study, our approach did highlight several weaknesses of current OD systems that should not be ignored. By addressing the challenges toward user-driven, circular, inclusive and skill based OD ecosystems, the OD systems of today will continue to deliver significant value to our societies in a sustainable manner and can be considered rightfully OD ecosystems.

## References

Alexopoulos, C., Loukis, E., Mouzakitis, S., Petychakis, M., & Charalabidis, Y. (2018). Analysing the characteristics of open government data sources in Greece. Journal of the Knowledge Economy, 9(3), 721-753.

Alexopoulos, C., Zuiderwijk, A., Charapabidis, Y., Loukis, E., & Janssen, M. (2014). Designing a second generation of open data platforms: Integrating open data and social media. In International Conference on Electronic Government (pp. 230-241). Springer, Berlin, Heidelberg.

Atenas, J., & Havemann, L. (2015). Open data as open educational resources: case studies of emerging practice. Retrieved from

Barbero, M., Bartz, K., Linz, F., Mauritz, S., Wauters, P., Chrzanowski, P., Osimo, D. (2018). Study to support the review of Directive 2003/98/EC on the re-use of public sector information. Retrieved from https://ec.europa.eu/digital-single-market/en/news/impact-assessment-support-study-revision-public-sector-information-directive

Barthe, E. (2020). Une chronologie. Open data et données juridiques publiques en France : les pouvoirs publics sont lancés. La preuve par 10, Precisement.org. https://www.precisement.org/blog/Open-data-et-donnees-juridiques-publiques-en-France-les-pouvoirs-publics-sont.html

Benbasat, I., Goldstein, D. K., & Mead, M. (1987). The case research strategy in studies of Information Systems. MIS Quarterly, 11(3), 369-386. doi:10.2307/248684

Boley, H., & Chang, E. (2007). Digital ecosystems: Principles and semantics. Paper presented at the 2007 Inaugural IEEE-IES Digital EcoSystems and Technologies Conference.
BSI Group. (no year). Executive Briefing: BS 8001 – a Guide: The world’s first standard for implementing the principles of the circular economy in organizations. Retrieved from https://www.bsigroup.com/Sustainability/BS8001_Executive_Briefing.pdf

Chalhoub-Deville, M. (1997). Theoretical models, assessment frameworks and test construction. Language Testing, 14(1), 3-22.

Charalabidis, Y., Alexopoulos, C., Diamantopoulou, V., & Androutsopoulou, A. (2016). An open data and open services repository for supporting citizen-driven application development for governance. In 2016 49th Hawaii International Conference on System Sciences (HICSS) (pp. 2596-2604). IEEE.

Charalabidis Y., Zuiderwijk A., Alexopoulos C., Janssen M., Lampoltshammer T., Ferro E. (2018). The Multiple Life Cycles of Open Data Creation and Use. Chapter 2 in: The World of Open Data. Public Administration and Information Technology, vol 28. Springer, Cham.

Chinn, D., Hieronimus, S., Kirchherr, J., & Klier, J. (2020). The future is now: Closing the skills gap in Europe's public sector. Retrieved from https://www.mckinsey.com/industries/public-and-social-sector/our-insights/the-future-is-now-closing-the-skills-gap-in-europes-public-sector#

Consiglio Nazionale delle Ricerche. (2018). Ricerca e open data. L'agricoltura di precisione per tutti. Retrieved from https://www.cnr.it/en/news/8142/ricerca-e-open-data-agricoltura-di-precisione-per-tutti

Coughlan, T. (2020). The use of open data as a material for learning. Educational Technology Research and Development, 68(1), 383-411.

Crusoe, J., Simonofski, A., Clarinval, A., & Gebka, E. (2019). The impact of impediments on open government data use: insights from users. IEEE 13th International Conference on Research Challenges in Information Science (RCIS) (pp. 1-12).

DataOpen Portal. (no year). DatiOpen.it. Retrieved from http://www.datiopen.it/it/caricamentoDati

Davies, T., Walker, S. B., Rubinstein, M., & Perini, F. (2019). The state of open data: Histories and horizons: African Minds.

Dawes, S. S., Vidiasova, L., & Parkhimovich, O. (2016). Planning and designing open government data programs: An ecosystem approach. Government Information Quarterly, 33(1), 15-27.

Deloitte. (2017). Study on emerging issues of data ownership, interoperability, (re-)usability and access to data, and liability. A study for the EC. Retrieved from https://ec.europa.eu/digital-single-market/en/news/study-emerging-issues-data-ownership-interoperability-re-usability-and-access-data-and

Denis, J., & Goëta, S. (2017). Rawification and the careful generation of open government data. Social studies of science, 47(5), 604-629.
DESI. (2018). Open Data Digital Economy and Society Index. Retrieved from https://ec.europa.eu/digital-single-market/en/digital-economy-and-society-index-desi

Drigas, A. S., & Leliopoulos, P. (2014). The use of big data in education. International Journal of Computer Science Issues (IJCSI), 11(5), 58.

Dubé, L., & Paré, G. (2003). Rigor in information systems positivist case research: current practices, trends, and recommendations. MIS Quarterly, 27(4), 597-635.

Economist Intelligence Unit. (2012). The deciding factor: Big data & decision making. Retrieved from https://www.capgemini.com/wp-content/uploads/2017/07/The_Deciding_Factor_Big_Data_Decision_Making.pdf

Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532-550.

eu2020.de. (2020). Berlin declaration on Digital Society and Value-Based Digital Government in the European Union at the ministerial meeting during the German Presidency of the Council of the European Union on 8 December 2020. Retrieved from

European Commission. (2010). Legal aspects of public sector information (LAPSI). Retrieved from https://ec.europa.eu/digital-single-market/en/news/legal-aspects-public-sector-information-lapsi-thematic-network-outputs

European Commission. (2011). Digital agenda: Commission's open data strategy, questions & answers. Retrieved from http://europa.eu/rapid/pressReleasesAction.do?reference=MEMO/11/891&format=HTML&aged=1&language=EN&guiLanguage=en

European Commission. (2013a). Directive 2013/37/EU of the European Parliament and of the Council of 26 June 2013 amending Directive 2003/98/EC on the Re-use of Public Sector Information Retrieved from http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:175:0001:0008:EN:PDF

European Commission. (2013b). Legal aspects of public sector information 2.0 (LAPSI 2.0). Retrieved from https://ec.europa.eu/digital-single-market/en/news/legal-aspects-public-sector-information-lapsi-thematic-network-outputs

European Commission. (2018, April 25, 2018). Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions: "Towards a common European data space". Retrieved from https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52018DC0232&from=EN

European Commission. (2019a). Digital Europe Programme. Digital Single Market. Retrieved from https://ec.europa.eu/digital-single-market/en/europe-investing-digital-digital-europe-programme
European Commission. (2019b). Directive (EU) 2019/1024 of the European Parliament and of the Council of 20 June 2019 on open data and the re-use of public sector information (recast). Retrieved from https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32019L1024

European Commission. (2020a). Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. A European Strategy for Data. Retrieved from https://ec.europa.eu/info/sites/info/files/communication-european-strategy-data-19feb2020_en.pdf

European Commission. (2020b). Digital Economy and Society Index (DESI) 2020 - Human capital. Retrieved from https://ec.europa.eu/digital-single-market/en/human-capital-and-digital-skills

European Commission. (2020c). Economic Impact of Open Data. Retrieved from https://data.europa.eu/sites/default/files/the-economic-impact-of-open-data.pdf

European Commission. (2020d). The single digital gateway. Retrieved from https://ec.europa.eu/growth/single-market/single-digital-gateway_en

European Data Journalism Network. (2017). About. Retrieved from https://www.europeandata-journalism.eu/eng/About

European Members of the International Council for Science. (2018). Open data in science in Europe. Retrieved from http://www.euro-isc.org/activities/opendata/documents/European_ICSU_Members_Open_Data_Statement_2018.pdf

European Union. (2020). Digital Public Administration factsheet 2020. Retrieved from https://joinup.ec.europa.eu/sites/default/files/inline-files/DigitalFact-sheets_EU_vFINAL.pdf

Evans, A. M., & Campos, A. (2013). Open Government Initiatives: Challenges of Citizen Participation. Journal of Policy Analysis and Management, 32(1), 172-185. doi:10.1002/pam.21651

Gascó-Hernández, M., Martin, E. G., Reggi, L., Pyo, S., & Luna-Reyes, L. F. (2018). Promoting the use of open government data: Cases of training and engagement. Government Information Quarterly, 35(2), 233-242.

Geiger, C. P., & von Lucke, J. (2012). Open government and (linked) (open) (government) (data). Journal of e-Democracy and Open Government, 4(2), 265-278. doi:https://doi.org/10.29379/jedem.v4i2.143

Global Partnership for Sustainable Development Data. (2017). Data Roadmaps for Sustainable Development. Assessment and Lessons Learned. Retrieved from https://www.data4sdgs.org/sites/default/files/services_files/RoadmapsAssessmentReport_Dec2017_FINAL.pdf

Gregor, S. (2006). The nature of theory in Information Systems. MIS Quarterly, 30(3), 611-642.
Grimmelikhuijsen, S. G. (2012). Transparency and trust: an experimental study of online disclosure and trust in Government. Utrecht: Utrecht University.

Gurin, J. (2014). Open data now. The secret to hot startups, Smart investing, savvy marketing, and fast innovation. New York: Mc Graw Hill Education.

Gurstein, M. (2011). Open data: empowering the empowered or effective data use for everyone? First Monday, 16(2), n.p. Retrieved from http://firstmonday.org/ojs/index.php/fm/article/view/3316/2764#author

Harrison, T. M., Pardo, T. A., & Cook, M. (2012). Creating open government ecosystems: a research and development agenda. Future Internet, 4, 900-928. doi:10.3390/fi4040900

Hossain, M.A., Dwivedi, Y. K., & Rana, N. P. (2016). State-of-the-art in open data research: Insights from existing literature and a research agenda. Journal of organizational computing and electronic commerce, 26(1-2), 14-40.

Huijboom, N., & van den Broek, T. (2011). Open data: an international comparison of strategies. European Journal of ePractice, 12(1), 4-16. Retrieved from http://www.epractice.eu/files/European%20Journal%20epractice%20Volume%2012_1.pdf

Huyer, E., & van Knippenberg, L. (2020). The Economic Impact of Open Data: Opportunities for Value Creation in Europe: European Commission.

IDC, & Open Evidence. (2017). European Data Market. SMART 2013/0063. Retrieved from https://www.key4biz.it/wp-content/uploads/2018/04/SMART20130063_Final-Report_030417_2.pdf

Independent Reporting Mechanism. (2015). Aligning supply and demand for better governance. Retrieved from http://www opengovpartnership.org/sites/default/files/IRMReport-OpenData.pdf

Janssen, M., Charalabidis, Y., & Zuiderwijk, A. (2012). Benefits, adoption barriers and myths of open data and open government. Information Systems Management, 29(4), 258-268. doi:10.1080/10580530.2012.716740

Jean, B. (2017). Open Law*, le Droit ouvert -- Genèse du développement d'un mouvement appliquant les méthodes d'innovation ouverte au monde du droit. La semaine juridique - édition générale, LexisNexis, 9, 82-86.

Jetzek, T. (2017). Innovation in the open data ecosystem: Exploring the role of real options thinking and multi-sided platforms for sustainable value generation through open data. In Analytics, Innovation, and Excellence-Driven Enterprise Sustainability (pp. 137-168): Springer.

Kaarbo, J., & Beasley, R. K. (1999). A practical guide to the comparative case study method in political psychology. Political psychology, 20(2), 369-391.
Kapoor, S., Mojsilovic, A., Strattner, J. N., & Varshney, K. R. (2015). From open data ecosystems to systems of innovation: A journey to realize the promise of open data. Paper presented at the Bloomberg Data for Good Exchange Conference, New York City.

Laker, D. R., & Powell, J. L. (2011). The differences between hard and soft skills and their relative impact on training transfer. Human resource development quarterly, 22(1), 111-122.

Magnussen, R., & Elming, A. L. (2017). Student re-design of deprived neighborhoods in Mine-craft: Game-assisted community-driven urban development. Paper presented at the 12th Interna-tional Conference on Computer Supported Collaborative Learning, Philadelphia, USA.

Magnussen, R., Hamann, V. D., & Stensgaard, A. G. (2019). Educating for co-production of com-munity-driven knowledge. Electronic Journal of E-Learning, 17(3), 222-233.

Maurel, L. (2017). Quelles Perspectives pour l'Open Access en Sciences Juridiques Apres la loi Republique Numerique. Journal of Open Access to Law, 5(1).

Morelli, N., Mulder, I., Pedersen, J. S., Jaskiewicz, T., De Götzen, A., & Arguillar, M. (2017) Open Data as a New Commons. Empowering Citizens to Make Meaningful Use of a New Re-source. In: Kompatsiaris I. et al. (eds) Internet Science. INSCI 2017. Lecture Notes in Computer Science, vol 10673. Springer, Cham. https://doi.org/10.1007/978-3-319-70284-1_17

Mulder, I., Jaskiewicz, T., & Morelli, N. (2019). On digital citizenship and data as a new commons: Can we design a new movement? Cuaderno 73, 108, 97-109.

Nardi, B. A., & O'Day, V. L. (1999). Information Ecologies: Using Technology with Heart. Cam-bridge, MA, USA: MIT Press.

Olausson, K. (2016). A step towards aligning supply and demand?-User involvement in supply of open data among ten Dutch public sector bodies.

Open Data Institute, & World Wide Web Foundationatation. (2013). Open Data Barometer 2013 - Global Report. Retrieved from http://www.opendataresearch.org/dl/odb2013/Open-Data-Barometer-2013-Global-Report.pdf

Open Knowledge Foundation. (2015). Open Definition version 2.0. Retrieved from http://opendefinition.org/od/

Peled, A. (2011). When transparency and collaboration collide: The USA open data program. Journal of the American society for information science and technology, 62 (11), 2085-2094.

Peled, A. (2013). Re-designing open data 2.0. Paper presented at the Conference for E-Democracy and Open Government.

Pollock, R. (2011). Welfare gains from opening up Public Sector Information in the UK. University of Cambridge: Cambridge, UK, 4.
Rhind, D. (2014). What is the value of open data? Paper presented at the Proceedings of an APPSI Seminar on What is the Value of Open Data, Rhind, D., ed. 2014. "What is the Value of Open Data?" What is the Value of Open Data? APPSI Seminar on 28January 2014, London, April 2014. Accessed December 4, 2015.

Roethlisberger, F. J. (1977). The elusive phenomena: an autobiographical account of my work in the field of organizational behavior at the Harvard Business School. Boston, Massachusetts: Harvard Business School.

Saddiqa, M., Larsen, B., Magnussen, R., Rasmussen, L. L., & Pedersen, J. M. (2019). Open data visualization in Danish schools: A case study. Paper presented at the 27th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, Plzen, Tjekkiet.

Saddiqa, M., Rasmussen, L., Magnussen, R., Larsen, B., & Pedersen, J. (2019). Bringing open data into Danish schools and its potential impact on school pupils. Paper presented at the 15th International Symposium on Open Collaboration (OpenSym '19), Skövde, Sweden.

Sanders, F. (2020). Jaarverslag PDOK over 2019 (Annual report PDOK concerning 2019). Retrieved from https://www.pdok.nl/documents/1901824/0/PDOK+jaarverslag+2019+v1.0.pdf

Share-PSI 2.0. (2016). Deliverable 7.2 Stable Version of the Share-PSI 2.0 Best Practices. Share-PSI 2.0 Standards for Open Data and Public Sector Information. Retrieved from https://www.w3.org/2013/share-psi/bp/eode/

Sieber, R., & Johnson, P. (2015). Civic open data at a crossroads: Dominant models and current challenges. Government Information Quarterly, 32(3), 308-315. doi:10.1016/j.giq.2015.05.003

Snape, P. (2017). Enduring Learning: Integrating C21st Soft Skills through Technology Education. Design and Technology Education, 22(3), n3.

Spaghetti Open Data. (2019). Spaghetti Open Data. Italiane e italiani che fanno cose con i dati. Retrieved from http://spaghettiopendata.org/

SPIDER. (2019). The map of SDI education. Retrieved from https://sdispider.eu/wp/sdi-education-map/

Susha, I. (2015). Participation in open government. Örebro university,

Susha, I., Grönlund, Å., & Janssen, M. (2015). Organizational measures to stimulate user engagement with open data. Transforming Government: People, Process and Policy, 9(2), 181-206.

Ubaldi, B. (2013). Open Government Data: Towards Empirical Analysis of Open Government Data Initiatives. Retrieved from http://dx.doi.org/10.1787/5k46bj4f03s7-en

United Nations Education. (2018). Open data in schools. Retrieved from https://www.europeandataportal.eu/en/highlights/education-open-data-schools
van Loenen, B. (2018). The Development of Open Data in the Netherlands. In B. van Loenen, G. Vancauwenberghe, & J. Crompvoets (Eds.), Open Data Exposed (Vol. 30, pp. 215-237). The Hague: TMC Asser Press.

van Loenen, B. (2018). Towards a User-Oriented Open Data Strategy. In B. van Loenen, G. Vancauwenberghe, & J. Crompvoets (Eds.), Open Data Exposed (pp. 33-53): Springer.

Van Loenen, B., Ubacht, J., Labots, W., & Zuiderwijk, A. (2017). Log file analytics for gaining insight into actual use of open data. Paper presented at the 17th European Conference on Digital Government, Lisbon, Portugal.

van Loenen, B., Vancauwenberghe, G., Crompvoets, J., & Dalla Corte, L. (2018). Open data exposed. no place: T.M.C. Asser Press.

Verschuren, P., Doorewaard, H., & Mellion, M. (2010). Designing a research project (Vol. 2): Eleven International Publishing The Hague.

Vickery, G. (2011). Review of recent studies on PSI re-use and related market developments. Retrieved from Paris: http://ec.europa.eu/information_society/policy/psi/docs/pdfs/report/psi_final_version_formatted.docx

Village de la Justice. (2021) Comment devient-on "juriste de demain". https://www.village-justice.com/articles/comment-devient-juriste-demain,29637.html

Welle Donker, F., Braggaar, R., & Loenen, B. v. (2019). Hergebruikers van open data in beeld (Reusers of open data in the picture). Research commenced by the Dutch Ministry of Internal Affairs and Kingdom Relations. Retrieved from

Welle Donker, F., & Van Loenen, B. (2016). Sustainable business models for public sector open data providers. JeDEM-eJournal of eDemocracy and Open Government, 8(1), 28-61.

Welle Donker, F., & van Loenen, B. (2018). Societal costs and benefits of high-value open government data: a case study in the Netherlands. Paper presented at the AGILE 2018: 21st AGILE Conference on Geographic Information Science.

Wright, K., Zamith, R., & Bebawi, S. (2019). Data Journalism beyond Majority World Countries: Challenges and Opportunities. Digital Journalism, 7(9), 1295-1302.

Yin, R. K. (2018). Case study research and applications: Design and methods. Thousand Oaks, CA, USA: Sage publications.

Zuiderwijk, A. (2015). Open data infrastructures: The design of an infrastructure to enhance the coordination of open data use. Delft: Delft University of Technology.

Zuiderwijk, A., Janssen, M., & Davis, C. (2014). Innovation with open data: essential elements of open data ecosystems Information Polity, 19(1-2), 17-33. doi:10.3233/IP-140329
Zuiderwijk, A., Shinde, R., & Janssen, M. (2018). Investigating the attainment of open government data objectives – Is there a mismatch between objectives and results? International Review of Administrative Sciences, 86(1). doi:10.1177/0020852317739115

## About the Authors

### Bastiaan van Loenen

Dr. Bastiaan van Loenen is an associate professor at the Faculty of Architecture and the Built Environment at Delft University of Technology in the Netherlands. His current research interests include governance and legal mechanisms stimulating re-use of geographic information, and (open) data driven cities.

### Anneke Zuiderwijk

Dr. Anneke Zuiderwijk is an assistant professor at the Faculty of Technology, Policy and Management at Delft University of Technology in the Netherlands. Her research focuses on open data, and more specifically, on theory development concerning infrastructural and institutional arrangements that incentivize open data sharing and use behavior by governments, researchers, companies and citizens.

### Glenn Vancauwenberghe

Dr. Glenn Vancauwenberghe is research manager and senior researcher at the Spatial Applications Division Leuven (SADL) of the KU Leuven. His main areas of expertise are the governance of spatial data infrastructures (SDI), SDI performance assessment and capacity building on geographic information, SDI and open - geospatial - data.

### Francisco J. Lopez-Pellicer

Francisco J. Lopez-Pellicer works as associate professor at the Universidad de Zaragoza (Spain). Doctor in Computer Engineering and Master in Business Administration, his research work focuses on the improvement of technologies related to the discovery and availability of geospatial data in Open and Spatial Data Infrastructures.

### Ingrid Mulder

Ingrid Mulder is an associate professor of design techniques, Faculty of Industrial Design Engineering, Delft University of Technology. Her ongoing teaching and research activities combine strategic design with diffuse design to explore the interplay between top-down policy and bottom-up participatory innovation. Recent projects include Rotterdam Open Data and Open4Citizens.

### Charalampos Alexopoulos

Charalampos (Harris) Alexopoulos is an adjunct lecturer of open, big and linked data management at the Department of Information and Communications Systems Engineering of the University of the Aegean. He is also a researcher in the Information Systems Laboratory of the same department, publishing on open data, decision support, smart cities and e-government.

### Rikke Magnussen

Rikke Magnussen is an associate professor in citizen science and digital learning design in STEM education at Department of Communication and Psychology at Aalborg University in Copenhagen. She focuses on citizen science and community-driven research and how different types of communities can collaborate in solving scientific, technical or health related challenges in their local area.
## Mubashrah Saddiqa

Mubashrah Saddiqa is a PhD student at Department of Electronic Systems, Aalborg University, Denmark. Her main research interest is how existing Open Data could facilitate the educational process in the Danish public schools and how technology could be integrated in schools in a way that benefits students' digital and learning skills.

## Mélanie Dulong de Rosnay

Mélanie Dulong de Rosnay, PhD in law, is associate research professor at French National Centre for Scientific Research (CNRS) and director of the Center for Internet and Society of CNRS, a research unit and a national network. Her research focuses on digital commons, regulation by technology, IT law and policy.

## Joep Crompvoets

Joep Crompvoets is professor and research manager at KU Leuven Public Governance Institute (Belgium) holding the chair on 'information management in the public sector', and secretary-general of EuroSDR - an European spatial data research network. He has been involved in numerous (inter)national projects related to e-governance, digital transformation, public sector innovation, and (spatial) data infrastructures.

## Andrea Polini

Andrea Polini is associate professor at University of Camerino. His main research interests are in the areas of Quality Assurance for complex software systems, Business Process Management and e-Government. He has been Scientific Leader in the EU project Learn Pad (EU FP7-ICT-2013-11 GA:619583) and Unit coordinator and WP leader in the CHOReOS project (EU FP7-ICT-2009-5, GA:257178).

## Barbara Re

Barbara Re is associate professor of Computer Science at the University of Camerino. Her research interests refer to the area of Business Process Management from modeling to analysis. Particular attention is paid to push the use of formal methods as methodological and automatic tools for the development of high-quality process-aware information systems.

## Cesar Casiano Flores

Cesar Casiano is a postdoctoral researcher at the Public Governance Institute in KU Leuven. He has a Ph.D. in Innovation and Governance for Sustainable Development. He is member of the National Researchers System in Mexico (SNI-CONACYT) and qualified university lecturer in The Netherlands (UTQ/BKO). He has also participated in different European projects such as the H2020 its4land.