# Participatory Data Gathering for Public Sector Reuse: Lessons Learned from Traditional Initiatives

Nathalie Stembert, Peter Conradie, Ingrid Mulder, Sunil Choenni

► To cite this version:

Nathalie Stembert, Peter Conradie, Ingrid Mulder, Sunil Choenni. Participatory Data Gathering for Public Sector Reuse: Lessons Learned from Traditional Initiatives. 12th International Conference on Electronic Government (EGOV), Sep 2013, Koblenz, Germany. pp.87-98, 10.1007/978-3-642-40358-3_8. hal-01490924

HAL Id: hal-01490924

https://inria.hal.science/hal-01490924

Submitted on 16 Mar 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Participatory Data Gathering for Public Sector Reuse: Lessons Learned from Traditional Initiatives

Nathalie Stembert<fnref n="1" />, Peter Conradie<fnref n="1" />, Ingrid Mulder<fnref n="1" />,<fnref n="2" />, and Sunil Choenni<fnref n="1" />,<fnref n="3" />

**Abstract.** Local governments are increasingly looking for new ways to involve citizens in policy and decision-making, for example by combining public sector data sources with data gathered by citizens. Several examples exist of data gathering where personal mobile devices act as data collectors. While these efforts illustrate the technical capability of data sourcing, they neglect the value of local knowledge where people use their senses to capture and interpret data. Traditional data gathering initiatives, however, exploit this local knowledge to inform policy makers, e.g., neighborhood policing. To understand data gathering processes of these traditional data gathering initiatives, three cases are examined. We analyze these cases, focusing on the various elements they contain, concluding how digital data gathering can be informed by these traditional variants, concerning what the benefits of using digital means can be for data gathering and how traditional initiatives ensure data re-use by the public sector.

**Keywords.** Data Gathering, Participatory Citizenship, Local knowledge, Open Data

## 1 Introduction

Local governments aim for new forms of policy and decision-making processes, with an emphasis on greater citizen involvement and participatory government, where active partnerships and collaboration between citizens, the private sector and the municipality are stimulated [1]. Internet has shown to be a promising platform for eParticipation [2] and local governments are increasingly using digital tools to inform and communicate with citizens [3]. This is also manifested in the many ‘Open’ movements, e.g., in Open Data Initiatives, where government data is released for reuse [4].

In this paper we focus on how traditional data gathering initiatives can inform digital means of data gathering, with the data being re-used by the public sector to contribute to policy and decision-making, and how data gathering can benefit from digital tools. Digital means that enable people to passively gather data are emerging, among

<footnote n="1">¹ Creating 010, Rotterdam University of Applied Sciences</footnote>

<footnote>n.stembert@hr.nl, p.d.conradie@hr.nl</footnote>

<footnote n="2">² ID-StudioLab, Delft University of Technology</footnote>

<footnote>I.J.Mulder@tudelft.nl</footnote>

<footnote n="3">³ Research and Documentation Centre, Ministry of Justice, The Netherlands</footnote>

<footnote>r.choenni@minvenj.nl</footnote>
others to map noise pollution [5] indicate quality of roads [6] or congestion [7]. These
examples highlight how mobile devices can be used as data gathering tools. Involving
people as carriers of such sensors can be seen as successors to traditional forms of
data gathering. However, in contrast with digital data gathering, traditional initiatives
use human senses and intelligence to observe and interpret local events, such as crime
prevention initiatives, where people walk inspection rounds to map neighborhood
safety [8]. Digital means might help overcome disadvantages of traditional data gath-
ering like data credibility, non-comparability of data, data in-completeness and logis-
tical issues [9]. Yet, traditional data gathering approaches, still offer certain ad-
vantages such as making better use of qualitative knowledge imbedded in communi-
ties [10][11]. In this paper we explore what digital data gathering processes can learn
from traditional data gathering initiatives, to inform local governments on the organi-
zation of digital data gathering initiatives and empower people to gather data in col-
laboration with local authorities to contribute to policy and decision-making. Our
research question for this study is: *How can digital data gathering processes benefit
from traditional data gathering initiatives?*

The remainder of the current work is structured as follows: Section 2 describes other data gathering projects exploiting the potential of emerging technologies and discusses the value people can contribute to data gathering initiatives, distinguishing initiatives involving people as mobile sensor carriers as well as those involving people as sensors. Section 3 introduces our approach and provides an overview of the cases studied. In Section 4 we present our findings from the multiple case studies and describe important elements in traditional data gathering initiatives. Section 5 discusses how these elements can inform digital data gathering, the challenges associated with it, the benefits of using digital means, and ensuring data re-use by the public sector. In Section 6 we elaborate on directions for future activities.

## 2 Related work

Benefits associated with involving citizens through data gathering are widely
acknowledged in decision making, planning, and policy development [9][11][12].
These benefits include education of citizens [9][13], cost effectiveness [14], or having
access to information that non-residents might not be aware of [15]. Also, when given
training, citizens can provide high quality data using less expensive methods [16].

The increased availability of mobile devices and emerging technologies has en-
couraged projects where data is collected digitally. These include Pothole Patrol [6],
where sensor data submitted by smartphones is used to assess road quality, the Co-
penhagen Wheel [7], where sensors attached to city bicycles submit pollution, road
conditions and congestion data, or data mining uploaded photos to map tourist move-
ment [17]. Gathering data in a digital way improves the validation of results and in-
creases access, in addition to offering better ways of exploring and communicating
findings about the data [9]. This type of data carrying refers to ‘citizens as mobile
sensor carriers’, where submission and gathering of data is digital, and citizens do not
actively decide what to submit. Although these examples illustrate the technical pos-
sibility of submitting or analyzing mined data, they do not necessarily make use of local contextual knowledge found in communities. Firsthand experience, sometimes only available to local residents, can be important to experts in planning or developing policy [18][19][15].

Digital data gathering stands in contrast with more traditional efforts of data gathering, for example logging water quality [16], hunters providing wildlife samples [15], monitoring pollution with bees [14], or the three cases introduced in this paper. Within this category, data gathering has traditionally been analog and people apply contextual knowledge while gathering data. In this 'citizens as sensors' category, citizens actively contribute to the data collection, by gathering data through their senses, and applying contextual knowledge when finding facts.

Due to the benefits of technology, as mentioned earlier, examples of 'citizens as sensors' enabled by digital means are appearing. One such example is FixMyStreet [20], where citizens can log problems in the public space, such as broken lanterns or pavements. Here, citizens, empowered by digital tools sense data and apply contextual knowledge and judgment on what is being logged and submitted.

In this 'citizens as sensors' category, the information is mostly qualitative, as a result of life experience and is instrument independent [11]. Given this, data gathering, validation, and testing by local residents differ largely from the methods and techniques of professional practitioners. Despite evidence that local knowledge can offer valuable insights, the differences in methods and techniques can cause professionals to view the public as having either a deficit of technical understanding or as solely complementing the work of experts [21], while data credibility, logistical issues, non-comparability and incompleteness of data [9] are cited as issues, posing a challenge for 'citizens as sensors' initiatives.

Co-production could be an approach to overcome disagreement about credibility, validation and testing methods and techniques [22], since all stakeholders are accepted as potential contributors and hard distinctions between expert and novice are rejected. Joint fact-finding, in turn, similarly assists in increasing data credibility, while also contributing to more cohesive relationships among stakeholders and a better understanding of differing views [23].

With the advantages of 'citizens as sensors' approaches, combined with using digital means to gather, store and analyze data, we propose examining existing, successful traditional data gathering initiatives. We introduce an analysis of the processes these initiatives currently use, to inform digital initiatives and better understand re-use by the public sector of the data gathered by these traditional initiatives.

## 3 Approach

In order to understand the human involvement in analog data gathering initiatives, a multiple case study approach was used to study traditional data gathering initiatives within real-life contexts [24]. This case study setup allows us to analyze within and across settings, to understand similarities and differences between cases [25].
## 3.1 Case introduction

The three cases are set in Rotterdam, the second largest city in The Netherlands. They were selected based on a predefined set of criteria: (a) it is an initiative where citizens use their senses to gather data, (b) ownership of the initiative lies with citizens, and (c) the data is gathered to influence local policy and decision making. The cases are briefly described below.

**Case 1, Drugs in Color (DC):** This initiative attempts to lower drug related nuisance. Groups of trained volunteers walk together with police officers, representatives of housing corporations and community workers in inspection rounds through the neighborhood. They search for predefined 'drug-related objects' and rate their observations according to a five-step analog color standardization.

**Case 2, Housing Report (HR):** To better understand housing shortages, a neighborhood led initiative was started to identify the causes of a lack of living space. To do so, an objective researcher, together with the local council, social housing company and statistical bureau of the municipality, analyzed data about the situation. This research was complemented with qualitative interviews with neighbors.

**Case 3, Citizen Blue (CB):** Residents patrol the neighborhood, in collaboration with the municipality, public maintenance service and local police to increase safety and foster social cohesion. During inspection rounds, volunteers map if the neighborhood is clean, intact and safe, i.e. by paying attention to overflowing dumpsters, broken streetlights and drug dealing. Observations are reported over a handheld tranceiver and are summarized by a trained citizen and depending on the origin of the issue, presented to the authority responsible.

Table 1 presents an overview of the three cases, their goal, type and frequency of data gathering as well as the actors involved in the initiatives.

<table><caption>Table 1. Overview of the three cases analyzed</caption><thead><tr><th></th><th>Drugs in Color (DC)</th><th>Housing Report (HR)</th><th>Citizen Blue (CB)</th></tr></thead><tbody><tr><th>Goal</th><td>Stop long-term annoyance of drug nuisance in the neighborhood</td><td>Raise attention for the housing shortage in the neighborhood</td><td>Stop long-term annoyance of disturbance in the neighborhood</td></tr><tr><th>Data</th><td>Quantitative</td><td>Quantitative & Qualitative</td><td>Qualitative</td></tr><tr><th>Frequency</th><td>Quarterly</td><td>Non-recurrent half year project</td><td>Once every two weeks</td></tr><tr><th>Supporters</th><td>Community worker, Local Police</td><td>Community center</td><td>Community center, Local Police</td></tr><tr><th>Gatherers</th><td>Concerned citizens</td><td>Independent researcher</td><td>Concerned citizens</td></tr><tr><th>Interpreters</th><td>Independent interpreter</td><td>Independent researcher</td><td>Trained citizens</td></tr><tr><th>Data Recipients</th><td>Municipality, Local police, Housing corporation</td><td>Local residents, Municipality, Housing corporation</td><td>Municipality, Local police, Housing corporation, maintenance service</td></tr></tbody></table>

## 3.2 Data collection

From each of the case studies we derived a large amount of data, in the form of direct observations and raw interview material. Observations of each case were documented
and analyzed, mapping the actors, triggers for data gathering, data transactions and the level of data enrichment during every transaction. Analysis of all three cases were compared and led to insights in the process steps of traditional data gathering initiatives. Interviews were conducted with members of citizen initiatives (n=5), community workers (n=1), members of municipality (n=6), data gatherers (n=9), independent researchers associated with data gathering (n=2), local police (n=3), and were transcribed, interpreted and up to a total of 433 statement cards were categorized. Statement cards show quotes, interpretations and paraphrases of the data found in our interviews and observations. This method allows a team to collectively organize and reorganize data to discuss interpretations, observe similarities and draw conclusions [26]. The actor analysis together with the statement card analysis led to an overview of their gathering processes and six elements that distinguish these processes.

# 4 Findings

## 4.1 Actors in traditional data gathering initiatives

From the analysis we identified, process supporters, data gatherers, data recipients and data interpreters, a set of actors that were found in all cases examined. Process supporters, like a community worker, are actors who organize the process and give guidance to the other actors in the process, while they can also give legitimacy to an initiative. Data gatherers are the actual data gatherers, citizens who actively gather data in their local environment. Data interpreters, receive, interpret and enrich the data gathered, after which they provide the other actors with verbal, written or visual feedback. This role is largely determined by the ability and expertise to analyze and add value to data.

## 4.2 Processes of a traditional data gathering initiative

To illustrate a traditional data gathering process we describe the process of the DC initiative. First, the community worker is triggered by the complaints of drug nuisance in the neighborhood and organizes inspection rounds. The method of data gathering is standardized, allowing the initiative to be taken seriously by local authorities and to receive their support. The community worker continuously plans inspection rounds and motivates citizens to participate. In the second step, citizens actively gather data and give the data to the community worker. While walking the inspection rounds, citizens are supported by local police, who can directly intervene when necessary and can ensure the safety of data gatherers. In the third step, the community worker (re)arranges the data and gives the data to the autonomous chairman. In this stage, the data is only synthesized and not enriched. In step four, the chairman analyzes, interprets and enriches the data, after which he provides the recipients with feedback (step five). This feedback is given in the form of a quarterly feedback meeting, where data serves as a tool to form a common vision. The feedback meetings are attended by local authorities, i.e. police officers, representatives of housing corporations, members
of municipality and community workers, and the data is presented in a presentation that all actors can understand. Everyone receives the same information and local authority directly provides feedback on solved or unsolved matters and explains underlying causes based on their domain expertise.

The other two cases were analyzed accordingly and the process steps of the cases were identified. These process steps were abstracted and merged into our presentation of these traditional data gathering processes (figure 1).

<figure><img src="image_2.png" /><figcaption>Fig. 1. Representation of traditional data gathering processes</figcaption></figure>

## 4.3 Elements identified as important in traditional data gathering initiatives

In this representation of traditional data gathering processes we define six elements: (1) support through mother organizations, (2) internal and external motivational triggers, (3) legitimacy through authority & partnerships, (4) standardization & methodology, (5a) data gathering, (5b) data interpretation, (5c) data presentation, (5d) data acknowledgement and finally, (6) feedback about short, mid and long-term outcomes. Below we elaborate on each of these elements.

**Support through mother organizations.** Mother organizations understand existing social structures, have access to communication channels and have ties with local authorities. The three examined cases all built on local neighborhood collectives, which acted as a launch platform for the gathering initiatives. They offered access to potential subsidies and domain experts due to their existing network. In our cases, mother organizations initially acted as catalysts for data collection, after which they all became semi-autonomous working groups within the community organization that also undertakes other community actions.

**Internal and external motivational triggers.** Actors within the mother organizations of the cases, articulated a local problem that formed the trigger and main motivation to start gathering data. DC was triggered by the degree of problems caused by drugs in the neighborhood, whereas in HR, the lack of suitable housing motivated residents to hire an objective researcher, while CB came into existence as a result of high levels of crime and public problems in the neighborhood, in combination with dissatisfaction with the action undertaken by local authorities to combat the issues.

**Legitimacy through authority and partnerships.** A central issue for gatherers was the need to have data taken seriously. Through interviews with public sector organiza-
tions involved with data gathering (police, local council, bureau of statistics or envi-
ronmental protection agencies), we found that these organizations have specific data
norms they adhere to. To assure legitimacy, groups involved local authorities or trust-
ed third parties. In the case of DC and CB, the police, local council, housing corpora-
tion actively partake in the project, while stressing their non-ownership. The HR re-
search was performed by an objective third party with domain knowledge, while the
housing company provided statistics for the researchers.

**Standardization and methodology.** In order for qualitative measurements to be usa-
ble by external organizations, actors must agree with standardized measurements. It
was important that data gatherers involve, or consult the earlier mentioned objective
third parties, to decide together what will be measured and how the data will be gath-
ered, i.e.; with a digital camera or notepad. In DC, this occurred at the initial phases of
the project, when a decision was made about the types of nuisances to record, when to
record them, and how they should be classified in the system. Similarly, HR worked
in collaboration with all actors to understand how the types of data recorded and pre-
sented by the social housing company, the council, and the statistics department can
be interpreted. CB also agreed on the types of data being collected and how disturb-
ances in the public sphere could be recorded.

These four elements conclude the organization and planning phase of the process.
What follows are two potentially iterative and repeating elements: active gathering
and feedback.

Data Gathering: *Gathering, Interpretation, Presentation and Acknowledgement.*

*Gathering:* The initial active element is physical data collection, where citizens gather data using the chosen standardization and methodology. During this process they are supported by trusted third parties or involved authorities. The degree of support offered can vary from participating in inspection rounds, to education and logistical support. Notable during this step is the pre-interpretation and decisions made by gatherers to not capture certain data because of contextual knowledge. This includes the occurrence of homeless persons that are not considered a nuisance (DC), tolerance towards broken up street areas as a result of construction work (CB), or reliance on storytelling (HR).

*Interpretation:* Having gathered the data, a certain amount of data interpretation is needed in order to gain insights. This interpretation is either done by a trained volunteer, or a paid professional. Local authorities can also take a role, by offering domain expertise, as is the case with DC, where police officers actively explain certain drug related issues. In our cases, care is taken during the interpretation phase to guarantee data quality and validity, making sure the data retains its legitimacy.

*Presentation:* Following the interpretation, actors are presented with the gathered data in a tailored form. This might include graphs, statistics or text summaries. Results were compared to earlier data gathering moments or data supplied by local authorities. Presenting insights in the presence of third parties and local authorities enabled discussion and clarification of the data by domain experts. For example HR used a
special neighborhood newspaper for direct stakeholders, combined with a special supplement in the local newspaper for other interested parties. CB, in contrast keeps track of data using a spreadsheet, centrally visible at the physical gathering place. The data is also communicated to the appropriate authorities, either whilst attending, or via email. These platforms are also notable for including data that is logged during un-official sightings, occurring outside the regular times of data gathering.

*Acknowledgement:* The acknowledgement phase, introduces the last iterative step in the data gathering element, where local authorities react to the gathered data in short feedback loops. This is an important element, since it is an acknowledgement of the effort of gathering, and can act as a motivator. Additionally it has a controlling function to make sure that action is planned, although it does not necessarily include active change. To illustrate, CB sends their reports of the week's activities to the local authorities and informs gatherers about the prospective feedback by the council. DC chose quarterly meetings to get the feedback of local authorities, where HR received feedback on their gathering at a final event where the report was presented.

**Feedback about short, mid and long-term outcomes.** This final element of the process is the outcome of data gathering initiatives. During this phase, we define short-term, mid-term and long-term outcomes. At the very least, as a short-term outcome, common ground is hoped for from which understanding about the problem is cultivated from both sides of the issue. Mid-term outcome focuses on the concrete actions. This might be a more pro-active approach to garbage collecting, alternative route suggestions for police patrol, or more inspection rounds in certain areas. Long-term outcomes can mean lower crime or a behavior change of one of the parties involved. In the case of DC, the data gatherers have a better understanding of the issues facing the police, while in HR, parties understand the problem better due to the increased availability of information.

This section presented an analyzed process of traditional data gathering by citizens, where people apply community knowledge by gathering data, enabling gathered data to be re-used by the public sector. In the following section, we discuss how this process can inform data gathering with digital means, in addition to what the benefits of using digital tools and techniques might be.

# 5 Discussion

As mentioned in Section 2, local knowledge during data gathering can yield important insights [9][11][19]. Issues like differing methods and techniques [21], data credibility, logistical issues, non-comparability and incompleteness of data [9] can prevent the data gathered from being re-used by the public sector. Digital means of data gathering offer advantages such as the ability to better validate results, increased access, in addition to better means of exploring the data and communication about the findings [9]. Digital means may be exploited to support and accelerate traditional data gathering initiatives, while the lessons learned from traditional gathering initiatives may be
beneficial to digital means for data collecting purposes. As we have observed, contextualizing collected data is important in the interpretation of the data, to retain community knowledge. A major challenge of digital means for collecting purposes is to capture and process the context that pertain to data, i.e., to contextualize data.

Digital means potentially solve traditional logistical issues by offering automatized processes, potentially resulting in efficiency and cost gains. Where the triggers for traditional data gathering are mostly introduced by local mother organizations, digital means can likewise allow likeminded but scattered groups to connect, which is beneficial. Also, digital means may bring emerging triggers to attention. By processing large amounts of collected data real-time, emerging phenomena may be exposed, serving as early warning triggers for local authorities [9].

Traditional data gathering initiatives however, have proven to be effective and rely on human senses and insight in addition to having their data re-used by the public sector. We ascribe this partially to the support of underlying mother organizations, which function as backbone of traditional data gathering initiatives. Digital data gathering initiatives on the other hand are loosely bound together by common interest, but without backing of an existing mother organization, it may prove challenging to have the same level of logistical support and connections as traditional initiatives. Furthermore, local authority commonly is not the initiator of data gathering, but is often attracted by the mother organization to collaborate in a later stadium, offering valuable feedback and legitimacy. Not having legitimizing partnerships in digital initiatives can make outcomes uncertain and can be a hurdle to realize change, since it is important to communicate that authority supports the data gathering initiative and that a valuable outcome of some sort is guaranteed to volunteers. Attracting an objective trusted third party that can act as the ‘face’ of the initiative could substitute the absence of a mother organization, since it can provide access to an existing community and can address local authorities to lend legitimacy to the data gathering initiative.

When actively gathering data, digital means offer the potential to allow incidental, ad-hoc measurement, for example through using a smartphone on location, rather than data measuring during predefined walks through the neighborhood. Moreover, digital standardization cannot be influenced by human inconsistencies and can increase the possibility to gather more credible, complete and comparable data [9]. However, care must be taken to allow human observations and interpretations into the methodology determination. Choosing and agreeing on standards and methods is important, since qualitative information gathered by citizens as human sensors is valuable in traditional data gathering. However, involving qualitative information increases complexity, making the training of data gatherers crucial to assure data credibility. In traditional initiatives this preceding training and support during data gathering is provided by mother organizations, which in the digital case would entail different type of support, such as a tutorial. Moreover, traditional efforts in our cases involve authorities in gathering the data from the field, enabling joint fact-finding, which can be beneficial for the relationship among actors [22].

In the examined traditional data gathering initiatives, interpretation of the data plays a major role and is often done by an objective trusted third party, making on-the-spot clarification possible. In digital efforts transparency about this process is also
important, giving participants insight into how data is translated, compared and com-
bined. It must be clear who presents the data, a role that can be fulfilled by an objec-
tive third party who is trusted by all stakeholders to interpret and communicate the
results. Digital means make it easier to combine and compare data sets from different
stakeholders, and just like in traditional means, to be presented to all stakeholders in
an understandable and transparent way. Furthermore, an analysis of combined data
sets results in a more comprehensive description of a phenomenon compared to an
analysis of a single set. Digital variants could benefit with near-instant translation and
presentation of results, but might lack the ability to consult domain experts on why
results are translated and presented in a certain way. Providing certain methods of
consultation on data interpretation can make this process more transparent.

However, adequate acknowledgement of local authority on the data collected stays
important. In traditional initiatives local authorities give data gatherers credit in the
form of appreciation of their efforts, resulting in engaged and active participants. In
line with the importance of the element of acknowledgement, we note that providing
data gatherers with feedback about short, mid and long-term outcomes, even when
change has not been realized, empowers citizens and acknowledges their efforts. Sim-
ilar efforts in digital data gathering initiatives would be beneficial.

Finally, we emphasize the need to be transparent in cases of automatic digital data
gathering. In our examined cases, participants actively participate, with full
knowledge of the data collection and intent. This transparency remains important in
digital data gathering initiatives.

6 Conclusion and future work

Digital data gathering has taken a leap and already has shown the potential of support-
ing citizens to contribute to policy and decision-making processes. Local governments
express the need to be informed by citizen data, but are not familiar with the organiza-
tion of digital data gathering processes. In this paper we propose that digital data
gathering can learn from traditional data gathering initiatives that have proven to be
effective. We also argue that traditional means stand to benefit from digital tools and
techniques. A traditional data gathering process with six elements that can serve as
guidance for local governments is presented, to achieve this.

We aim to contribute to the efforts of combining government data and citizen
gathered data for public sector reuse to contribute to policy and decision-making.
Also important is the potential of data gathering to build better relationships between
authority and citizens through joint fact-finding. Due to the limited amount of cases,
we emphasize that the presented process elements need to be tested in future research.

Our own efforts will focus on the standardization and methodology, where we aim
to investigate how existing standards, as determined by government, can be translated
to measurement standards that can incorporate human knowledge, whilst still retain-
ing legitimacy of the data. With the development of a use case based on the data gath-
ering demand articulated by local authorities in Rotterdam and an accordingly select-
ed target group, we aim to develop a mobile application according to an iterative re-
search and design approach.

**Acknowledgements.** We thank our colleagues of the Rotterdam University involved in the Rotterdam Open Data project and the participants for this study. The work has been partly funded by SIA RAAK Publiek.

**References**

1. Reddel, T., Woolcock, G.: From consultation to participatory governance? A critical review of citizen engagement strategies in Queensland. Australian Journal of Public Administration. 63, 75–87 (2004).

2. Amichai-Hamburger, Y.: Potential and promise of online volunteering. Computers in Human Behavior. 24, 544–562 (2008).

3. Johannessen, M.R., Flak, L.S., Sæbø, Ø.: Choosing the Right Medium for Municipal eParticipation Based on Stakeholder Expectations Theoretical Premises: Technology Evaluation through Genres. In: Tambouris, E., Macintosh, A., and Sæbø, Ø. (eds.) Fourth International Conference on eParticipation (ePart 2012). pp. 25–36. , Kristiansand, Norway (2012).

4. Maier-Rabler, U., Huber, S.: “Open”: the changing relation between citizens, public administration, and political authority. JeDEM. 3, 182–191 (2011).

5. Maisonneuve, N., Stevens, M., Ochab, B.: Participatory noise pollution monitoring using mobile phones. Information Polity. 15, 51–71 (2010).

6. Eriksson, J., Girod, L., Hull, B., Newton, R., Madden, S., Balakrishnan, H.: The Pothole Patrol: Using a Mobile Sensor Network for Road Surface Monitoring. MobiSys 2008 (2008).

7. Savage, N.: Cycling through data. Communications of the ACM. 53, 16 (2010).

8. Levine, C., Fisher, G.: Citizenship and service delivery: The promise of coproduction. Public Administration Review. 44, 178–189 (1984).

9. Gouveia, C., Fonseca, A., Câmara, A., Ferreira, F.: Promoting the use of environmental data collected by concerned citizens through information and communication technologies. Journal of environmental management. 71, 135–54 (2004).

10. Coleman, S., Gøtze, J.: Bowling Together: Online Public Engagement in Policy Deliberation Online Public Engagement in Policy Deliberation. (2001).

11. Corburn, J.: Community knowledge in environmental health science: co-producing policy expertise. Environmental Science & Policy. 10, 150–161 (2007).

12. Corburn, J.: Bringing Local Knowledge into Environmental Decision Making: Improving Urban Planning for Communities at Risk. Journal of Planning Education and Research. 22, 420–433 (2003).

13. Nicholson, E., Ryan, J., Hodgkins, D.: Community data-where does the value lie? Assessing confidence limits of community collected water quality data. Water Sci Technol. 45, 193–200 (2002).

14. Bromenshenk, J.J., Preston, E.M.: Public participation in environmental monitoring: A means of attaining network capability. Environmental Monitoring and Assessment. 6, 35–47 (1986).

15. Stokes, P., Havas, M., Brydges, T.: Public participation and volunteer help in monitoring programs: An assessment. Environmental Monitoring and Assessment. 15, 225–229 (1990).
16. Au, J., Bagchi, P., Chen, B., Martinez, R., Dudley, S.A., Sorger, G.J.: Methodology for public monitoring of total coliforms, Escherichia coli and toxicity in waterways by Canadian high school students. Journal of Environmental Management. 58, 213–230 (2000).

17. Girardin, F., Blat, J., Calabrese, F., Fiorre, F.D.: Digital Footprinting: Uncovering Tourists with User-Generated Content. Pervasive Computing, IEEE. 7, 36–43 (2008).

18. Fischer, F.: Citizens, experts, and the environment: The politics of local knowledge. Duke University Press (2000).

19. Corburn, J.: Bringing Local Knowledge into Environmental Decision Making Improving Urban Planning for. 420–433 (2003).

20. King, S.F.S., Brown, P.: Fix my street or else: using the internet to voice local public service concerns. Proceedings of the International Conference on Theory and Practice of Electronic Governance. pp. 72–80. , Macau (2007).

21. Yearley, S.: Experts in Public: Publics’ Relationships to Scientific Authority. Making Sense of Science: Understanding the Social Study of Science. pp. 113–128. SAGE Publications Ltd, London (2004).

22. Susskind, L., Elliott, M.: Learning from Western Europe. Paternalism, Conflict, and Coproduction: Learning from Citizen Action and Citizen Participation in Western Europe. p. 3. Plenum Press (1983).

23. Ehrmann, J., Stinson, B.: Joint Fact-finding And The Use Of Technical Experts. The Consensus Building Handbook: A Comprehensive Guide to Reaching Agreement. pp. 375–400. Sage Publications (1999).

24. Yin, R.: Case Study Research Design and Methods. SAGE, London (2009).

25. Baxter, P., Jack, S.: Qualitative case study methodology: Study design and implementation for novice researchers. The Qualitative Report. 13, 544–559 (2008).

26. Visser, F.S., Lugt, R. Van Der, Stappers, P.J.: Participatory design needs participatory communication: New tools for sharing user insights in the product innovation process. Proceedings of 9th European Conference on Creativity and Innovation - September 2005. pp. 1–17. , Lódz, Poland (2005).