# Balancing Potential and Risk: The Living Lab Approach in Mobile Participation Research

Magnus Adenskog, Joachim Åström, Titiana Ertiö, Martin Karlsson, Sampo

Ruoppila, Sarah-Kristin Thiel

► To cite this version:

Magnus Adenskog, Joachim Åström, Titiana Ertiö, Martin Karlsson, Sampo Ruoppila, et al. Balancing Potential and Risk: The Living Lab Approach in Mobile Participation Research. 9th International Conference on Electronic Participation (ePart), Sep 2017, St. Petersburg, Russia. pp.12-23, 10.1007/978-3-319-64322-9_2. hal-01703323

HAL Id: hal-01703323

https://inria.hal.science/hal-01703323

Submitted on 7 Feb 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Balancing Potential and Risk: the Living Lab Approach in Mobile Participation Research

Magnus Adenskog¹ª, Joachim Åström²ª, Titiana Ertiö³b, Martin Karlsson⁴ª, Sampo Ruoppila⁵b, and Sarah-Kristin Thiel⁶c

ªÖrebro University, Urban Studies, Sweden

magnus.adenskog@hkr.se, joachim.astrom@oru.se, martin.karlsson@oru.se

ᵇUniversity of Turku, Department of Social Research, Finland

titiana.ertio@utu.fi, sampo.ruoppila@utu.fi

ᶜAustrian Institute of Technology, Vienna, Austria

sarah-kristin.thiel@ait.ac.at

**Abstract.** Living labs as a research approach have been said to hold many promises regarding the evaluation of state-of-the art technologies in real-world contexts, for instance by allowing close cooperation with various stakeholders. At the same time, a living lab approach is connected with substantial complexity and increased risk. This paper elaborates on a conducted living lab with the objective to explore challenges and opportunities of mobile participation. For this purpose a novel mobile application enabling interaction between citizens and city authorities was tested over a period of five months in Turku, Finland. In this paper, we describe identified risks associated with a living lab approach to mobile participation research. We conclude with an overall evaluation regarding the appropriateness of the living lab approach within the e-participation research field and provide recommendations on how to balance potential and risk in future projects.

**Keywords:** mobile participation, citizen participation, urban planning, living lab, trust

## 1 Introduction

Governments around Europe are trying to improve ways to integrate citizens in public decision-making processes [15]. Hopes have been placed especially in information and communications technologies to broaden the scope of involved citizens [21]. Most recently, great expectations have evolved around smartphones as platforms to achieve these aims. Always carried along, they offer affordances to participate wherever and whenever, including reflecting on a topic in situ [12]. Besides written comments, phones also enable the supply of sensor data such as GPS. Nonetheless, the urban governance and planning applications have yet to exploit this potential [7, 9].

To take a step further in exploring advanced mobile participation, our research group set the task to define the pre-requisites (e.g. [8]), build a working
prototype, and test it in a living lab. The Täsä app (here in local dialect) was
tested in real-world circumstances in Turku, Finland, from May to October 2015.

'Living lab' is a research and innovation methodology that entails involving end users in an early stage of the research process as well as conducting experimentation in real-world contexts rather than in a controlled setting [1]. As a research approach, living labs suggest a lucrative opportunity for studying the impact of state-of-the-art technologies in real-world contexts. Optimally they may provide great potential for scientific discoveries as well as the development of applications and practices. This opportunity stands in contrast to much of the existing research on e-participation and democratic innovations, which concentrates either on experimental research conducted in controlled environments, and addressing specific research questions (e.g. mini publics, deliberative polls, online discussion experiments), or research on real world e-participation cases, but based on broad evaluative frameworks (e.g. studies of e-petitioning websites, everyday talk etc.). While the first category might have access to state-of-the-art technologies, it tends to lack applicability [17]. The latter category, on the other hand, suffers from a reactive approach to the current practices, limiting its perspective on real-world complexities to a frame based on previous knowledge, and lacking access both to the most promising technologies and to the most challenging forms of participatory governance.

Against this backdrop, it is evident that a living lab approach offers im-
portant advantages for e-participation research, combining state-of-the-art tech-
nology, collaborative project design between practitioners and researchers, and
implementation in real-world context [3]. However, living labs are also associated
with increased risk. Emerging technologies might cause usability problems, and
real-world circumstances might have more complex settings than expected, both
causing vulnerabilities. Hence, implementing a living lab entails a balancing act
between potential and risk.

In this paper, we reflect on our living lab experience in studying the im-
plications of the newly introduced mobile participation app Täsä. The project
produced valuable insights on how a mobile setting can enrich participatory
planning. In this paper we focus on the identified risks of a living lab approach,
which constitute an important part of our overall results. We conclude with
recommendations to future developers of mobile participation processes.

2 Running the Living Lab

In this chapter, we outline the main features of Täsä, who used it and the
collaboration with the municipality.

2.1 Täsä application

The mobile participation application Täsä, developed and tested in the Build-
ing Pervasive Participation project ("b-Part, 2013-16), allowed citizens to be-
come involved in urban planning and place-based development in various ways.
User-generated geo-referenced pieces of content were central to the applications concept. In contrast to traditional reporting apps, Täsä allowed for a further differentiation of posts into the contribution types: Issue, Idea, Opinion, and Poll. With those, citizens could make their intentions clear. All contributions could be supplemented by adding a photo, a point of interest and an icon that corresponded to the persons perception of the addressed situation. Other citizens were able to browse contributions on a map and as a list, express their support by liking and leave textual comments to spark discussions. In order to achieve a two-way communication between a city and its citizens, city officials were encouraged to respond to citizens input. As an additional element, city representatives could create *missions* (e.g. asking for feedback or calling for ideas) that citizens could respond to by tagging their contributions with the corresponding mission identifier. To further encourage citizens, several game elements were incorporated in Täsä [20]. In general, the system served as a crowdsourcing tool and indicator for hot topics, providing planners an overview of citizens concerns and opinions.

## 2.2 Usage results

The results on who participated self-selectively in the living lab, what kind of topics they addressed through the mobile app, and what was the spatial pattern of participation, have been reported at length in other papers, but are only summarized here to give more information about the living lab. The analysis draws from usage data stored in the backend and user surveys – one sent to the users immediately after registration (pre-survey hereafter) and another at the end of the trial (post-survey). The first survey was designed to collect various sorts of background information, whereas the second focused on experiences related to the application and the trial in general.

Altogether 780 citizens downloaded Täsä, and one third (32%) of them produced one or several kinds of content - contributions, comments, likes, or votes. Similarly to many ambitious e-participation initiatives previously, most of the Täsä-users were younger and had higher level of education than Turku inhabitants on average. Regarding ownership of the most recent devices as well as motivation and skills required to use them, mobile participation sets apart as a typical novelty along the expected lines of a 'digital divide', which is partly age-related. Yet, it showed potential in involving a group notoriously absent from face-to-face forms of citizen participation: the young to middle aged citizens [10]. Further, an analysis taking into account attitudinal predispositions such as an interest in politics and satisfaction with democracy revealed, importantly, that mobile participation can contribute to a constructive re-engagement with 'critical citizens' interested in politics but dissatisfied with democracy [2]. All in all, the results showed that groups who care about politics are more interested in participating - whether they are satisfied or dissatisfied with the current state of affairs. Citizens who are not interested in politics are less likely to participate, even with new tools.
The qualitative analysis of the content produced showed that most (81%) of the citizens postings were about the urban environment (e.g. infrastructure, green areas), transportation planning, or various recreational activities. Theoretically participation focused on 'common good' issues, which is an important finding. The protectionist NIMBY (Not In My Backyard) attitude, often associated with citizen participation by the skeptics (see section 3.2), was totally absent [18].

The spatial analysis revealed more usage in the city center than in the suburbs. This logically matches the 'common good' topics: the most frequented public spaces or green areas and the most intense transportation planning conflicts, for instance, are found in the city center (see [18]).

In contrast to our expectations, majority of Täsä-users were primarily interested in bringing their own issue to the attention of the municipality, and showed little interest in interactive features, especially discussing other peoples postings [10]. The usage was thus characteristically individualist, and almost resembled a typical use of reporting apps. Moreover, the incorporated game elements were considered meaningless in supporting motivation to engage [19]. The respondents of our end-survey were critical of the usability of the prototype (see section 3.1), but highly supportive of developing the mobile participation concept. The affordances of mobile participation, such as not being restricted by temporal and spatial constraints, and ability for reflection 'on site', were considered valuable [10].

## 2.3 Collaboration with the municipality

Implementing the living lab required active collaboration with the City of Turku. Their attitude towards becoming a platform for mobile participation trial was straight-forward, or even enthusiastic. In practice, the research team hosted several workshops with the city officials to prepare them in advance to think about potential benefits of the app, asking them to propose topics for missions (i.e. participatory tasks given by the municipality), and helping them with communications once the application went live. The City of Turku granted the Täsä-app an official participation platform status during the trial, and actively marketed it through their communication channels online as well as via printed and social media. The research team also did its share of marketing through social media, appearances in the local newspapers and radio, citizen workshops, and other efforts.

While the collaboration with the municipality ran quite smoothly all in all, there were three challenging occasions, which reflect how the new ideas of citizen participation, represented by the trial, did not easily match with the current governance culture regarding citizen participation, and hence were not optimal for the trial.

First, as we were encouraging the planners to suggest missions that could be implemented during the trial, they tended to come up with only light-weight and uncontroversial tasks, indicating an interest in harmless participation thereafter rather than serious discussions and significant citizen insights on urban
development policies. We needed to return to the issue a few times to get them
to suggest at least some potentially more controversial tasks. On one occasion,
we agreed with an urban planner to include a mission in which citizens could
suggest potential new uses to an old power station owned by the city and soon
to be vacated. However, once the real estate department found out about this
plan, they insisted on withdrawing it, because they "did not want public par-
ticipation anymore than was absolutely necessary" (excerpt from e-mail). This
illustrates general skepticism on the benefits of planning, presented by many
local politicians and planners, discussed in section 3.2.

Secondly, our aim was to develop an interactive application, meaning that
we wanted to encourage discussions among citizens, but also between citizens
and city authorities. Responding to this, the city insisted on synchronizing the
use of the application with their permanent electronic participation system, a
web-based (at the time of the trial) rather clumsy model, which people use
predominantly to report problems and concerns, but which can also be used to
transmit new ideas. For that service, the municipality has in place a follow-up
system, in which more than 100 city officials from different departments are
involved to give an answer to each message within a few days. Instead of getting
people to discuss in Täsä, we were obliged to transfer the input generated in Täsä
to the other system, and the answers given back to Täsä. The main problem
here was the way the city officials responded. Characteristically, they did not
participate in discussion, were not eager to learn more, but gave 'the answer' that
almost every time killed the discussion going on in Täsä instead of encouraging
it. Instead of being able to harness a new participation culture, the new tool was
forced to integrate into an existing but unfitting form of communication.

Thirdly, how and to what extent citizen feedback was taken into considera-
tion, basically depended on the judgment of each city official who engaged with
the feedback. There is no way to follow up on this type of engagement. Most
likely, individual actors viewpoints in this matter, and therefore their implemen-
tation, differ substantially. The unclear status of citizen input is however the
single most obvious problem in the Finnish system of participatory planning [4]
- not only in this trial.

3 Challenges Encountered in the Living Lab

In this section, we develop the two challenges we identified during the Täsä trial.

3.1 Challenges of the new participation application

Testing a novel kind of participation application brought about a number of
challenges, mainly related to its usage. Although we communicated that it was
a prototype produced within a research project and not a polished product, the
users did not seem to be lenient towards any shortcomings. The user experi-
ence expectations are set high by social media apps, for instance, and delays or
crashes experienced - inevitable in a prototype - increase the risk of losing users.
Admittedly, it took quite long time for instance to load the map and display all the contribution and missions icons on it, and some shortcuts between displays could have been added. Based on the feedback we received during the trial, and actively collected, we elaborated on the user experience.

Among the people who registered as Täsä-users, 68% did not produce any content with the app. In order to understand this behavior, we approached a random group of such users (during the trial, via e-mail) to conduct interviews. It turned out that some had in fact taken a passive onlookers role, while others had stopped using Täsä altogether. The previous group included many who felt they did not have anything to contribute on the topic (but were interested in what others did), while the latter group comprised many who had abandoned the app for the usability issues, e.g. feeling the app was "clumsy and hard to use". It also turned out that one big challenge was communicating to the citizens what they could do with the app. We assumed that users were familiar with the use of apps from other activities such as entertainment, social media or work, but as one informant pointed, the concept of missions (i.e. tasks given by the city administration) was difficult to grasp. Similarly many of the game elements included in Täsä were not even recognized by number of users, as it turned out in the post-survey.

Among other questions, the post-survey asked participants how the app worked. 45% of respondents answered that it worked fairly or very poorly, 37% satisfactorily (value 3 on a 5-point Likert scale) and 19% fairly well. Surprisingly, many respondents gave minute feedback by answering the open questions. While the respondents were generally supportive of developing the mobile participation further and recognized the project as a "good start" - they were very critical of the usability of the app. Some of the feedback was highly general, e.g. "the app was slow and crashed", "it should have worked properly", "difficult to use, it reduced my participation" or "the app was badly designed", which indicates that for some users, the usability challenges made them abandon the app altogether. On the other hand, many other users, who did indeed spend some time with the app, provided more specific comments, which can be quite helpful for refining the application. Examples include "when I wanted to add a location, it showed strange places nearby", "faster loading map", "it was hard to add the picture", "the registration process annoyed me" or "it would have been nice to see some visual summaries in the app". The amount of feedback received from the users signals their interest in a better functioning application in the future and hence encourages developing mobile participation further.

More generally, participants reported to be indifferent towards the game elements we introduced in Täsä [19]. Points that were awarded for in-app activities (e.g. commenting) also did not raise participants’ interest. These incentives should have been linked to rewards that provide meaning and thus make the incentives more attractive [16]. Some of the (game) elements were not even recognized as such. Altogether, they did not serve as the motivators we had anticipated. As somebody put it, "the game elements brought no pleasure since the app worked poorly". Interestingly enough, when asked to evaluate how the app
worked, more than half of participants indicated that Täsä worked satisfactorily (37%) and only 13% indicated that it worked very poorly.

In retrospect, not only the usability of the app was considered troublesome, but the entire idea of having only the app. During the trial, several participants expressed a wish for a possibility to participate using a personal computer. Some more knowledgeable participants stated that they would have preferred an open, mobile-optimized web page, which could have been accessed across many types of devices. In fact, the respondents ranked a web-based platform (83%) as a more important channel for interacting with the local government than a mobile application (65%), although both ranked higher than other alternatives. While the preference for (any, not only our) mobile application was not found to depend on age (rs= -.211, p=.142), it was positively influenced by perceived mobile skills (rs= .315, p=.026). Those favoring web-based applications were not only less experienced in using mobile devices, but were also older (rs= -.294, p=.038). These insights are in line with the finding that older generations feel less invited by mobile technologies in the context of public participation, which might be a reason for them having been less active. Moreover, those rating mobile interaction channels as important were also more active participants than those who did not (rs= .218, p=.017).

In the Täsä-trial, the basic access barriers, such as owning a relatively good smartphone, being motivated to participate in urban planning and having the skills to do so, also played a role in who could participate in the first place. Although smartphone ownership is relatively high in Finland, a suitable device running a rather recent operating system was needed for the Täsä-application to work well. This was due to having employed a framework that allowed for cross-platform development, making it possible to have the exact same app for all major operating systems (i.e. iOS, Android, and Windows). This in turn, contributed to the self-selection of active, working age participants, with a good socio-economic status, and good skills in using a mobile phone [10].

In summary, participants encountered a number of technology-related obsta-
cles that negatively influenced their participation behavior. While some aspects
were specific to the implementation of our application, others apply more to the
individuals attitudes towards adopting novel technologies.

## 3.2 Political distrust and skepticism on citizen participation

The relationship between trust and citizen participation is both intricate and multifaceted. Substantial trust between actors is required for constructive and cooperative processes of citizen participation. In order for citizens to devote time and effort to participatory procedures, with uncertain impact on policy, they must trust the intentions and compliance of planners and policy makers. Oppositely, both policy makers and planners must trust the capacity, intentions and knowledge of citizens' engagement and input in the participatory procedure in order to be willing to delegate power to implement their suggestions [11].

Yet, at the same time, citizen participation in general and democratic inno-
vations in particular are often hailed as a means to restore trust between citizens
and elites in situations of mutual and wide-spread distrust [22]. Citizen partic-
ipation is thought to strengthen trust through consensus building [6], fostering
an increased understanding of the 'other' [5] and strengthened problem solving
capacity through knowledge sharing and cooperation [14].

To map the political context in Turku, we conducted a round of interviews
among local politicians and civil servants between 2014 and 2015. Through the
interviews we identified a wide-spread skepticism regarding the potential of cit-
izen participation to inform policy-making and planning in Turku. The most
common concern raised in the interviews relating to the opinions, knowledge
and ideas generated by the citizens in participatory initiatives was that the cit-
izens were unable to see, or take interest in, 'the big picture' or the 'universal
interests' of the city. Instead the interviewees found that citizens through partici-
patory processes again and again represented and communicated their particular
self-interest. One such example is that NIMBY (Not In My Backyard) issues,
according to civil servants and politicians, were a common theme among citizens.
Similar observations were made among civil servants indicating that citizens tend
to become more interested in issues close to their neighborhood rather than gen-
eral or large scale issues. Interestingly enough, however, the participatory input
in Täsä was free from NIMBY content, as it was used mostly to indicate 'com-
mon good' topics (see section 2.2). All in all, the attitudes and perceptions of
the policy makers and planners indicate a distrust in the ability and intention of
citizens to represent or advocate the general interest in participatory processes.
Citizens are perceived to advocate their particular interest as well as abstain
from participation unless they are directly affected by the issue at hand.

Another recurring theme in the interviews was that the politicians and plan-
ners perceived the citizens of Turku to resist all or most changes in the urban
area. Hence, there is a widespread perception of skepticism to change in urban
planning in the modus operandi of the engaged citizen. For instance, several in-
terviewees stated that when a public meeting is called for, it is primarily citizens
who are critical towards the project at hand that show up. Hence, citizens were
viewed as neither constructive nor cooperating with the policy makers and plan-
ners, but rather as opponents creating obstacles in the process. Taken together,
these experiences and attitudes suggest that the input of the citizens is seen as
overtly critical. This in turn legitimizes the view that the knowledge provided
by the citizens is not considered salient and that the politicians therefore many
times make their decisions without considering the opinions provided by the
citizens in the participation initiatives.

So far, we have focused on describing the lack of trust in citizens among
policy makers and planners. Yet, an equally important part of the equation is
understanding the citizens' and particularly the participating citizens' attitudes
and dispositions towards the political institutions and actors. Based on the sur-
veys conducted among registered Täsä-users, we could analyze their level of trust
for political institutions.

Many of the Täsä-users (living lab participants) shared a relatively low level
of trust in politicians, parties, and the local government of Turku. More than four
out of ten participants expressed distrust in the local government as they scored
on the lower half of the ten-point trust scale. Five out of ten participants distrusts
politicians in general and six out of ten distrusts political parties. Further, there
seemed to be a discrepancy between the participation level of general satisfaction
with how democracy works in Finland and their level of trust for local political
institutions and actors. More than a third (36%) of the participants, who in
general were satisfied with how democracy works in Finland, still lacked trust in
the local government of Turku. Hence there seems to be evidence of a reciprocal
distrust between citizens and local city officials.

Moreover, it also seems that the citizens’ distrust is stronger regarding the
local level, and hence is connected to the local context rather than an expression
of a general discontent with the Finnish democracy. On the whole, the context
of the living lab must be described as contentious as we find evidence of a
reciprocal distrust between local city authorities (i.e. politicians and planners)
and citizens. Implementing an experimental and progressive mobile participation
experiment in such a contentious context was challenging as neither planners,
policy makers, nor citizens were ready to trust the genuine engagement and
willingness to cooperate of the other.

4 Conclusions and Discussion

Living labs are projects in which the high potential is correlated with high risk, and vice versa. By raising the degree of novelty, and the complexity of the project one also increases the risk levels. As the Täsä living lab explored how novel technology could be applied in a real urban planning context, this project ought to be categorized a high risk project. In this paper we identified two major challenges that correspond to potential risks with living lab projects: (1) challenges of introducing novel and emerging technologies and (2) conflict dimensions in the political context. We believe that an understanding and anticipation of these identified risks are of importance for the fields of mobile participation as well as for future living lab projects within the field of political participation in general. Based on these identified risks, we recommend future projects to reflect on the following potential counter measures.

**Cross-platform e-participation solutions to serve diverse participants.** The introduction of innovative participatory tools creates new divides when it comes to usage. Anticipating the changes is hard and often leads to unreasonable expectations. Our living lab trial with Täsä confirmed again that new tools or applications are most likely used by 'early adopters'. Even more so, some citizens prefer only to be informed (than actively participate) or even choose to 'rationally ignore' invitations to participate [13].

In order to avoid generational divides and achieve a more inclusive participa-
tion process, we recommend – at least for the time being – to provide a solution
accessible with both mobile and personal devices, phones and computers. The
setting of our project specifically sought to investigate a novel pervasive par-
ticipatory tool. For near future use, we recommend as a workable solution to
aligning mobile apps with a web-based participation alternative. Interestingly enough, our trial showed that the digital divide is no longer placed at the intersection of online and offline, but between mobile phone vs. PC and app vs. webpage. Overall, while designing an inclusive participation process should be the top priority for any participatory trial, the pre-conditions should be fulfilled as well: citizens need to be aware of a) the opportunity to participate and b) the benefits active participation entail.

**Designing participatory processes to act as a mediating institution between political actors.** The lack of trust in institutions among citizens is well documented, but the distrust of politicians and planners in the citizens' competencies and intentions is often disregarded [23]. This project identified a prominent reciprocal distrust between citizens and public officials that challenged the implementation of the living lab. Wide-spread distrust within the context of the living lab may have hindered broader engagement among citizens as well as stronger commitment from public officials. Future projects ought to strive to design participatory processes to act as mediating institutions. Such designs could include third party actors with a mediating role between citizens and public officials as well as introducing a participants' ombudsman within the institutional structure that can strengthen the position of citizens in the participatory process.

Lifting our eyes from the specific challenges of mobile participation to the potential of the living lab approach in e-participation research in general, we found that it lends itself well to the type of exploratory research that testing new technologies entails. It offers a tough test of novel technologies that is effective in identifying challenges and risks as experimentation is conducted among users rather than test subjects, and in the context of intended implementation rather than a lab. In the context of our project, the living lab format made possible the discovery of challenges and issues that would not have been identified in a strictly experimental test of Täsä. A strictly experimental test of the Täsä application would not have been able to identify the usability challenges connected to introducing a novel participation technology in a real world setting. Further, under strictly experimental conditions we would not have been able to identify the challenges of implementing an e-participation process in a context characterized by wide-spread distrust. Hence, in the balancing act between potential and risk – as conducting a living lab might be described – we ultimately opted for raising the stakes rather than playing it safe.

**Acknowledgements** The research project Building Pervasive Participation (b-Part) was funded by JPI Urban Europe. The authors would like to thank the City of Turku for their collaboration.
References

1. Almirall, E., Wareham, J.: Living labs and open innovation: Roles and applicability. eJOV: The Electronic Journal for Virtual Organization & Networks 10 (2008).

2. Åström, J., Karlsson, M.: Will e-participation bring critical citizens back in? In: International Conference on Electronic Participation. pp. 8393. Springer (2016).

3. Åström, J., Ruoppila, S., Ertiö, T., Karlsson, M., Thiel, S.K.: Potentials and challenges of a living lab approach in research on mobile participation. In: Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers. pp. 795800. ACM (2015).

4. Bäcklund, P., Mäntysalo, R.: Agonism and institutional ambiguity: Ideas on democracy and the role of participation in the development of planning theory and practice-the case of Finland. Planning Theory 9(4), 333350 (2010).

5. Carr, D.S., Halvorsen, K.: An evaluation of three democratic, community-based approaches to citizen participation: Surveys, conversations with community groups, and community dinners. Society & Natural Resources 14(2), 107126 (2001).

6. Cheryl, K., Stivers, C.: Government is us: Public administration in an anti-government era (1998).

7. Desouza, K.C., Bhagwatwar, A.: Citizen apps to solve complex urban problems. Journal of Urban Technology 19(3), 107136 (2012).

8. Ertiö, T., Ruoppila, S.: Supporting 'participation' in mobile participation. In: Proceedings of the ePart 2014 Conference, Dublin, Ireland. pp. 310 (2014).

9. Ertiö, T.P.: Participatory apps for urban planning - space for improvement. Planning Practice & Research 30(3), 303321 (2015).

10. Ertiö, T.P., Ruoppila, S., Thiel, S.K.: Motivations to use a mobile participation application. In: International Conference on Electronic Participation. pp. 138150. Springer (2016).

11. Harding, M., Knowles, B., Davies, N., Rouncefield, M.: HCI, Civic engagement & Trust. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 28332842. ACM (2015).

12. Korn, M.: Situating engagement: Ubiquitous infrastructures for in-situ civic engagement. Ph.D. thesis, Aarhus Universitet, Science and Technology, Department of Computer Science (2013).

13. Krek, A.: Rational ignorance of the citizens in public participatory planning. In: 10th symposium on Information-and communication technologies (ICT) in urban planning and spatial development and impacts of ICT on physical space, CORP. vol. 5, p. 420 (2005).

14. Lawrence, L., Debbie, A., Deagen, R.: Choosing public participation methods for natural resources: a context-specific guide. Society & Natural Resources 14(10), 857872 (2001).

15. Michels, A.: Innovations in democratic governance: how does citizen participation contribute to a better democracy? International Review of Administrative Sciences 77(2), 275293 (2011).

16. Nicholson, S.: A user-centered theoretical framework for meaningful gamification. Games+ Learning+ Society 8, 1 (2012).

17. Parkinson, J., Mansbridge, J.: Deliberative systems: deliberative democracy at the large scale. Cambridge University Press (2012).

18. Ruoppila, S., Åström, J., Lybeck, R., Karlsson, M.: Putting citizen participation in place: Will mobile phones change the spatiality of participatory planning? Article manuscript (2017).
19. Thiel, S.K., Ertiö, T.P.: Play it to Plan it? The Impact of Game Elements on Usage of a Urban Planning app. In: Saqib Saeed, T.R., Mahmood, Z. (eds.) User-Centric E-Government - Challenges and Opportunities, vol. 1. Springer, 1 edn. (2017), forthcoming.

20. Thiel, S.K., Lehner, U.: Exploring the effects of game elements in m-participation. In: Proceedings of the 2015 British HCI Conference. pp. 6573. British HCI 15, ACM, New York, NY, USA (2015).

21. Vogt, M., Fröhlich, P.: Understanding cities and citizens: Developing novel participatory development methods and public service concepts. In: Real Corp 16: 21st International Conference on Urban Planning and Regional Development in the Information Society GeoMultimedia 2016. pp. 263272 (2016).

22. Wang, X., Wan Wart, M.: When public participation in administration leads to trust: An empirical assessment of managers perceptions. Public Administration Review 67(2), 265278 (2007).

23. Yang, K.: Public administrators trust in citizens: A missing link in citizen involvement efforts. Public Administration Review 65(3), 273285 (2005).