# Hands-on guideline for e-participation initiatives

Sabrina Scherer, Maria A. Wimmer, Stefan Ventzke

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Sabrina Scherer, Maria A. Wimmer, Stefan Ventzke. Hands-on guideline for e-participation initiatives. Joint IFIP TC 8 and TC 6 International Conferences on E-Government, E-Services and Global Processes (EGES) / Global Information Systems Processes (GISP), Held as Part of World Computer Congress (WCC), Sep 2010, Brisbane, Australia. pp.49-61, 10.1007/978-3-642-15346-4_5 hal-01054634

HAL Id: hal-01054634

https://inria.hal.science/hal-01054634

Submitted on 7 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Hands-on guideline for e-participation initiatives

Sabrina Scherer, Maria A. Wimmer, Stefan Ventzke

University of Koblenz-Landau, Institute for IS Research, Research Group E-Government,
Universitätsstraße 1, 56070 Koblenz, Germany
{scherer,wimmer,sventzke}@uni-koblenz.de

**Abstract.** E-participation applications enable online participation of citizens and interested stakeholder groups in political debates and strategic decision-making. The tools, channels and devices through which online participation takes place require proper design to support citizens, politicians and other actors. To incorporate the needs of these actors into the functionalities of an eParticipation platform, this contribution proposes a hands-on guideline for e-participation initiatives. It has been generated from the experiences of two European projects: VoicE and VoiceS. The paper describes the six-step iterative process to successfully plan and realize e-participation initiatives.

**Keywords:** citizen participation, e-participation, project implementation guideline

## 1 Introduction

Although e-participation has been introduced as a new discipline only some years ago, the use of Information and Communication Technologies (ICT) in political participation has a longer tradition. The potential of using this medium to reach many people increased with the spread of the Internet [1]. This development has reached another push with the recent evolution and wide take-up of technologies summed up under the buzzword "Web 2.0". The "participatory web" promised new possibilities for political online participation [2].

Nevertheless till 2010, civic participation has in many countries still not reached the expectations affiliated with the hype of e-participation and Web 2.0. One reason is that the possibilities of e-participation have not yet been sufficiently exploited. Too little interaction takes place between the different parties in the policy life-cycle<fnref n="1" />. While information offerings are often high level, active participation opportunities are hard to find or limited (e.g. in the German Parliament as a study from 2008 evidences [3]). Additionally, often a conceptual integration in the political process is missing: "Participants are unable to understand the purpose of the debate, to identify the addressee of the postings and to see in which form the results are further processed in the political process" ([3] p. 47).

<footnote n="1">¹ The stages of the policy life-cycle and levels of participation are detailed in [36].</footnote>
The success of e-participation solutions depends heavily on the organizational
planning and the incorporation into the policy making or political processes. Current
solutions in the area of e-participation suffer from insufficiently responding to
requirements specific for e-participation. A successful introduction of e-participation
does not only require an adaptation of given processes. Sometimes new participation
procedures, which have not existed before, need to be introduced. The lack of a
sophisticated guideline with a holistic approach is tackled in the contribution at hand.
The paper introduces a hands-on guideline for projects and organisations that build up
e-participation initiatives. The guideline is based on a well-defined iterative process.
It results from findings of the projects VoicE<fnref n="2" /> and VoiceS<fnref n="3" />.

The VoicE Internet platform promotes the dialogue between citizens from two
European regions (Baden Württemberg, Germany<fnref n="4" /> and Valencia, Spain<fnref n="5" />) and policy
makers from the European Parliament as well as from other institutions of the
European Union (EU) and regional assemblies. In terms of contents, the project
focuses on the policy field of consumer protection in the EU. Citizen participation in
VoicE is targeted at the legislation proposal formation stage and the debate on draft
legislation [4]. In the follow-up project VoiceS, the VoicE platform is improved and
complemented by adding a series of new features such as a serious games, semantic
search, social networking tools and a toolkit for regional e-participation [5].

To implement such a toolkit, this contribution gives instructions for effective
transfer of knowledge and good practice cases and describes a guideline for e-
participation knowledge transfer. With it, this paper summarizes results and lessons
learned from the VoicE and VoiceS projects. The next section introduces the
methodology to construct the guideline for e-participation initiatives. Section 3
describes the guideline in detail. Concluding remarks and an outlook for further
research are provided in section 4.

## 2 Methodology

According to Macintosh et al. [6], the evaluation of e-participation experiments is still
in its infancy as the nature of e-participation is fragmented, unfocused and
geographically dispersed. A number of researchers and networks started to develop
evaluation frameworks (see e.g. [7, 8, 9]). Many approaches provide
recommendations usable to establish e-participation or e-democracy policies (see e.g.
[10, 11]). Some approaches provide general recommendations or requirements for e-
participation, which are usable in practice when first time an e-participation
application is to be set up [8, 9, 12].

<footnote n="2">² VoicE: Giving European people a voice in EU legislation, www.give-your-voice.eu</footnote>

<footnote n="3">³ VoiceS: Integrating semantics, social software and serious games into e-participation, www.eu-voices.eu</footnote>

<footnote n="4">⁴ www.bw-voice.eu</footnote>

<footnote n="5">⁵ www.voice.gva.es</footnote>
In this paper, we generalise lessons, experiences and results from VoicE and VoiceS, and we ground them with insights from literature studies. Subsequently, we structure them and make them usable for other e-participation projects in an easy-to-understand step-by-step guideline.

The hands-on guideline for e-participation initiatives gives step-by-step recommendations for e-participation project implementation therewith answering the following questions:

* How can e-participation processes be planned and implemented in an effective way?

* How can actors' needs be incorporated in the design of e-participation features and platform structures, how to choose the appropriate tools, and how to develop the e-participation platform to best fit the actors' needs?

* How to handle the preparation of the information related to topics to be discussed?

## Approach to develop the hands-on guideline

Fig. 1 visualises the methodology applied to answer the three questions and ground the experiences of VoicE and VoiceS with literature study. The three lanes in the diagram separate the work performed in the projects from grounding results in literature studies. The boxes represent activities performed as part of the investigations. The shapes for documents represent results achieved and documented. The arrows show the flow of activities or the next usage of a result. Results achieved at a given step are further elaborated in the subsequent steps. The investigations in the projects have always been supported with findings from literature studies.

In order to elaborate the hands-on guideline for e-participation initiatives, the following nine activities have been performed:

1. The VoicE project started with a requirements analysis, which was based on two surveys, one with politicians and one with citizens. It was complemented by use case and goal analysis. The final requirements for the VoicE platform are formulated in the "End users' requirements report" [13].

2. The design process of the VoicE portal was an iterative process, influenced by the heuristic analysis performed by project partners and the empirical testing with pilot users. The results and the applied usability engineering process are documented in [14]. The VoicE usability engineering process intends the involvement of users in different stages of e-participation platform design and implementation. The iterative process consists of a requirements analysis phase, a design, testing and development phase and a deployment phase. Each step is accompanied by an evaluation against usability goals defined in the requirement analysis stage.

3. In a next iteration after the empirical testing phase, field observations with a more recent platform version resulted in a catalogue of critical points and weaknesses of the existing VoicE platform [15].

4. Results from the requirements analysis survey conducted in VoiceS (the follow-up project) have been taken into consideration to generalise and detail requirements and recommendations for e-participation. The interviews and questionnaires aimed to gather the VoiceS' requirements from citizens' point of view. The online questionnaire was filled out by 71 citizens from Spain, Germany and Austria.
Additionally 22 interviews were conducted in Germany and Spain to get more detailed answers from the target group [15].

<figure><img src="image_3.png" /><figcaption>Fig. 1. Overview of methodological steps in elaborating the hands-on guideline</figcaption></figure>

5. In order to fit participation in the VoiceS project to the European decision-making processes and thereof to have the best possible result of participation, a detailed analysis of processes and possible points of participation was conducted. The resulting process and workflow models were used to optimize the VoiceS
processes and platform features (for details see [15]). Usefulness of the process models was evaluated through a survey among the project partners [16].

6. A survey among visitors of the German VoicE platform was undertaken in July/August 2009 with the aim to analyse the usage of Web 2.0 features. The survey (filled out by 164 respondents) investigated how the contents and Web 2.0 features were accepted by the users and if the use of such features resulted in an added value to achieve the aims of the project (e-participation) [17].

7. From afore mentioned results, a list of functional and non-functional requirements was extracted. The results were complemented by a desk research in order to validate or discard the generality of the observed requirements and recommendations [18].

8. Analysis of the project implementation steps of VoicE and VoiceS was conducted in order to generalise a guideline for e-participation knowledge transfer [19].

9. The hands-on guideline for e-participation knowledge transfer was generalised and grounded on literature studies.

**Related work and relevant literature the hands-on guideline grounds in**

Kalampokis et al. present a model for the e-participation domain [20], which consists of the following layers: democratic processes, participation areas, participatory techniques, categories of tools and ICT technologies. The model is based on the e-participation framework put forward by Tambouris et al. [21]. The model for the e-participation domain summarizes the most important aspects and relations for e-participation in a conceptual model [20]. Thereby, the domain e-participation is divided into the three areas: actors, participation processes and information and communication tools (ICT tools). The areas are modeled separately and finally combined. The model put forward in [20] gives an overview of the complexity of the domain. It is not aimed at providing solutions how an e-participation initiative should proceed. This is where our approach digs into. The hands-on guideline conceptually builds on the aspects described in the domain model for e-participation and the underlying framework in its single steps.

Phang and Kankanhalli examine the suitability of various ICT tools for the achievement of e-participation objectives [8]. This work is based on Glass, who analysed offline participation techniques regarding the achievement of different objectives of citizen participation programs [22]. Phang and Kankanhalli transmit his results to ICT to technically support the participation techniques of Glass. A similar investigation was initiated in the DEMO-net project<fnref n="6" />, where different categories of tools (e.g. forum) were analyzed for their characteristics und usage for e-participation [23]. Based on the chosen participation processes<fnref n="7" /> according tools are selected. These results are considered in different steps of the hands-on guideline, too.

Phang and Kankanhalli also present a process for implementing e-participation initiatives, which consists of three steps: 1. Identification of the objective, 2. choosing the best participation techniques, and 3. choosing the electronic tools which support the participation techniques and thereof the achievement of the objectives. The hands-

<footnote n="6">⁶ Project title: Network of excellence for e-participation, see www.demo-net.org</footnote>

<footnote n="7">⁷ Phang and Kankanhalli call them participation techniques [8].</footnote>
on guideline model bases on these steps, but focuses more on the design of participation techniques and tools. Beyond that, further steps are described in our hands-on guideline. Islam presents a process model with key phases of e-participation [24]. This model describes a meta-view of different phases of participation. Our approach focuses specifically on the design of such participation processes.

The guideline presented in the next section aims at supporting both, bottom-up and top-down e-participation initiatives thereby focussing on

* the e-participation processes and functionalities to be supported,

* the actors' needs regarding sustainable use of the application, and

* key aspects related to content and the topics to be discussed on the platform.

## 3 Hands-on guideline for e-participation initiatives

The hands-on guideline is structured along four phases for implementing an e-participation project: designing the initiative, preparation of ICT used and information material, realisation and evaluation of the initiative. Each phase consists of different implementation steps. Fig. 2 shows the guideline in a six-step iterative process.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Hands-on guideline for e-participation initiatives: a six-step iterative process</figcaption></figure>
## Step 1: Initiation of the e-participation project

When an e-participation initiative is initiated, the first step is to agree upon the objectives. This is also seen by Glass as an “important element of any participatory program” ([22], p. 180) and supported by Phang and Kankanhalli [8]. Therefore it is essential to focus on the first two of the three critical design factors around every engagement process [25]: Why are you doing it and who should be involved?<fnref n="8" /> Consequently, the first guideline step aims on deciding the project objectives and (based on this decision) the stakeholders of the initiative. This step also includes the definition of project goals (expectations). In the following, the objectives of VoicE are described to give an example:

VoicE aims at enabling citizens to have a voice in EU legislation. In terms of content it focuses on the consumer protection policies. This fulfils the recommendation to choose an interesting and important topic [15], because consumer protection issues have an imminent effect on each citizen. VoicE is further based on the principles of a regional focus, i.e. making or keeping information as simple as possible, ensuring credibility, fostering personal opinions and making sure that the citizens’ participation has a value [26].

## Step 2: Design participation

Many e-participation projects failed and are still failing because participation processes are barely integrated into political processes (see e.g. [3, 22,]). Results of participation should at least be forwarded to the political decision makers [27, 28]. Everything else leads to disappointment and disinterest in political participation in the end. Therefore, the aim of step two is to plan the participation processes and not to design the technical features (as also proposed by Phang and Kankanhalli with “select best-matching participatory techniques” ([8], p.131)). It is crucial in this step to plan the participation processes and the involvement of elected representatives in detail. The participation processes need to be integrated in and adapted to political processes. In order to fit the participation processes to the legislative/political processes and thereof to have the best possible impact of participation, a detailed analysis of processes and possible points of participation needs to be conducted in advance. Fig. 3 shows a model on how to integrate participation processes into political processes. The involvement of elected representatives needs to be organised from the beginning of the initiative. It is also an important objective of this step to formulate the expectations of the participation process. In this step, the use of multiple channels for participation – on and offline – needs to be investigated, too (cf. [10]).

As a result from this step, the project team decides which of the e-participation areas<fnref n="9" /> will be supported to achieve the objectives of the initiative. Besides that, decisions will be made on how these areas can be implemented, i.e. which activities are performed. The participation processes should only be decided with the involvement of (all/key) stakeholders. For this, requirements analysis focussing on the

<footnote n="8">⁸ The third proposed factor (How to do it?) is decided in the subsequent steps of the guideline.</footnote>

<footnote n="9">⁹ E-participation areas proposed by the network of excellence for e-participation DEMO-net are introduced e.g. in [23].</footnote>
processes has to be conducted thoroughly. Finally, the goals of each activity have to be agreed with measurable values, e.g. the impact to be achieved.

In the beginning of the VoiceS project, a detailed analysis of processes and participation possibilities was conducted. We modelled the participation processes with the Business Process Modelling Notation (BPMN)<fnref n="10" />. The information and process models were useful and facilitated the analysis and design of political and participative processes [16].

<figure><img src="image_5.png" /><figcaption>Fig. 3. Integration of participation processes into political processes (detailing step 2)</figcaption></figure>

### Step 3: Design electronic participation

Based on the participation processes and user needs analysed in steps 1 and 2, the most appropriate media and channels have to be selected. If applicable, this includes as well deciding the choices of tools that are to be integrated into the platform to support the participation processes (cf. also [8]). The use of particular tools for e-participation cannot be recommended on a general level, because a) the usage should depend on the aims of the project and the processes established (see step 1, step 2 and [8]) and b) investigations did not unveil any preferred tools [14]. In terms of usability, the use of different participation features must be well-considered to not overload the users. Beyond that, participation features should only be provided in the case where the voice of participants is really heard by responsible authorities: An e-participation feature can only be used if the integration of the processes and results into the overall political process is ensured. It must be ensured that the users can see that their engagement will be recognized.

<footnote n="10">¹⁰ The BPMN specification is available online at www.bpmn.org/.</footnote>
The design of the electronic participation means (i.e. ICT) should involve the real users in order to analyse requirements and design user-friendly services [14]. Respondents of the survey conducted for the requirements analysis of the VoiceS project [15] stressed that “electronic tools usually cannot replace face-to-face discussions”. It is substantial for e-participation that communication by electronic means should not be more complicated than necessary. Hence, usability of e-participation platforms is of significant importance for the success of the project. E-participation features must base on easy-to-use tools in order to avoid usability flaws that could discourage people from online participation. Widely established tools and user paradigms ease the participation for users.

The usability of services is also of high relevance because usability evaluation plays a crucial role in e-participation evaluation methodologies [29, 30]. The usability and usefulness of the systems influence all e-participation evaluation perspectives, i.e. the socio-technical as well as the project and democratic perspectives. Small changes in the user interface of an e-participation application can result in completely different evaluation results. According to [31], bad usability in government web sites may even destroy the strategy of the whole website. This applies to e-participation platforms alike. Therefore systematic usability engineering is necessary – at least to detect minor design flaws that influence usability [32]. In this regard, the VoicE usability engineering lifecycle shaped up as usable to improve the system by iterative design of the systems' features, the interaction design and the user interface. In consequence, beyond the design of the platform, this step has to ensure usability of the e-participation means.

**Step 4: Prepare information and implement the platform**

The preparation of information should be organised in parallel with the setup of the platform functionalities and structure (implementation).

On the one hand, background information needs to be prepared. This task was based in VoicE on the selection of current European legislative proposals in the area of consumer protection and the review of content for the usage in the project [33]. The proposals should be easily accessible for consumers with little or no knowledge of European politics, so as to be a low-threshold starting point for participation. Furthermore, the legislative proposals should have a direct impact on citizens' everyday life. The careful preparation of information helped the project to be in time with the launch of the platform and to be able to easily update the contents for new developments. As a result from this experience, it is recommended to follow a well-structured process for information preparation in this step. This process should be related to the questions: What do you want to prepare, why and how do you want to prepare it? Another recommendation is to prepare questions for discussion in advance to stimulate discussions. This does not mean that all questions have to be posted at the outset, but can be thrown in at regular timeframes, e.g. one new question each week [33]. Such a preparation allows the fast reaction on developments as well as in case sparse discussions take place in the forum.

Besides the preparation of background information, it is of particular importance to prepare information about the participation process and the expectations to the participating parties. This information should answer the following questions:
* How does the participation process proceed (time frame, steps)?

* What can participants contribute to the process?

* What are the expectations and the goals of the participation process and how is it achieved?

This means that the users are informed about the results from step 1 and step 2 of the guideline in order to make the initiative transparent. When users are informed about how their contributions will be used, their willingness to participate may increase.

**Step 5: Extensive marketing (involving stakeholders) and maintenance**

Extensive marketing is an ongoing and important task. Yet, it needs to be seen in context with updating permanently the platform and publishing news (see maintenance). The marketing should already be initiated in advance and at least be active at the launch of the platform.

Maintaining the platform refers to activities necessary in order to keep the platform alive, to monitor the forum, to support the users, to publish news etc. The effort for doing so should not be underestimated and is an ongoing task through the whole project lifecycle. In order to support this step, the design phase of the electronic participation tools has to reflect the maintenance needs and has to plan the respective processes.

Maintenance should facilitate and support different instances with the same content in different languages in a multi-lingual context. Also the moderators and content administrators should be supported in summarising conclusions of discussions in simple and effective way.

**Step 6: Evaluation**

Millard recommends that e-participation initiatives should incorporate on-going evaluation [10]. Some evaluation frameworks are referenced in section 2. Evaluation of the e-participation platform, the processes and the actors' participation shall give insight into whether the goals of the e-participation initiatives are met and impact is reached with the electronic participation. This includes the assessment of the different perspectives (project, socio-technical and democratic) as well as the evaluation against the project goals formulated in earlier steps (see steps 1, 2, and 3). The evaluation results show whether the e-participation initiative is successful. Critical points, which need to be revised and improved in an iterative design cycle, are identified.

As indicated in Fig. 2, the guideline introduced in this section describes an iterative process to successfully plan and implement e-participation initiatives. Hence, the insights gained in the evaluation (step 6) feed back to revisions in earlier steps of the hands-on guideline.
## 4 Concluding remarks

The guideline presented in this contribution sums up lessons and experiences from two e-participation projects: VoicE and VoiceS. The lessons and experiences from the projects are grounded and counterbalanced with insights from studying existing scientific works. The hands-on guideline helps projects and organisations to build up successful e-participation initiatives based on an iterative process. Case studies are necessary with projects in other e-participation areas in order to test the generality of the guideline.

Even though the guideline covers the whole e-participation project life-cycle, it may require further effort to give more specific guidance in certain respects to support e-participation decision makers, for example:

* Currently, no specific evaluation framework is recommended. Different evaluation strategies might be reviewed and updated in order to align them with the evaluation of goals as emphasized in steps 1-3 of the hands-on guideline. Subsequently, the guideline may require updates in order to streamline the goals definition (steps 1-3) with the evaluation (step 6).

* E-participation processes have so far not been extensively modelled and standardised. There is a lack of reference models for process patterns and process chains describing common processes in e-participation [34]. Further research is necessary to identify and model reference participation processes that support different e-participation areas. Such reference processes would be usable in step 2, where projects could easily choose and adapt reference processes for their objectives and needs.

* A toolbox of technical building blocks to support e-participation, and a reference architecture to combine these blocks efficiently, are needed. Such a toolbox should support the use of standardised reference processes (step 3) and the implementation step (step 4a). Requirements such as interoperability of services [34] need to be considered.

* Traditional project management is often used for software development, even though a variety of projects are suffering under this type of management. Implementation projects in e-participation often have to cope with constantly changing demands of citizens, governments and politicians, and they have to counterbalance technical and non-technical factors of electronic participation. Agile processes promise to integrate rapidly changing requirements and prioritizing better in the process (see e.g. [35]). The application of agile project management in e-participation with public administrations should be investigated. If agile methods shape up as useful, the guideline needs to be adapted.

Fine-tuning in the above aspects could improve the hands-on guideline and make it a key reference model for e-participation implementation. Subsequent research will focus on the development of the reference model.

The hands-on guideline presented in this contribution is a useful framework to support the effective and efficient implementation of e-participation projects.
Acknowledgement

VoicE and VoiceS are co-funded by the European Commission under the eParticipation Preparatory Action. This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use, which may be made of the information contained therein.

References

1. Harth, T.: Internet und Demokratie - neue Wege politischer Partizipation: Überblick, Potential, Perspektiven. In Woyke, W., ed.: Internet und Demokratie. uni studien politik. Wochenschau Verlag, Schwalbach/Ts. (1999) 8–24

2. Maier, J.: Web 2.0 - Moderatorenrechte für alle? Gibt es eine E-Partizipation 2.0 im Web 2.0? In: E-Partizipation: Beteiligungsprojekte im Internet. Number 21 in Beiträge zur Demokratieentwicklung. Stiftung Mitarbeit, Bonn (2007) 282–296

3. Albrecht, S., Kohlrausch, N., Kubicek, H., Lippa, B., Märker, O., Trénel, M., Vorwerk, V., Westholm, H., Wiedwald, C.: eParticipation - Electronic Participation of Citizens and the Business Community in eGovernment. Study on Behalf of the Federal Ministry of the Interior (Germany), Division IT 1 (1 2008)

4. Holzner, M., Schneider, C.: Consumer Protection, European Decision-Making and the Regions - the eParticipation Project VoicE. In Cunningham, P., Cunningham, M., eds.: Collaboration and the Knowledge Economy: Issues, Applications, Case Studies Part 1, IOS Press (2008) 351–356

5. Scherer, S., Holzner, M., Karamagioli, E., Lorenz, M., Schepers, J., Wimmer, M.A.: Integrating Semantics, Social Software and Serious Games into eParticipation: The VoiceS Project. In: Macintosh, Tambouris (eds.). Electronic Participation: Proceedings of ongoing research and projects of ePart 09, Schriftenreihe Informatik 31, Trauner Druck: Linz (2009) 151–158

6. Macintosh, A. (ed.): The Initial DEMO-net Landscape. Deliverable D4.1, DEMO-net Consortium (5 2006)

7. Frewer, L.J., Rowe, G.: Evaluating public participation exercises: Strategic and practical issues. In: Evaluating Public Participation in Policy Making. OECD (2005) 85

8. Phang, C.W., Kankanhalli, A.: A framework of ICT exploitation for e-participation initiatives. Communications of the ACM 51 (12) (12 2008) 128–132

9. Whyte, A., Macintosh, A.: Analysis and Evaluation of e-consultations. e-Service Journal 2 (1 "e-democracy in Practice") (2003)

10. Millard, J.: eParticipation recommendations - focusing on the European level. Deliverable 5.1. Study and supply services on the development of eParticipation in the EU (July 2009)

11. Committee of Ministers: Recommendation CM/Rec(2009)1 of the Committee of Ministers to member states on electronic democracy (e-democracy). Recommendation, Committee of Ministers (February 2009)

12. Bicking, M., Wimmer, M.A.: Evaluation framework to assess eParticipation projects in Europe. In Tambouris, E., Macintosh, A., eds.: Electronic Participation: Proceedings of Ongoing Research, General Development Issues and Projects of ePart 2009, Schriftenreihe Informatik # 31, Trauner Druck: Linz (9 2009) 73–82

13. Karamagioli, E., Titorencu, M.: Deliverable D2.1. End-users' requirements report. VoicE Consortium (5 2008)
14.Scherer, S., Karamagioli, E., Titorencu, M., Schepers, J., Wimmer, M.A., Koulolias, V.: Usability Engineering in eParticipation. European Journal of ePractice (7) (2009)

15.Scherer, S., Wimmer, M.A., eds.: VoiceS D 2.1 Requirements analysis report: Specification of (user) requirements for the VoiceS platform with focus on process models and user roles. Deliverable, VoiceS Consortium (5 2009), pp. 78 - 91

16.Scherer, S., Wimmer, M.A., Ventzke, S.: Modellierung von Prozessen für E-Partizipation in BPMN. In Fischer, S., Maehle, E., Reischuk, R., eds.: Informatik 2009, Im Focus das Leben, LNI # 154, Köllen Druck+Verlag GmbH: Bonn (9 2009) 205, 1804–1813

17.Augustin, A.: Bedeutung und Gegenstand von Web 2.0 Technologien im Rahmen von bestehenden E-Partizipations-Projekten der Europäischen Kommission und besondere Analyse dieser Technologien im Bezug auf das Projekt VoicE/VoiceS. Project report, University of Koblenz-Landau (8 2009)

18.Scherer, S., Wimmer, M.A., Ventzke, S.: Requirements and recommendations for eParticipation applications. In Prosser, A., Parycek, P., eds.: EDEM 2009 - Conference on Electronic Democracy 2009, books@ocg vol 251, Druckerei Riegelnik: Wien (9 2009) 187–197

19.Scherer, S., Wimmer, M.A., Ventzke, S.: VoicE D 6.4 Guideline for e-participation knowledge transfer. Deliverable, VoicE Consortium (10 2009)

20.Kalampokis, E., Tambouris, E., Tarabanis, K.A.: A Domain Model for eParticipation. In Mellouk, A., Bi, J., Ortiz, G., Chiu, D.K.W., Popescu, M., eds.: ICIW, IEEE Computer Society (2008) 25–30

21.Tambouris, E., Liotas, N., Kaliviotis, D., Tarabanis, K.: A framework for scoping eParticipation. In: dg.o ’07: Proceedings of the 8th annual international conference on Digital government research, Digital Government Society of North America (2007) 288–289

22.Glass, J.J.: Citizen Participation in Planning: The Relationship Between Objectives and Techniques. Journal of the American Planning Association 45(2) (4 1979) 180 – 189

23.Thorleifsdottir, A., Wimmer, M.A., eds.: Deliverable 5.1: Report on current ICTs to enable Participation. DEMO-net Consortium (4 2006)

24.Islam, M.S.: Towards a sustainable e-Participation implementation model. European Journal of ePractice (5) (10 2008)

25.Acland, A., ed.: Dialogue by Design: A Handbook of Public & Stakeholder Engagement. Dialogue by Design (3 2008)

26.Günes, L., Barahona, C.M., Setzen, F.: VoicE D4.1. Strategy for Marketing and Awarness- rasing Çampaign. Deliverable, VoicE Consortium (7 2008)

27.Coleman, S.: A Tale of Two Houses: the House of Commons, the Big Brother house and the people at home. Parliamentary Affairs 56(4) (2003) 733–758

28.Coleman, S., Gøtze, J.: Bowling Together: Online Public Engagement in Policy Deliberation. Report (2002)

29.Lippa, B., Aichholzer, G., Allhutter, D., Freschi, A.C., Macintosh, A., Westholm, H.: Deliverable D 13.3: eParticipation Evaluation and Impact. DEMO-net Consortium (2008)

30.Macintosh, A., Whyte, A.: Towards an evaluation framework for eParticipation. Transforming Government: People, Process and Policy 2(1) (2008) 16 – 30

31.Esteves, J.: A Semiotic Analysis of Spanish Local e-Government Websites. In: Proceedings of the 7th European Conference on E-Government 2007: ECEG, Academic Conferences Ltd (2007)

32.Nielsen, J.: Usability Engineering. Acad. Press, Boston, Mass. (1993)

33.S. Fink and M. Holzner, eds: VoicE D 3.1 Editorial Report. Deliverable, VoicE Consortium (5 2008)
34.Scherer, S., Liotas, N., Wimmer, M.A., Tambouris, E., Tarabanis, K.: Interoperability Requirements, Recommendations and Standards in E-Participation. In Charalabidis, Y., ed.: Interoperability in Digital Public Services and Administration: Bridging E-Government and E-Business. Chapter 6. IGI Global (5 2010), to appear

35.Chin, G.: Agile project management: how to succeed in the face of changing project requirements. McGraw-Hill Professional (2004)

36.Macintosh, A.: Characterizing e-participation in policy-making. In: Proceedings of the 37th Annual Hawaii International Conference on System Sciences, (HICSS-37), Big Island, Hawaii, 5th-8th January [online]. Volume 5. IEEE Computer Society, Los Alamitos, CA, USA (1 2004)