metadata: authors: 0: author: Spada,Paolo
1: author: Mellon,Jonathan
2: author: Peixoto,Tiago Carneiro
3: author: Sjoberg,Fredrik Matias
count: Brazil
docna: 0: docna: Effects of the internet on participation :
            study of a public policy referendum in Brazil
docty: Policy Research Working Paper
subtopic: Social Analysis,ICT Applications,Public Sector Economics,Educational Sciences,Gender and Development,Public Financial Management,Public Finance Decentralization and
            Poverty Reduction
teratopic: Governance,Industry,Information and Communication Technologies
abstracts: cdata!: Does online voting mobilize citizens who
            otherwise would not participate? During the annual
            participatory budgeting vote in the southern state of Rio
            Grande do Sul in Brazil -- the world's largest --
            Internet voters were asked whether they would have
            participated had there not been an online voting option
            (i-voting). The study documents an 8.2 percent increase in
            total turn-out with the introduction of i-voting. In support
            of the mobilization hypothesis, unique survey data show that
            i-voting is mainly used by new participants rather than just
            for convenience by those who were already mobilized. The
            study also finds that age, gender, income, education, and
            social media usage are significant predictors of being
            online-only voters. Technology appears more likely to engage
            people who are younger, male, of higher income and
            educational attainment, and more frequent social media users.
geo_reg_key: 517191,119228
repnme: repnme: Effects of the internet on participation :
            study of a public policy referendum in Brazil
display_title: Effects of the internet on
            participation : study of a public policy referendum in Brazil
listing_relative_url: /research/2015/02/24060569/effects-internet-participation-study-public-policy-referendum-brazil
topicv3: Budget Preparation and Management
            Techniques m1326766 2060,Education m1327048 470,Social Media m1327099 662,Political Participation m1327389 926,Gender m1327621 615
publishtoextweb_dt: 2015-02-26T00:00:00Z
historic_topic: Governance,Industry,Information and Communication Technologies
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
no_of_pages: 38
author_keys: 0: author_key: 1218844
1: author_key: 1099489
2: author_key: 1277589
model_extractions: WPS7204 Policy Research Working Paper 7204 Effects of the Internet on Participation Study of a Public Policy Referendum in Brazil _Paolo Spada Jonathan Mellon Tiago Peixoto Fredrik M Sjoberg_ Governance Global Practice Group February 2015
Policy Research Working Paper 7204

# **Abstract**

Does online voting mobilize citizens who otherwise would not participate? During the annual participatory budgeting vote in the southern state of Rio Grande do Sul in Brazil—the world’s largest—Internet voters were asked whether they would have participated had there not been an online voting option (i-voting). The study documents an 8.2 percent increase in total turn-out with the introduction of i-voting. In support of the mobilization hypothesis, unique survey data show that i-voting is - mainly used by new participants rather than just for con venience by those who were already mobilized. The study also finds that age, gender, income, education, and social media usage are significant predictors of being online-only voters. Technology appears more likely to engage people who are younger, male, of higher income and educational attainment, and more frequent social media users.

This paper is a product of the Governance Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at tpeixoto@ worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Effects of the Internet on Participation: Study of a Public Policy Referendum in Brazil

Paolo Spada, University of British Columbia* Jonathan Mellon, Oxford University Tiago Peixoto, World Bank Fredrik M Sjoberg, Data-Pop Alliance February 2015 Words: 7,368 Abstract Does online voting mobilize citizens who otherwise would not participate? During the annual participatory budgeting vote in the southern state of Rio Grande do Sul in Brazil – the world’s largest – Internet voters were asked whether they would have participated had there not been an online voting option (i-voting). The study documents an 8.2 percent increase in total turnout with the introduction of i-voting. In support of the mobilization hypothesis, unique survey data show that i-voting is mainly used by new participants rather than just for convenience by those who were already mobilized. The study also finds that age, gender, income, education, and social media usage are significant predictors of being online-only voters. Technology appears more likely to engage people who are younger, male, of higher income and educational attainment, and more frequent social media users.

Classification (JEL): P16, D72, H41, and H7. Keywords: Participation, Voting, Participatory Budgeting, Internet, and Digital Divide.

*Note that author order is determined using bounded randomization. All authors contributed in equal shares.

Acknowledgements: we would like to thank Hollie Russon Gilman and Quinton Mayne for help with the survey instrument and implementation. Thank you to David Schmidt, Paulo Coelho, and Ricardo Almeida at Rio Grande do Sul’s Secretariat of Planning for facilitating this work. We would also like to thank Ricardo Fritsch, Carlson Aquistapasse, and Guilherme Donato at PROCERGS for providing the data on the PB vote. We are also thankful for comments from Deborah Wetzel (World Bank), Graham Smith (University of Westminster), Hélène Landemore (Princeton), Jonathan Fox (American University) and Roland Clarke (World Bank). Finally, thanks to Amy Chamberlain for editorial assistance. This work has been funded by the Governance Partnership Facility and led by the World Bank’s Digital Engagement Evaluation Team (DEET).
# 1. Introduction

In the last two decades, attempts to apply the Internet to the act of voting have multiplied. Governments have experimented with Internet voting (i-voting)<sup>1</sup> both in local and national elections (Alvarez, Hall, and Trechsel 2009; Goodman, Pammett, and DeBardeleben 2010; Mendez 2013). Parties have introduced i-voting to select candidates and conduct internal referendums (Done 2002; Lanzone and Rombi 2014). In a similar vein, a variety of participatory governance processes have introduced i-voting to increase citizen engagement, ranging from participatory budgeting in Brazil to referenda in Switzerland (Peixoto 2009; Sampaio, Maia, and Marques 2011; Nitzsche, Pistoia, and Elsäßer 2012; Stortone and De Cindio 2014; Mendez 2013).

While the number of applications is steadily increasing and the literature on the subject is burgeoning, the empirical evidence on the effects of i-voting is still limited. Major questions remain open (Carter and Bélanger 2012; Pammett and Goodman 2013). Does i-voting increase turnout? Are there citizens willing to vote only via the Internet? If so, what are the socioeconomic characteristics of this group? What are the effects of Internet voting on inclusiveness and diversity? Or does introducing i-voting distract people from traditional forms of participation?

The objective of this paper is to contribute to answering these questions by providing evidence on the effects of the Internet voting on participation. But apart from its empirical character, this contribution is also relevant for two other reasons. First, it assesses the effect of i- voting in an understudied field of i-voting, that is, in participatory governance (i.e., nonelectoral) processes. Second, while the majority of i-voting studies have focused on the US and Europe, this study looks at an experience from a middle-income country, Brazil.

We present the results of a unique survey of over 22,000 Internet voters from the southern Brazilian state of Rio Grande do Sul during a referendum on state-level spending priorities.<sup>2</sup> This referendum is part of a large multi-channel democratic innovation that simultaneously combines both online and offline voting. This process, entitled State System for Citizen Participation in the Budget Process<sup>3</sup> , allows citizens to influence the formulation of projects for the allocation of public spending and to select which of the projects will be implemented via a

> 1 Oostveen and Besselaar (Oostven and Besselaar 2004, 2) define i-voting as an election system that uses encryption to allow a voter to transmit his or her secure and secret ballot over the Internet.

> 2 This survey was the fruit of a collaborative effort involving the World Bank, the Secretariat of Planning of Rio Grande do Sul Government, and the state’s data processing company PROCERGS.

> 3 Translation from original Portuguese “ciclo orçamentário do Sistema Estadual de Participação Popular e Cidadã.”

2
referendum. It is a form of participatory budgeting (PB) applied at the state level (henceforth State PB).<sup>4</sup> Here we analyze the last stage of the State PB, that is, the vote in the final referendum, during which the population prioritizes the projects to be funded.

More specifically, we investigate the traits and attitudes of those that cast their ballot via Internet using a post-vote survey. The survey was implemented as an online exit poll at the time of the vote in early July 2012 and consists of 27 questions.<sup>5</sup> The questions aimed to identify the socio-economic characteristics of the respondents, their media and ICT usage habits, as well as their previous level of engagement with politics and civil society.

This study focuses on three key research questions.

- 1) Does the possibility of online voting increase turnout by attracting a group of ‘onlineonly voters’?

- 2) If so, what is the socioeconomic profile of this group? More specifically, are they predominantly young, educated, rich, males that are familiar with the Internet - as we would expect from the literature on digital divide?

- 3) What is the level of pre-existing engagement of these voters? Are online-only voters already engaged in the public sphere, or does the new venue of participation attract previously disengaged portions of society?

Anticipating the key results of the analysis, nearly two-thirds of respondents (63.1 percent) say that they would not have taken part in the vote if i-voting was not available.<sup>6</sup> This evidence supports the perspective of i-voting as a valid means to increase turnout and the existence of a specific group of individuals who would not have participated otherwise. With regard to the second question, employing simple logistic regressions, this study finds that young, educated and wealthier males that are active social networks users are more likely to identify themselves as online-only voters.

> 4 There are many definitions of what constitutes a participatory budgeting process. At an abstract level participatory budgeting is a democratic innovation that allows citizens to affect the formulation of a budget. Most participatory budgeting processes occur at the city level. They are based on repeated negotiations between the city government and the participants, combining elements of deliberative, participatory, and representative democracy. In order to give a more precise operational definition of this democratic innovation Sintomer et al. (Sintomer et al. 2013) include five additional criteria that distinguish participatory budgeting from other similar programs: (1) the financial and/or budgetary dimension must be discussed; (2) the city level must be involved, or a (decentralized) district with an elected body and some power over administration; (3) it has to be a repeated process (one meeting or one referendum on financial issues are not examples of participatory budgeting); (4) the program must include some form of public deliberation within the framework of specific meetings/forums (the opening of administrative meetings or classical representative instances to ‘‘normal’’ citizens are not participatory budgeting); (5) some accountability on the output is required.

> 5 See appendix 3 for full list of survey questions.

> 6 We use the same set of respondents that we use in these models to generate these summary statistics. These exclude missing values.

3
With regard to the third research question, i-voting seems to mobilize rather unusual suspects, as the majority of i-voters were previously politically inactive. Approximately nine out of ten i-voters surveyed had not participated in any existing public discussion or assembly over the state budget, and 55.7 percent of participants also declare not having been active in a variety of common civic engagement activities.<sup>7</sup> The majority of participants also claim not to have participated in any meetings of the most common civil-society organizations in the previous 12 months.<sup>8</sup> Overall, our study shows that the introduction of i-voting in the participatory engagement process does not lead to a substitution effect. For the most part, those who participate offline continue to do so despite the introduction of i-voting. I-voting increases participation among previously non-engaged strata of the population, promoting the inclusiveness of the process as a whole. However, these new participants – the online-only voters – are likely to be socioeconomically more privileged.

The rest of this paper proceeds as follows. Section 2 briefly presents the key research questions of the literature on i-voting and identifies testable hypotheses. Section 3 describes the introduction of i-voting in Brazilian participatory engagement processes and introduces the State PB. Section 4 presents the results of our empirical analysis. Section 5 discusses these findings in light of the literature on convenience voting, participatory governance and collective intelligence; concluding with implications for future practice and research.

# 2. Extant Theory

Before delving into the relevant literature, it is important to introduce some terminology. Internet voting mechanisms are a subset of electronic voting mechanisms (e-voting), where e- voting refers to all forms of voting that uses electronic means to count or cast the votes (e.g. voting machines). Given that there are voting machines that use the Internet to transmit the votes to a central repository, not all Internet voting mechanisms are remote. The class of remote voting mechanisms also includes non-electronic means of transmission (e.g., mail), and mechanisms that use dedicated technology (e.g., special TV sets that allow voting). To simplify the terminology, from now on we will use the term i-voting to refer to remote Internet voting

> 7 See questions 21 and 23 in appendix 3 for a detailed list of the engagement activities. 8 See Figure 1 in the appendix.

4
performed using a non-dedicated device (e.g. a laptop or a smartphone).<sup>9</sup> A clear distinction between i-voting mechanisms and e-petitions and e-surveys that share similar technologies does not exist in the literature. In this essay, we use the provisional distinction that i-voting produces a binding decision on the selection of candidates or the implementation of public policies, as in the case we study.

While a detailed overview of the vast literature on the variety of Internet voting mechanisms and their impact is beyond the scope of this paper, for our purposes it is important to highlight the two key issues of turnout and inclusiveness. The debate on the benefits of remote voting in increasing voter turnout started with postal voting (Moreton Jr 1985; Kousser and Mullin 2007; Stein and Vonnahme 2011; Mann and Mayhew 2012) and has seen a series of iterations that have followed the evolution of information and communication technologies (Trechsel et al. 2007; Alvarez, Hall, and Trechsel 2009; Bochsler 2010; Carter and Bélanger 2012). Similar to all other forms of convenience voting, the notion of Internet voting as a means to increase participation follows a rational choice approach, assuming that the act of voting implies both costs and benefits<sup>10</sup> (Downs 1957; Riker and Ordeshook 1968; Aldrich 1993). Thus, the relationship between the convenience of voting and participation is established in the following manner: holding all other factors constant, the probability of participation will be negatively correlated with the costs of participation. Therefore, given the growing access to the Internet in both developed and developing countries, i-voting scholars have often highlighted the Internet’s potential to increase turnout by reducing its costs (Alvarez and Hall 2004; Trechsel et al. 2007; Carter and Bélanger 2012).

The concept of inclusiveness in voting processes refers to the diversity of participants and the extent to which groups traditionally excluded from a process may be effectively included (Roberson 2006). In this respect, optimistic views support a ‘mobilization hypothesis’, which states that the introduction of online modes of participation will increase the participation of citizens previously marginalized in participatory processes (Norris 2001). While there is significant reason to believe that Internet voting has the potential to boost turnout and inclusiveness, the empirical literature finds mixed results. The effects of i-voting appear to be elu-

> 9 The definition matches Alvarez and Hall’s definition of ‘remote Internet voting’, as contrasted with Kiosk Internet voting, Polling Place Internet voting, and Precinct Internet voting (Alvarez and Hall 2004).

> 10 These costs may be manifested either materially or immaterially, such as transport costs and time spent voting. Similarly the benefits might be expressive and immaterial (general elections), or might be concrete and measurable as in the case of the allocation of public goods in participatory budgeting.

5
sive, with some cases presenting minor results and others suggesting no effect whatsoever (Vassil and Weber 2011; Pammett and Goodman 2013).<sup>11</sup> In fact, at odds with the mobilization thesis, much of the digital divide literature suggests that unequal access to the Internet will disproportionately increase the representation of economically advantaged groups who are already politically active (Schlozman, Verba, and Brady 2010; Bélanger and Carter 2011; Brandtzæg, Heim, and Karahasanović 2011; Alvarez and Nagler 2000). Often referred to as the ‘reinforcement thesis’, such studies posit that people with the resources and motivation to participate, who are usually the better off, will be further empowered by the introduction of online modes of participation (Norris 2001; Davis 1998).

It is a well-established fact that Internet access is correlated with income and education (Hilbert 2010). While initial analyses of the digital divide focused on Internet penetration (Compaine 2001), modern investigations in developed countries where Internet penetration has reached high percentages of the population center on citizens’ capacity to effectively use the Internet (Hsieh, Rai, and Keil 2008; Brandtzæg, Heim, and Karahasanović 2011; Sarkar et al. 2011).

Overall, the trajectory of this literature has evolved from explanations based on income and education, to more complex explanations that consider a variety of factors that affect the usage of ICT technologies, such as cognitive and social skills (Deursen, Helsper, and Eynon 2014; Ferrari 2012; Helsper and Eynon 2013). How applicable these studies are to the specific literature on i-voting is yet to be fully explored (Carter and Bélanger 2012). Furthermore, while most of the literature has centered on i-voting in general elections, less is known about its impact in other processes such as referenda and participatory budgeting, where turnout levels are often considerably lower.

Before moving to the next question, it is useful to highlight an important scope condition of our study. The critiques of the introduction of i-voting mechanisms are not limited to the concept of inclusiveness. There are also important debates regarding the quality of online engagement and the security of i-voting. Many refer to online participation as a form of “slacktivism,” referring to an activity that, while psychologically rewarding, has little impact on political decisions (Hindman 2009; Morozov 2012; Smith 2013). Furthermore, critics argue that slacktivism may substitute more meaningful forms of engagement, once an individual’s urge to take action is fulfilled by low-cost online participation (Shulman 2009; Lee and Hsieh

> 11 It is important to notice that, while in the majority of cases i-voting is offered in conjunction with traditional forms of voting, in a limited number of cases Internet voting is the only channel of participation available (See, for instance, (Peixoto 2009).

6
2013). Yet, the evidence on substitution effects remains mixed at best (Christensen 2011; Lee and Hsieh 2013), with a growing number of studies suggesting the opposite, that is, online participation may function as a gateway to further types of engagement (Hu 2014; Vie 2014; Vitak et al. 2011; Vissers and Stolle 2014; Breuer and Farooq 2012).

It is important to note that the idea that some forms of engagement are more desirable than others is not new, nor exclusive to the literature on online participation. The literature on participatory democracy provides numerous examples of participatory engagement processes that are described as tokenistic (Wampler 2007; Pateman 2012). The same literature offers a variety of classificatory schemes to rank engagement processes on the basis of their ability to provide more meaningful forms of participation (Arnstein 1969; Connor 1988; Collins and Ison 2006).

Our aim is limited to evaluating whether the online venue of participation attracts new citizens who would not have participated otherwise and, if so, to investigate the specific characteristics of this group of new voters. This is a first important step towards understanding whether there are substitution effects – i.e. online participation reduces offline participation – or if there are positive feedback effects and the two venues reinforce each other. Investigating the quality of the State PB as a democratic innovation is beyond the scope of our paper.<sup>12</sup> A number of authors also raise questions about the security of i-voting systems (Wolchok et al. 2012; Bélanger and Carter 2011). A consideration of the broader debate on the security of i-voting is also beyond the scope of this paper. In the specific case of the State PB, surprisingly, it appears that the online vote may be more secure than the face-to-face process, due to the way these two mechanisms are organized. The former is centralized and managed by a unit of the state government that has no stake in the results; the latter is decentralized and managed by volunteers and community leaders who do have a stake in the results of the vote. We will briefly return to this topic in the section describing the voting procedure.

In the next sections, we will first introduce briefly the specifics of the Brazilian case, and then we explain the procedure of i-voting in Rio Grande do Sul’s State PB in more detail.

# 3. ICT and Participatory Governance Innovations in Brazil

Brazil is an international leader in participatory governance innovations (Fung 2011). The Federal Constitution of 1988 mandated the introduction of public policy management coun-

> 12 A current evaluation by the World Bank’s Digital Engagement Evaluation Team (DEET) looks at this specific issue.

7
cils, thematic councils, and management councils (Coelho 2004; Moreira and Escorel 2009), and introduced an array of democratic decision-making innovations (e.g.; referendums, ballot initiatives, civil actions, national citizens conferences). Municipal participatory budgeting (PB) is probably the most famous of these institutions (Wampler 2007; Avritzer 2009; Heller, Baiocchi, and Silva 2011). The process, initially adopted by 13 cities in Brazil in 1989, has now spread to more than a 1,400 cities worldwide (Sintomer et al. 2013). Recent research has shown that PB processes have direct impacts on reducing mortality rates, promoting health care spending and the creation of civil society organizations (Touchton and Wampler 2013; Gonçalves 2014).

Participatory budgeting was also scaled up to the state level<sup>13</sup> , including the case we explore in this paper, which is the world’s largest process both in number of participants and geographic coverage. The first instance of a state level PB process occurred in the state of Rio Grande do Sul (RS) from 1999 to 2002. The experience was continued after the Workers Party’s (PT) loss of the governorship in 2003, with modifications. The new process included Internet voting for the first time and was called Consulta Popular (Goldfrank and Schneider 2006). After PT’s victory in 2010 the process was changed again to its current form, as we shall describe later.

In Brazil, the first experiments in hybridization – including both offline and online venues of participation – began in the late 1990s and the early years of the new century when a few cities adopted ICT tools to present proposals to voters or conduct voting in municipal PB processes (e.g., Porto Alegre, Ipatinga). The cities of Belo Horizonte and Recife implemented some of the earliest hybrid programs. The former uses separate online and offline PB processes that decide on the allocation of two different, specific budgets, and the latter integrates online voting into the face-to-face mechanisms (Best et al. 2010).

A number of cities around the world have adopted the online PB model of Belo Horizonte, without introducing a parallel offline process. This new form of PB is often referred to as e-PB. This process is particularly common in Europe (Nitzsche, Pistoia, and Elsäßer 2012). Interestingly, in many of these cities the use of ICT was introduced with the more or less explicit objective of attracting younger and more middle class participants. Yet, to date, studies on the use of ICT in PB have been inconclusive in terms of youth and middle class inclusion (Peixoto 2009; Sampaio, Maia, and Marques 2011). In the case of Rio Grande do Sul’s State PB the use of ICT is limited to the final vote of the process, where citizens can choose to cast their vote

> 13 Participatory budgeting processes beyond the municipal level include the cases of the Poitou-Charentes region in France and South-Kivu Province in the Democratic Republic of Congo.

8
either offline at ballot stations or via the Internet. The next section describes the process and the voting system.

# 3.1 Multi-channel Participatory Governance in the State of Rio Grande do Sul

After the Workers’ Party won the state gubernatorial elections in 2010, it introduced a comprehensive reform of the state level engagement processes. This reform – Sistema Estadual de Participação Popular e Cidadã (Sistema) – is incremental in nature, where the articulation among the different participatory institutions in the state is expected to happen sequentially. With support from the World Bank, Sistema reorganizes and expands a pre-existing array of participatory governance mechanisms. While it is too early to understand whether this reform will achieve the ambitious goals of better synergy between participatory, deliberative and representative institutions, Sistema has unique characteristics and represents one of the most ambitious projects of state level multi-channel participatory governance in the world (Oliveira and Karnopp 2013; Goldfrank 2013).

One component of this system – the State PB – is a participatory governance mechanism that allows citizens and CSOs to influence the allocation of a portion of the state investment budget. This mechanism is an update of the pre-existing Consulta Popular introduced in 2003 (Borowski 2012; Goldfrank 2013). There are three changes. First, the new procedure is embedded in the entire system of participatory governance, which includes the participatory multi-year plan, a process that defines the state government’s major objectives every four years.

Second, the process has a more complex ballot than the previous Consulta Popular, as described in the next section. Third, the ability of CSO representatives to oversee the implementation of projects has been strengthened. In its current version, the government provides monthly project status reports to the participatory assemblies that manage the process. The detailed mechanisms of the five stages of the State PB are described in the appendix<sup>14</sup> . In the next section we will focus on the voting process, the fourth stage.

## 3.2 The Voting System: Ballot Structure, Security Issues and Turnout

> 14 See appendix 4.

9
In the State PB the ballot is divided into three sections. In the first field, the voter can select up to four regional projects from a list of 10-20 projects.<sup>15</sup> In the second field, the voter can select two regional priorities from five possibilities. The voter is also asked to express his/her opinion on a few key policy issues, which constitutes the third field. These three fields on the ballot (cédula) are defined through a series of state, regional and municipal deliberative meetings that involve organized civil society, common citizens and representatives of the state government.<sup>16</sup> While the first field varies from region to region, the other two are identical statewide. The latter two fields are an innovation introduced in 2011 by the State PB.

Votes can be cast either in-person or remotely online. The online and the offline ballots are identical. In both processes, the voter is identified by his/her unique federal voter registration number, where each number can only be used once. Voters are not allowed to change their vote. The votes are centrally stored and tallied by the Center for Electronic Government Solutions (PROCERGS), a state government unit specifically created to manage all electronic government infrastructure and processes. The i-voting is monitored by the Regional Councils for Development (COREDES), which are composed of CSOs from their respective regions.<sup>17</sup> The in-person voting process is organized by coordinators – paid by the state – and by the delegates, that is, representatives of the participants that are selected through municipal assemblies.<sup>18</sup> The delegates do not receive compensation. On average, there are three to five polling officials at each location. The bylaws of the process specify that if there is a discrepancy of two percent between the votes in the ballot boxes and the signatures collected, the particular ballot box is invalidated.

Overall, turnout in the State PB vote is around 15 percent (see Table 1) of eligible voters, reaching over one million participants on a yearly basis, making the Rio Grande do Sul PB the largest such process in the world in terms of number of participants. Of these voters, the majority still vote in person, around 86-88 percent.

Table 1. Voters in the PB process, by type.

|Item|2011|2012|
|---|---|---|
|In person (paper)|998,145|907,146|
|Remotely (total, n)|114,571|124,211|
|Remotely (turnout, %)|10.2%|12.0%|
|Web|Na|99.3%|

> 15 As an illustrative example of a ballot, we attach a ballot from 2013 in appendix 1.

> 16 For a detailed description of the participatory process that defines the ballot see the appendix.

> 17  The COREDES were introduced in 1994 as a venue to allow civil society organizations to influence development plans for each of the 28 regions of the state. See appendix 4. 18 See appendix 4.

10
|Mobile|Na|0.7%|
|---|---|---|
|SMS|Na|0.008%|
|Vote turnout (%)|15.1%|14.0%|

* Notes: There were 7,348,996 registered voters in 2012 (data from PROCERGS). Turnout in 2011 is calculated using the 2012 data on the number of voters.

Concerns about the integrity of the voting process are rarely expressed. However, nothing prevents a participant from voting both online and offline, since the online and offline voter lists are not compared at any point in the process, nor is there any protection against voter coercion in the remote voting.<sup>19</sup> It is important to note, however, that among online engagement processes around the world, only a few adopt a more advanced security system that allows the voter to change the vote to discourage improper influence and use stronger authentication procedures (e.g. Estonia, Switzerland). Notably, stronger security e-voting measures are limited in their overwhelming majority to electoral processes. Furthermore, as the literature on election fraud has shown, electoral integrity continues to be a problem even in offline systems (Hyde 2011; Sjoberg 2012).

# 4. Methodology & Data

One approach to investigate the issue of turnout and the effect of digital divide on inclusiveness is to observe changes in aggregate-level turnout when i-voting is introduced. However, since electoral dynamics can change significantly from one election (at time t1) to another (t2), it is very difficult to attribute cause and effect with such a research design. The obvious problem is that there is no data on the counterfactual of no i-voting at t2. Thus in this paper we use an indirect approach. First, we identify online-only voters and then we examine the factors that explain the propensity of individuals to identify themselves as online-only voters. We can calculate the overall turnout effect attributable to the introduction of i-voting by using the proportion of self-reported online-only voters.

On July 4 2012 we conducted an online survey during the Rio Grande do Sul PB vote. All 124,211 voters who participated in the online vote were invited to complete the survey after they had voted. The survey instrument contained 27 questions, ranging from basic demographic details to participation profile and Internet usage (see appendix 3). A total of 22,300 people took the survey. There was substantial item non-response, meaning that the total

> 19 Some of the more sophisticated e-voting systems try to minimize the risk of so-called ‘over the shoulder attacks’ by enabling users to change their vote after they have been cast (e.g. Estonia’s electoral i-voting). Identifying solutions for mitigating the risk of multiple-voting and other factors that may distort outcomes is currently one of the activities supported by the World Bank’s Digital Engagement Unit.

11
sample size for the models in this paper – the number of respondents who gave an answer for each of the variables we use in the online-only voter models – is 18,235<sup>20</sup> . In the following we present the results obtained from the fully completed surveys from RS.

The overall response rate is 18 percent<sup>21</sup> This is a satisfactory ‘first contact’ response rate for both traditional (e.g. telephone) and online surveys (Cook, Heath, and Thompson 2000, Yeager et al. 2011, Pew 2012). We can analyze the location of each survey response using the respondent’s IP-address.<sup>22</sup> From the State capital of Porto Alegre there are 6,010 responses, and other cities with large number of respondents include: Pelotas (n=1,251), Caxias do Sul (n=1,122), and Santa Maria (n=1,098). Despite the reasonable response rate, concerns about non-response biases are still present: if respondents are missing at random, this is not a major concern, but it is also possible that there are systematic biases in non-response. It is therefore important to keep in mind that the results of this study are generated using a sample that may not be fully representative of the population of online voters. At the same time, however, we benefit from a very large sample drawn from across the entire state (Figure 1), and, we can infer a number of things by analyzing response patterns at the municipal level.

> 20 This reduced sample is partially the result of high non-response on the question of income, a notoriously sensitive question. To test whether this non-response affects our models, we reran the models without income both on the restricted sample of 18,235 that we use in the paper and a larger sample of 19,696 respondents who provided complete data except for income. Exactly the same set of parameters remained significant and in the same direction across both models and there were no large changes in effect size between the two. As a result, we conclude that item non-response is not driving the results in this paper.

> 21 The response rate is calculated based on the number of respondents compared with the total number of online voters.

> 22 Accuracy rates for naming the city using an IP address vary between 50%-80%, see http://whatismyipaddress.com/geolocation-accuracy, accessed May 19, 2014.

12
<!-- Start of picture text --> ane] 4<br>. S~ La<br>y<br>U éy }7‘<br>4af 5 iaw<br>— _/ S/<br>( } aN ;<br>]<br>al<br>Tarso vi<br>Alas- 456<br>4505- 506<br>BB 505- 55%<br>GN 555- 60x<br>WB 60s- 65%<br><!-- End of picture text -->
patory budgeting meetings in Porto Alegre.<sup>24</sup> Half of the respondents report they are working, while around a third report that they are studying or teaching. The number of participants that declare themselves unemployed is 7 percent and the proportion of retired respondents is around 5 percent. The unemployment rate in the state of Rio Grande do Sul is 6 percent, suggesting that the survey represents the general population relatively well with respect to employment status. The median age of participants is 31 years and three-quarters of the participants are between 25 and 44. The median age in the state is slightly higher around 35 years old. Quite surprisingly, a vast majority of the participants are not very active in the community. More than nine out of ten declare that they had not participated in any discussion regarding the budget before the vote.

With regard to Internet usage, more than 97 percent of the participants indicated that they had used the Internet before, but one in five had not used it in the previous three months. In this sense, it seems that i-voting attracts a non-negligible number of people who are traditionally considered as non-Internet users<sup>25</sup> . Slightly more than half of the participants said they were voting from home, while around 40 percent were voting from their place of work. Most importantly, a stunning 63.1 percent of all respondents identified themselves as online-only voters by answering no on the question about whether they would have voted if they had not been able to cast a vote online.<sup>26</sup>

# 4.2 Statistical Model and Results

In this section we present the results of the following logistic model: logit(πi) = α + β1Xi1 +... + βk Xik + εi where π is the probability that individual i self-identifies as an online-only voter, and Xik are common predictors of engagement. The models include dummy variables for each of the regions (COREDES), meaning that we are only looking at variance between individuals within a region rather than explaining regional level variation.

> 24 The proportion of females in the PB sessions in Porto Alegre in 2009 (the last year for which there is survey data), was 55.0 % (Fedozzi et al. 2013).

> 25 Individuals are traditionally considered Internet users when they have accessed the Internet in the last three months. See, for instance, ITU Manual for Measuring ICT Access and Use by Households and Individuals, 201 4 Edition.

> 26 Note that, as would be expected, the proportion of ‘online-only’ voters is slightly higher among those who at the time of the vote were outside the state. As already mentioned, we focus only on survey responses coming from an IP address within the boundaries of Rio Grande do Sul.

14
<!-- Start of picture text --> Use social networks | r |<br>Male | e |<br>Education |o|<br>Income |e |<br>Retired | e |<br>wy Age |<br>< N Ki<br>a OF WOrKINE | e |<br>> Student | e<br>Contact govt | Ps |<br>Took part in rieeting e |<br>PB meeting | e<br>0.25 -0.20 “0.15 -0.70 -0.05 0.00 0.05<br>MARGINAL EFFECT (percentage posnt increase in online-only voting)<br><!-- End of picture text -->
Social media usage has the highest marginal positive effect with an increase of 3.9 percentage points. The reason for this, we hypothesize, is the combination of two factors. Social media users are likely to spend more time on the Internet and have more technological skills (Muñoz and Towner 2011; Hargittai and Litt 2011). In this respect, a predictor for i-voting would be the degree of Internet usage and skills.<sup>28</sup> Second, given that the government and a considerable number of i-voters (33.7 percent) share content related to i-voting on social media channels, social media users are more likely to come across content mobilizing users to participate and direct links to the i-voting platform.

With regard to socioeconomic characteristics, in line with the digital divide literature, socioeconomically privileged groups are more likely to be online-only voters according to our analysis. Income and education both have relatively large marginal effects of nearly 2.6 and 2.9 percentage points respectively. Since education is measured on a 10-point scale and income on a 9-point scale, these effects can be substantial when considering the difference between individuals classified as low and high on these measures. Going from one extreme of the education scale to the other translates into a 56.3 percent increase in the probability of being an online-only voter (Figure 4). The equivalent number for the income scale is 38.0 percent.

With regard to gender inclusiveness, the marginal effect of being male is 2.9 percentage points (Figure 2). Yet, combining these regression results with the fact that the majority of survey respondents were female (54.2 percent), offers a more nuanced understanding of the effect of i-voting in terms of gender. Women state that they are less likely to be an online-only voter, but they both vote online as well as fill in the majority of the surveys. Thus, it appears that women might simply engage more frequently both in offline and online processes than men, without necessarily having a preference between the two.

As previously mentioned, the introduction of i-voting in PB processes is motivated by the expectation of attracting young participants. Indeed, there is a substantive relationship between age and i-voting, with 69% of 18 year olds predicted to be online-only voters compared to 56% of 40 year olds, as illustrated in Figure 3 below<sup>29</sup> .

> 28 However, the effects do not change if we include ‘active Internet user’ as a control for Internet literacy, instead of the social networks question. Being an active Internet user, which is something that 81.4 percent of the respondents classify themselves as, is non-significant with a p-value of .881.

> 29 Logistic regression with online-only voter as the dependent variable (Question 17, see appendix). All other independent variables, full specification (model 3 in Table 2 see appendix), are held at their mean except for region (COREDES) which is set to be the first region (the results are robust to other region choices). The regression includes regional (COREDES) dummies to account for variation across different settings (ballots etc.). 95% confidence intervals are displayed (in grey).

16
<!-- Start of picture text --> es)<br><<br>9<br>a<br>2<br>°<br>lu<br>= 06<br>=<br>F<br>°<br>as<br>o<br>><br>-E as<br>=!<br>oO<br><=<br>fs)<br>&<br>a (4<br>o<br>lu<br>G<br>4<br>Ww<br>& 20 30 40 50 60<br>AGE<br><!-- End of picture text -->
<!-- Start of picture text --> 0.7<br>0.4 0,43<br>0.3<br>1 2 3 4 5 | | | |<br>EDUCATION<br>0.7<br>0.69<br>0.6<br>0.5<br>0.5<br>0.4<br>0.3<br>1 2 3 4 5 | | |<br>INCOME<br><!-- End of picture text -->
Another challenge with the research design is that the counterfactual of no online voting is a hypothetical scenario in 2012. The reason this is problematic is that the introduction of online voting might have had an effect on the deliberative process building up to the vote, especially considering that the actual vote is the final step in an over three-month-long deliberative process where many stakeholders are actively engaged. Had there been no online voting, the whole campaign surrounding the vote might have looked different, which in turn could have affected voters’ attitudes about participation. It is not immediately clear how this could affect the observed results. If organizations involved in voter mobilization put in less effort due to the introduction of online voting, then many people might respond affirmatively to the ‘online-only voter’ question, even if under conditions of an offline-only campaign they might have been mobilized to vote.<sup>32</sup> However, there is no evidence of there being less voter mobilization in 2012.

In general though, social desirability biases and errors usually make respondents claim to be more participatory and engaged than they actually are, with consistently higher numbers claiming to have voted in surveys than actually turned out (Karp and Brockington 2005, 2005; Zeglovits and Kritzinger 2013). This means that the size of the online-only vote could in reality be even larger than estimated here.

Finally, we assess the overall turnout effect of introducing online technology. In the literature it has been suggested that allowing for Internet voting increases turnout by 0.5-2.6 percent (Trechsel et al. 2007).<sup>33</sup> Despite the reasonable response rate in the current study, any inferences to the overall population of online voters must be made cautiously. If we ignore a possible non-response bias and consider the sample to be roughly representative of all online voters, we can approximate how many new voters the new voting technology brought in. As almost two-thirds of the survey respondents claim to be online-only voters, the introduction of online voting could be said to have brought in 78,377 new voters, given that the total number of online voters was 124,211. We estimate the total number of voters that would have voted anyway to be 952,941, combining offline voters and online voters who said they would have voted anyway. While bearing in mind the aforementioned caveats to the study, we could

> 32 In addition, it is also possible that the presence of an online vote meant that online voters did not take time to find out about the details of an offline vote. However, if there had not been that option, it is possible that some of the online voters would have made the effort to find out where to vote offline.

> 33 ‘In 2009 this turnout loss [overall] would have been 2.6 per cent, so it’s a small effect on turnout. It’s very clear a convenience factor is important’, an interview with Trecshel, published at http://www.eui.eu/news/2013/02-12Internetvotingasuccessintwoeuropeancountries.aspx, accessed April 18 2014.

19
consider that the increase in turnout attributable to the introduction of online voting is around 8.2 percent.<sup>34</sup> While we cannot directly observe the attributes of those who did not respond to the survey, it is reasonable to assume that non-respondents are less participatory than respondents on average. This would mean that the calculation of the effect of i-voting on turnout is a conservative estimate, given that online-only voters are less participatory than other online voters, as shown here. Non-response could also be correlated with factors that make online-only voting more likely. Respondents who spend more time on the Internet or have more time free in the day might also be more willing to take an online survey. However, these effects are somewhat less clear. Both variables had a much smaller effect size on online-only voting than the effects of prior participation. All in all, it therefore seems reasonable to believe that the estimate of the overall turnout effect is conservative, while also acknowledging that the estimate has a substantial degree of uncertainty.

# 5. Concluding Remarks

This paper focused on the effects of remote Internet voting in a budget priority vote in southern Brazil. The overall effect on participation levels is considerable: an estimated 8.2 percent increase in turnout. With regard to the socioeconomic profile of participants, we find that youth, male gender, income, education and social media usage are significant predictors of being online-only voters. We find no substitution effects with the introduction of i-voting. Instead, our findings support the mobilization hypothesis, with i-voting attracting citizens who had no prior experience of contact with the government and who were previously disengaged from the participatory budgeting process. When put together these findings raise a number of questions for researchers and policy makers working with i-voting and the use of ICT in citizen engagement processes.

In comparison with the literature on i-voting in elections and referenda (Carter and Bélanger 2012) we find a larger effect of Internet voting on turnout. It is beyond the scope of this paper to fully assess why the effect of online voting on turnout is so much higher in the Rio Grande do Sul context than in others. Yet, a possible explanation for this difference is the fact that Rio Grande do Sul’s participatory budgeting is a low-salience event when compared, for instance, with Swiss referenda and the Estonian electoral process. This explanation reso-

> 34 Note that this figure is calculated from the same sample that we use for the models (i.e. only respondents who answered all the questions). If we include the full sample, then the implied turnout increase is 8.1 percent.

20
nates with the literature on convenience voting, which suggests that lowering participation costs has the strongest effects in low-salience events (Karp and Banducci 2000; Rallings and Thrasher 2007; Sled 2008). Further exploring this research path could shed light on when and where the use of ICT is more likely to make a difference for participation levels.

From an institutional design perspective, given that traditionally the PB process has strongly emphasized its redistributive justice character (de Sousa Santos 1998; Avritzer 2006), the fact that many of those becoming involved in the process of online voting are socioeconomically privileged people might be a cause for concern. Nevertheless, much of the PB design that promotes a pro-poor logic precedes the participatory stage (Marquetti, da Silva, and Campbell 2012), such as the pre-allocation of budgets that prioritize poorer geographic areas and investments that favor poorer sections of society (e.g. water, basic health).

Another issue concerns the effects of i-voting on the attributes of the process in terms of collective intelligence. One of the rationales of participatory processes is that of leveraging the dispersed knowledge of citizens to shape decisions that affect them (Ober 2008; Lévy 2001; Peixoto 2013). In this respect, a growing literature in the fields of epistemic democracy and decision-making suggests that increasing the diversity of participants improves the quality of decisions made, as new cognitive tools, perspectives, heuristics and knowledge are brought together (Ober 2008; Ober 2013; Landemore 2013; Page 2008; Hong and Page 2004; Page 2014)<sup>35</sup> . Thus, a possible hypothesis is that, through the aggregation of inputs from a more diverse group of participants, i-voting may allow for superior choices towards the collective interest. If proven through further research, this could inform the literature on collective intelligence and temper frequent reservations with regard to the use of technology in participatory processes.

Bearing these considerations in mind, the effect of i-voting on the final outcomes of PB is uncertain, both in terms of redistributive impact and collective intelligence. These are essentially empirical questions that can only be addressed by further research. Striking a balance between the ideals of redistributive justice and democratic inclusiveness remains a normative issue to be addressed, ideally, by Rio Grande do Sul citizens themselves.

> 35 Part of that literature emphasizes a type of collective intelligence that emerges primarily from discursive / deliberative problem-solving (Landemore 2013; Hong and Page 2004), which does not apply to voting processes. Yet, another part that focus on the epistemic properties of judgment aggregation (Ober 2008; Ober 2013; Page 2008; Page 2014; List and Goodin 2001) lends support to a consideration of the potential epistemic character of processes such as that of the State PB voting.

21
# References

- Aldrich, John H. 1993. “Rational Choice and Turnout.” American Journal of Political Science, 246–78.

- Alvarez, R. Michael, and Thad Edward Hall. 2004. Point, Click, and Vote: The Future of Internet Voting. Brookings Institution Press.

- Alvarez, R. Michael, Thad E. Hall, and Alexander H. Trechsel. 2009. “Internet Voting in Comparative Perspective: The Case of Estonia.” PS: Political Science & Politics 42 (03): 497..

- Alvarez, R. Michael, and Jonathan Nagler. 2000. “Likely Consequences of Internet Voting for Political Representation, The.” Loy. LAL Rev. 34: 1115.

- Arnstein, Sherry R. 1969. “A Ladder Of Citizen Participation.” Journal of the American Institute of Planners 35 (4): 216–24.

- Avritzer, Leonardo. 2006. “New Public Spheres in Brazil: Local Democracy and Deliberative Politics.” International Journal of Urban and Regional Research 30 (3): 623–37.

- ———. 2009. Participatory Institutions in Democratic Brazil. Johns Hopkins University Press. Bélanger, France, and Lemuria Carter. 2011. “The Impacts of the Digital Divide on Citizens’ Intentions to Use Internet Voting.” International Journal On Advances in Internet Technology 3 (3 and 4): 203–11.

- Best, Nina Juliette, Manuella Maia Ribeiro, Ricardo Matheus, and José Carlos Vaz. 2010. “Usando a Internet Para a Participação Cidadã: As Experiências de Orçamento Participativo Digital No Brasil.” Cadernos PPG-AU/UFBA, Vol. 9, Edição Especial Democracia E Interfaces Digitais Para a Participação Pública.

- Bochsler, Daniel. 2010. “Can Internet Voting Increase Political Participation.” Remote Electronic.

- Borowski, Edson Moraes. 2012. “O Desequilíbrio Fiscal Do Estado Do Rio Grande Do Sul Causas, Políticas de Ajuste E Os Efeitos Sobre a Sociedade Gaúcha.” Dissertation, Porto Alegre.

- Brandtzæg, Petter Bae, Jan Heim, and Amela Karahasanović. 2011. “Understanding the New Digital divide—A Typology of Internet Users in Europe.” International Journal of Human-Computer Studies 69 (3): 123–38.

- Breuer, Anita, and Bilal Farooq. 2012. “Online Political Participation: Slacktivism or Efficiency Increased Activism? Evidence from the Brazilian Ficha Limpa Campaign.” Evidence from the Brazilian Ficha Limpa Campaign (May 1, 2012). .

22
- Carter, Lemuria, and France Bélanger. 2012. “Internet Voting and Political Participation: An Empirical Comparison of Technological and Political Factors.” ACM SIGMIS Database 43 (3): 26–46.

- Christensen, Henrik Serup. 2011. “Political Activities on the Internet: Slacktivism or Political Participation by Other Means?” First Monday 16 (2).

- Coelho, Vera Schattan P. 2004. “Brazil’s Health Councils: The Challenge of Building Participatory Political Institutions.” IDS Bulletin 35 (2): 33–39.

- Collins, Kevin, and Raymond Ison. 2006. “Dare We Jump off Arnstein’s Ladder? Social Learning as a New Policy Paradigm.”

- Compaine, Benjamin M. 2001. The Digital Divide: Facing a Crisis Or Creating a Myth?. MIT Press.

- Connor, Desmond M. 1988. “A New Ladder of Citizen Participation.” National Civic Review 77 (3): 249–57.

- Cook, Colleen, Fred Heath, and Russel L. Thompson. 2000. “A Meta-Analysis of Response Rates in Web- or Internet-Based Surveys.” Educational and Psychological Measurement 60 (6): 821–36.

- Davis, Richard. 1998. The Web of Politics: The Internet’s Impact on the American Political System. Oxford University Press, Inc.

- De Sousa Santos, Boaventura. 1998. “Participatory Budgeting in Porto Alegre: Toward a Redistributive Democracy.” Politics and Society 26: 461–510.

- Deursen, Alexander JAM, Ellen Helsper, and Rebecca Eynon. 2014. “Measuring Digital Skills: From Digital Skills to Tangible Outcomes Project Report.”

Done, Robert S. 2002. “Internet Voting.” Downs, Anthony. 1957. “An Economic Theory of Democracy.”

- Fedozzi, L, A Furtado, V Bassani, C Macedo, C Parenza, and M Cruz. 2013. “Porto Alegre’s Participatory Budget Profile, Evaluation, and Participants’ Perceptions.”

- Ferrari, Anusca. 2012. “Digital Competence in Practice: An Analysis of Frameworks.” Sevilla: JRC IPTS.(DOI: 10.2791/82116).

- Fung, Archon. 2011. “Reinventing Democracy in Latin America.” Perspectives on Politics 9 (04): 857–71.

- Goldfrank, Benjamin. 2013. “A Frondescent Tree of Subnational Democracy? The New Participation System in Rio Grande Do Sul.”

- Goldfrank, Benjamin, and Aaron Schneider. 2006. “Competitive Institution Building: The PT and Participatory Budgeting in Rio Grande Do Sul.” Latin American Politics &amp; Society 48 (3): 1–31.

- Gonçalves, Sónia. 2014. “The Effects of Participatory Budgeting on Municipal Expenditures and Infant Mortality in Brazil.” World Development, Decentralization and Governance, 53 (January): 94–110.

- Gonzalez-Ocantos, Ezequiel, Chad Kiewiet de Jonge, Carlos Meléndez, Javier Osorio, and David W. Nickerson. 2012. “Vote Buying and Social Desirability Bias: Experimental Evidence from Nicaragua.” American Journal of Political Science 56 (1): 202–17.

- Goodman, Nicole Janine, Jon H. Pammett, and Joan DeBardeleben. 2010. A Comparative Assessment of Electronic Voting. Elections Canada.

- Hargittai, Eszter, and Eden Litt. 2011. “The Tweet Smell of Celebrity Success: Explaining Variation in Twitter Adoption among a Diverse Group of Young Adults.” New Media & Society 13 (5): 824–42.

- Heller, Patrick, Gianpaolo Baiocchi, and Marcelo Silva. 2011. Bootstrapping Democracy: Transforming Local Governance and Civil Society in Brazil. Stanford University Press.

- Helsper, Ellen Johanna, and Rebecca Eynon. 2013. “Distinct Skill Pathways to Digital Engagement.” European Journal of Communication.

23
Hilbert, Martin. 2010. “When Is Cheap, Cheap Enough to Bridge the Digital Divide? Modeling Income Related Structural Challenges of Technology Diffusion in Latin America.” World Development 38 (5): 756–70.

- Hindman, Matthew. 2009. The Myth of Digital Democracy. Princeton University Press.

- Hong, Lu, and Scott E. Page. 2004. “Groups of Diverse Problem Solvers Can Outperform Groups of High-Ability Problem Solvers.” Proceedings of the National Academy of Sciences of the United States of America 101 (46): 16385–89.

- Hsieh, JJ Po-An, Arun Rai, and Mark Keil. 2008. “Understanding Digital Inequality: Comparing Continued Use Behavioral Models of the Socio-Economically Advantaged and Disadvantaged.” MIS Quarterly, 97–126.

- Hu, Chih-Wei. 2014. “Health Slacktivism on Social Media: Predictors and Effects.” In Social Computing and Social Media, 354–64. Springer. .

- Hyde, Susan D. 2011. The Pseudo-Democrat’s Dilemma: Why Election Monitoring Became an International Norm. Cornell University Press.

- Karp, Jeffrey A., and Susan A. Banducci. 2000. “Going Postal: How All-Mail Elections Influence Turnout.” Political Behavior 22 (3): 223–39.

- Karp, Jeffrey A., and David Brockington. 2005. “Social Desirability and Response Validity: A Comparative Analysis of Overreporting Voter Turnout in Five Countries.” Journal of Politics 67 (3): 825–40.

- Kousser, T., and M. Mullin. 2007. “Does Voting by Mail Increase Participation? Using Matching to Analyze a Natural Experiment.” Political Analysis 15 (4): 428–45.

- Landemore, Hélène. 2013. Democratic Reason: Politics, Collective Intelligence, and the Rule of the Many. Princeton University Press.

- Lanzone, Maria Elisabetta, and Stefano Rombi. 2014. “Who Did Participate in the Online Primary Elections of the Five Star Movement (M5S) in Italy? Causes, Features and Effects of the Selection Process.” Partecipazione E Conflitto 7 (1): 170–91.

- Lee, Yu-Hao, and Gary Hsieh. 2013. “Does Slacktivism Hurt Activism?: The Effects of Moral Balancing and Consistency in Online Activism.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 811–20. ACM.

- Lévy, Pierre. 2001. Cyberculture. Vol. 4. U of Minnesota Press.

- List, Christin, and Robert E. Goodin. 2001. “Epistemic Democracy: Generalizing the Condorcet Jury Theorem.” Journal of Political Philosophy 9 (3): 277–306.

- Mann, Christopher B., and Genevieve Mayhew. 2012. “Multiple Voting Methods, Multiple Mobilization Opportunities? Voting Behavior, Institutional Reform, and Mobilization Strategy.” Unpublished Manuscript (University of Miami).

- Marquetti, Adalmir, Carlos E. Schonerwald da Silva, and Al Campbell. 2012. “Participatory Economic Democracy in Action Participatory Budgeting in Porto Alegre, 1989–2004.” Review of Radical Political Economics 44 (1): 62–81.

- Mendez, Fernando. 2013. “EU Democracy and E-Democracy: Can the Two Be Reconciled?” In Democracy in Transition, 161–78. Springer.

- Moreira, Marcelo Rasga, and Sarah Escorel. 2009. “Municipal Health Councils of Brazil: A Debate on the Democratization of Health in the Twenty Years of the UHS.” Ciênc. Saúde Coletiva 14 (3): 795–806.

- Moreton Jr, Edward B. 1985. “Voting by Mail.” S. Cal. L. Rev. 58: 1261.

- Morozov, Evgeny. 2012. The Net Delusion: The Dark Side of Internet Freedom. Reprint edition. New York: PublicAffairs.

- Muñoz, Caroline Lego, and Terri Towner. 2011. “Back to the ‘wall’: How to Use Facebook in the College Classroom.” First Monday 16 (12).

- Nitzsche, Philipp, Adriano Pistoia, and Marc Elsäßer. 2012. “Development of an Evaluation Tool for Participative E-Government Services: A Case Study of Electronic Participatory

24
Budgeting Projects in Germany.” Revista“ Administratie Si Management Public”(RAMP), no. 18: 6–25.

- Norris, Pippa. 2001. Digital Divide: Civic Engagement, Information Poverty, and the Internet Worldwide. Cambridge ; New York: Cambridge University Press.

- Ober, Josiah. 2008. Democracy and Knowledge: Innovation and Learning in Classical Athens. Princeton University Press.

- ———. 2013. “Democracy’s Wisdom: An Aristotelian Middle Way for Collective Judgment.” American Political Science Review 107 (01): 104–22.

- Oliveira, Victor da Silva, and Erica Karnopp. 2013. “Consulta Popular E Os Planejamentos Regionais No Rio Grande Do Sul: Intersecções E Resultados.” RDE-Revista de Desenvolvimento Econômico 15 (27).

- Oostven, Anne-Marie, and Peter Van Den Besselaar. 2004. “Internet Voting Technologies And Civic Participation: The Users’perspective.” Javnost 11.

- Ostapczuk, Martin, Jochen Musch, and Morten Moshagen. 2009. “A Randomized-Response Investigation of the Education Effect in Attitudes towards Foreigners.” European Journal of Social Psychology 39 (6): 920–31.

- Page, Scott E. 2008. The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton University Press.

- ———. 2014. “Where Diversity Comes from and Why It Matters?” European Journal of Social Psychology.

- Pammett, Jon H., and Nicole Goodman. 2013. “Consultation And Evaluation Practices In The Implementation Of Internet Voting In Canada And Europe.”

- Pateman, Carole. 2012. “Participatory Democracy Revisited.” Perspectives on Politics 10 (01): 7–19.

- Peixoto, Tiago. 2009. “Beyond Theory: E-Participatory Budgeting and Its Promises for eParticipation.” European Journal of ePractice 7: 55–63.

- ———. 2013. “The Uncertain Relationship Between Open Data and Accountability: A Response to Yu and Robinson’s The New Ambiguity of‘ Open Government.’” UCLA L. Rev. Disc. 60: 200–248.

- Rallings, Colin, and Michael Thrasher. 2007. “The Turnout ‘gap’and the Costs of Voting–a Comparison of Participation at the 2001 General and 2002 Local Elections in England.” Public Choice 131 (3-4): 333–44.

- Riker, William H., and Peter C. Ordeshook. 1968. “A Theory of the Calculus of Voting.” The American Political Science Review, 25–42.

- Roberson, Quinetta M. 2006. “Disentangling the Meanings of Diversity and Inclusion in Organizations.” Group & Organization Managemen 31 (2): 212–36.

- Sampaio, Rafael Cardoso, Rousiley Celi Moreira Maia, and Francisco Paulo Jamil Almeida Marques. 2011. “Participation and Deliberation on the Internet: A Case Study of Digital Participatory Budgeting in Belo Horizonte.” The Journal of Community Informatics 7 (1-2).

- Sarkar, Urmimala, Andrew J. Karter, Jennifer Y. Liu, Nancy E. Adler, Robert Nguyen, Andrea López, and Dean Schillinger. 2011. “Social Disparities in Internet Patient Portal Use in Diabetes: Evidence That the Digital Divide Extends beyond Access.” Journal of the American Medical Informatics Association 18 (3): 318–21.

- Schlozman, Kay Lehman, Sidney Verba, and Henry E. Brady. 2010. “Weapon of the Strong? Participatory Inequality and the Internet.” Perspectives on Politics 8 (02): 487–509.

- Shulman, Stuart W. 2009. “The Case Against Mass E-Mails: Perverse Incentives and Low Quality Public Participation in US Federal Rulemaking.” Policy & Internet 1 (1): 23–53.

25
Sintomer, Yves, Carsten Herberg, Giovanni Allegretti, Anja Röcke, and Mariana Alves. 2013.

- “Participatory Budgeting Worldwide – Updated Version.” Global Civic Engagement – Service for Development Initiatives).

- Sjoberg, Fredrik M. 2012. “Making Voters Count: Evidence from Field Experiments about the Efficacy of Domestic Election Observation.” Columbia University Harriman Institute Working Paper, no. 1.

- Sled, Sarah Marie. 2008. “It’s in the Mail: The Effect of Vote by Mail Balloting on Voter Turnout and Policy Outcomes in US Elections.” Massachusetts Institute of Technology.

- Smith, Aaron. 2013. “Civic Engagement in the Digital Age.” Pew Internet & American Life Project.

- Stein, Robert M., and Greg Vonnahme. 2011. “Voting at Non-Precinct Polling Places: A Review and Research Agenda.” Election Law Journal 10 (3): 307–11.

- Stortone, Stefano, and Fiorella De Cindio. 2014. “From Online to Offline Participation and Viceversa: A Software Platform for New Political Practices.” In Communities and Technologies (C&T), Digital Cities 8.

- Touchton, M., and B. Wampler. 2013. “Improving Social Well-Being Through New Democratic Institutions.” Comparative Political Studies, December.

- Trechsel, Alexander, Guido Schwerdt, Fabian Breuer, Michael Alvarez, and Thad Hall. 2007. “Report for the Council of Europe: Internet Voting in the March 2007 Parliamentary Elections in Estonia.” Strasbourg: Council of Europe.

- Vassil, K., and T. Weber. 2011. “A Bottleneck Model of E-Voting: Why Technology Fails to Boost Turnout.” New Media & Society 13 (8): 1336–54.

- Vie, Stephanie. 2014. “In Defense of ‘slacktivism’: The Human Rights Campaign Facebook Logo as Digital Activism.” First Monday 19 (4).

- Vissers, Sara, and Dietlind Stolle. 2014. “Spill-over Effects between Facebook and On/offline Political Participation? Evidence from a Two-Wave Panel Study.” Journal of Information Technology & Politics,

- Vitak, Jessica, Paul Zube, Andrew Smock, Caleb T. Carr, Nicole Ellison, and Cliff Lampe. 2011.

- “It’s Complicated: Facebook Users’ Political Participation in the 2008 Election.” CyberPsychology, Behavior, and Social Networking 14 (3): 107–14.

- Wampler, Brian. 2007. Participatory Budgeting in Brazil: Contestation, Cooperation, and Accountability. University Park, Pa: Pennsylvania State Univ Pr.

- Wolchok, Scott, Eric Wustrow, Dawn Isabel, and J. Alex Halderman. 2012. “Attacking the Washington, DC Internet Voting System.” In Financial Cryptography and Data Security, 114–28.

- Zeglovits, Eva, and Sylvia Kritzinger. 2013. “New Attempts to Reduce Overreporting of Voter Turnout and Their Effects.” International Journal of Public Opinion Research..

26
## Appendices

# 1. Example of Ballot (2013)

|VOTAÇÃODE P|RIORIDADES-ORÇAMENTO2014 - REGIÃOMETROPOLITANA DELTA DO JACUÍ<br>Campo 1 - Itens de 1 a 15 vote: em até 04 (quatro) demandas<br>(Field1 - Items1to15 vote:up tofour(04) demands)||
|---|---|---|
|Area Temática|Demanda<br>Valo|r em R$|
||Fomento à Cultura - construção, reforma e modernização||
|Cultura e Inclusão Digital|1<br>( )<br>de espaços culturais, pontos de cultura, eventos culturais e<br>desenvolvimento da economia da cultura|1.500.000,00|
||2<br>( )<br>Telecentros<br>|500.000,00|
|DesenvolvimentoEconómico<br>|3<br>( )<br>Apoio àsÁreas e/ouDistritosIndustriais<br> <br><br>|2.000.000,00<br>|
|Saúde|4<br>( )<br>Fortalecimento daInfraestrutura do SUS<br>|10.000.000,00|
|Segurança Pública e Defesa Civil|5<br>( )<br>Aparelhamento dos Órgãos de Segurança Pública|5.000.000,00|
||Prevenção,Combate à Drogadição - Enfrentamento ao||
|Cidadania, Justiça eDireitosHumanos|6<br>( )<br> <br>Crack|4.000.000,00|
|Política para asMulheres|7<br>( )<br>Prevenção,Enfrentamento à Violência Contra asMulheres|2.000.000,00|
|Meio Ambiente e Recursos Hídricos|8<br>( )<br>Implementação de Planos, Programas e Projetos<br>Ambientais<br>|1.000.000,00|
||Áreas Esportivas, Eguipamentos, Eventos de Esporte e||
|Esporte eLazer|9<br>( )<br>Lazer|4.000.000,00|
|Turismo|10<br>( )<br>InfraestruturaTurística|1.000.000,00|
|Desenvolvimento Social e Erradicação da<br>Pobreza|11<br>( )<br>Apoio a Ações de Geração de Trabalho e Renda<br>Restaurantes, Populares, Galpões de Reciclagem e Apoio a<br>Ações Comunitárias, QualificaçãoProfissional|1.000.000,00|
|Habitaão Desenvolvimento Urbano e|12<br>( )<br>Recuperação e Pavimentação de Vias Urbanas|12.000.000,00|
|ç,<br>S|13<br>( )<br>Saneamento ePerfuração dePoçosTubulares|2.000.000,00|
|aneamento|14<br>( )<br>RegularizaçãoFundiária em ÁreasEstaduais|4.000.000,00|
|Educação|15<br>( )<br>Modernização Tecnológica - eguipamentos, mobiliários<br>básicosedegualificaçãodosespaçosescolares|2.000.000,00|
|Cam<br>(F<br>Área Temática|<br>po 2 - Itens de 21 a 25: Vote em até 02 (duas) Prioridades Regionais<br>ield2 - Items21-25:Voteforup to two (02)Regional Priorities)<br>Ação do PPA com Complemento||
|Habitação, Desenvolvimento Urbano e|21<br>( )<br>||
|Saneamento|<br> <br>Aguisição de área para assentamento emparceria como GovernoMunicipal,|EstadualeFederal|
|Saúde|22<br>( )<br>Hospital Regional||
|MeioAmbiente eRecursosHídricos|23<br>( )<br>Preservação deNascentes eAfluentes||
|Habitação, Desenvolvimento Urbano e|24<br>( )<br>Dulicaão da Estrada do Conde||
|Saneamento|<br> <br>pç||
|SegurançaPública|25<br>( )<br>QuartelparaBombeiros:construção,reforma emelhorias||
||Campo 3 -1 - Reforma Política em consulta<br>(Field 3–Consultationon Political Reform)||
||Alternativas (escolha 1 opção em cada questão)<br>||
|Questão|Alternatives (choose one option foreachquestion)<br><br><br>||
|à f lí|26<br>( )<br>Sou afavorderealizarumaReformaPolítica||
|1. Quanto Reorma Potica|27<br>( )<br>Sou contra umaReformaPolítica-afavorde deixarcomo está||
||28<br>( )<br>Pelo Congresso, comos atuais deputados e senadores||
||29<br>( )<br>Pelo próximo Congresso, a sereleito em 2014||
|2. Como deve ser feita a Reforma Política|<br>( )<br>Por meio de uma Constituinte exclusiva, com representantes eleitos para|esse fim, com prazo|
||30<br> <br> <br>definido||
|3 Quanto ao financiamento das camanha|<br>31<br>( )<br>Deve ser custeadopor recursospúblicos||
|. p<br>eleitorais|<br>32<br>( )<br>Deve sercusteado por recursos privados<br><br><br>||
||33<br>( )<br>Deve ser misto (continuarcomohoje: fundo público e privado^||
||Alternativas (escolha até 2 opções na questão 4)||
|Questão|<br>Alternatives (choose up to2optionsinquestion 4)||
||34<br>( )<br>Reforma do SistemaEleitorale dofinanciamento de campanhas||
|4. Quais os dois temas que você considera<br>|35<br>( )<br>Transparência e comportamento ético dos agentes públicos e privados<br><br><br>||
|mais importantes?|36<br>( )<br>Nova divisão derecursos entre a União,Estados eMunicípios<br>37<br>( )<br>Maiorparticipação da populaçãonas decisões públicas||

27
# 2. Supplementary Statistical Material

## Main Models

Table 2. Logistic Regression Predicting which Voters Claim to be Online-Only Voters (Would Not Have Voted Offline).

|||Basic||Digit|al Divide||Prior p|articipati|on|
|---|---|---|---|---|---|---|---|---|---|
||Estimate|Std. Error||Estimate|Std. Error||Estimate|Std. Error||
|(Intercept)|0.6070|0.3706||0.4817|0.3753||0.2631|0.3840||
|Age|-0.0361|0.0078|***|-0.0358|0.0078|***|-0.0191|0.0079|*|
|Age^2|0.0000|0.0001||0.0000|0.0001||-0.0001|0.0001||
|Male|0.0654|0.0329|*|0.0699|0.0330|*|0.1271|0.0338|***|
|Student|-0.0958|0.0395|*|-0.0989|0.0395|*|-0.0723|0.0404|.|
|Retired|0.1517|0.0886|.|0.1466|0.0887|.|0.0258|0.0903||
|Not working|0.0023|0.0641||-0.0002|0.0641||-0.0459|0.0651||
|Income (ordinal)|0.1183|0.0122|***|0.1187|0.0122|***|0.1141|0.0125|***|
|Education (ordinal)|0.1049|0.0147|***|0.1048|0.0147|***|0.1252|0.0150|***|
|Density (log)|0.0224|0.0156||0.0224|0.0156||0.0121|0.0159||
|Log dist. to PA (km)|-0.0855|0.0490|.|-0.0850|0.0490|.|-0.0700|0.0501||
|Uses social networks||||0.1117|0.0532|*|0.1655|0.0544|**|
|Took part in meeting|||||||-0.4589|0.0393|***|
|Contacted the govt|||||||-0.2930|0.0388|***|
|PB meeting|||||||-0.9086|0.0585|***|
|n||18,235||1|8,235|||18,235||

* Notes: region (COREDES) dummies included.

28
<!-- Start of picture text --> Gender Work status<br>225 e<br>8 —<br>Ss © 5<br>éS85. a25.<br>Income Education<br>2 © 75-<br>& 7s- 3<br>Fa8a 58Ss<br>5 so- 3<br>£25§ 254<br><!-- End of picture text --> <!-- Start of picture text --> =°33<br>Age<br><!-- End of picture text -->
zg5 <!-- Start of picture text --> Log distance to capital<br><!-- End of picture text --> mall :
# 3. Questionnaire

1. In which city do you live (reside)?

2. What is your age?

3. What is your Gender?

4. Which of the following activities best describe your occupation in the last seven days? (You can select more than one answer)

- paid work

- educational activities ( school, university )

- unemployed and actively seeking work

- unemployed, wanting a job but not actively seeking a job.

- person with disabilities

- retired

- community Service or Military

- housework , caring for children or other persons

- other occupation (specify below)

5. What is your education level?

- literate

- first grade incomplete

- first grade completed

- second grade incomplete (high school)

- medium completed  (high school)

- B.A. incomplete

- B.A. completed

- master’s degree

- doctorate

6. Have you ever used the Internet before?

- yes

- I never used the Internet, it is the first time today

7. When did you use the Internet last?

- less than 3 months ago

- between 3 months and 12 months ago

- more than 12 months ago

8. In which of these places have you used the Internet in the past? (You can select more than one answer)

- at home

- at work

- in school (or educational institution )

- at the home of another person ( friend, neighbor or family )

- free public access center (kiosk, library , community organization , etc.).

- paid public access center  (lan house, cyber cafe , Internet cafe , etc. . )

- anywhere using a mobile phone

9. From what location are you using Internet in this moment to cast the vote?

- home

- at work

- school (or educational institution )

- at the home of another person ( friend, neighbor or family )

- free public access center (kiosk , library , community organization , etc.).

- paid public access center (lan house , cyber cafe , Internet cafe)

- anywhere using a mobile phone

10. If you already used the Internet, which social media sites below do you use? (You can select more than one answer) 31
- I do not use social media

- Facebook

- Orkut

- Google + ( Google Plus )

- Twitter

- Linked-In

- Sonico

- Foursquare

- Ning

- others, indicate in the space below

11. Do you own a cell phone?

- yes

- no

12. Do you send and receive text messages via cell phone?

- I only receive messages

- I only send messages

- I send and receive messages

- I do not send nor receive messages

13. How did you hear about the process of voting the priorities of the budget? (You can select more than one answer)

- television

- radio

- newspaper

- e- mail sent by the administration

- through a family member

- through another person ( colleague, friend )

- through the Internet

- others, indicate in the space below

14. On the Internet, how did you learn about the process? (You can select more than one answer)

- I did not know about the process from Internet

- from news websites

- from social networks (Orkut , Twitter , etc. . )

- from personal blogs

- from an email (electronic mail)

- others, indicate in the space below

15. Did you share, comment on or ‘‘like’’ some content about the process of voting the priorities of the budget through social networks?

- yes

- not

- 16. Which of the following means did you use to share, comment on or ‘‘like’’ content? (You can select more than one answer)

- email

- Facebook

- Orkut

- Google +

- Twitter

- Linked-in

- Sonico

- Foursquare

- blogs

- Ning

- Other, indicate in the space below

17. Would you have voted if you did not have the opportunity to cast your vote via Internet?

32
- Yes, I probably would have voted in person rather than using the Internet.

- No, I probably would not have voted.

18. Before the vote today, did you already participate in some discussion or meeting on the 2013 budget?

- Yes

- No

19. In previous years, did you already vote via Internet for the budget priorities?  Yes

- Not

20. How often do you talk about politics or the state government with other people (friends, family, colleagues)?

- at least once a week

- at least once a month

- never

21. Which of the following actions did you perform in the last 12 months? (You can select more than one answer)

- contacting someone from the government or the administration about a subject that seemed important to you

- attending a meeting in your community on a local problem or issue

- attending a Participatory Budgeting meeting in my municipality

- attending a meeting of the Regional Development Councils (COREDES)

- attend in person or via the Internet ‘ Governor Asks ‘ or ‘ Governor Responds ‘

- not participated in any of these activities

22. Did you hear about the possibility of voting the priority via mobile phone?

- I did not , but I believe it will be useful

- I did not , but I believe it will not be useful

- I did, but I believe it will be useful

- I did, but I believe it will not be useful

23. Below is a list of related organizations, entities and activities that few people participate. Indicate whether you participated in some of these organizations or activities in the last 12 months. (You can select more than one answer)

[for each question the respondent could answer yes/no/I do not know]

- A cultural or arts organization

- A trade union

- A shopkeepers’ association or other firms

- A religious event (Mass, worship, etc.).

- A political party

- Another organization or entity

24. About the online voting system, what were your main difficulties? (You can select more than one answer)

- I did not experience difficulties

- finding information about my preferred priority

- the page was slow to load (very heavy page)

- it was difficult to enter required data (voter registration , identity)

- I had trouble understanding the instructions or the instructions were inadequate

- other ̧ indicate in the space below

25. In the table below indicate the approximate monthly income of your household, ie, the sum of the monthly income of all members of your household including your income.

- Less than R $ 622.00

- R $ 622.01 up to R $ 1,244.00

- R up to R $ 1,244.01 $ 1,866.00

- R up to R $ 1,866.01 $ 3,110.00

- R up to R $ 3,110.01 $ 6,220.00

33
- R up to R $ 6,220.01 $ 12,440.00

- R up to R $ 12,440.01 $ 18,660.00

- More than R $ 18,660.00

- Do not know

26. What do you think of the idea of receiving updates via cell phone on the progress of budget priorities that you selected?

- totally useless

- little useful

- reasonably useful

- very useful

27. Please give us suggestions on how the voting process on budget priorities could be improved (open ended) 34
# 4. The five stages of the Citizens Budget<sup>36</sup>

The Citizens’ Budget is managed by a commission composed by four representatives of the state government and four representatives of the regional councils of development (COREDES). COREDES were introduced in 1994 as a venue to allow civil society organizations to influence development plans for each of the 28 regions of the state. Each region has its own bylaws (Regimento Interno) that details which organization is eligible to participate in each COREDE.

This statewide commission (Coordenação Estadual da Participação Popular e Cidadã) organizes the meetings, defines the advertising strategy and monitors the implementation of projects. The process is divided in 5 stages and starts officially in March and ends in September. In the first stage 28 public regional assemblies (Audiência Pública Regionais) are organized by each COREDE. These assemblies are public, but most participants are from civil society organizations that are invited by each COREDE.

In these assemblies the state government first presents the overall financial situation of the state, mapping out investments, the amount that has been invested in the region, the guidelines that each sub-branch of the state government has for developing the budget, and the multi-year priorities of the budget. The guidelines for the period 2012-2015 were defined by another participatory engagement process that occurred in 2011 – the multi-year participatory plan (PPA participativo). Then representatives of COREDE present their development plan for the region. Then there are two votes. 1) The participants select the thematic areas and the regional priorities that will affect the typology of projects that can be presented in the participatory budgeting process. 2) The participants vote for three representatives that will enter the Coordenação Regional da Participação Popular e Cidadã. These regional coordination commissions are composed of three representatives of the state government, three representatives of COREDES, and three representatives of the public regional assembly. These meetings are mandated by the law to last at maximum three hours.

From May to June COREDES, in collaboration with its municipal level counterpart –the council for municipal development (Conselho Municipal de Desenvolvimento COMUDES) – organize one assembly in each municipality (497 in 2012). These assemblies are advertised to the public and thus effectively open to all citizens. The minimal quorum for one of these as-

> 36 The sources of this section are 1) an interview with Ricardo Almeda at SEPLAG, 2) the bylaws of the process that can be found together with other official documents pertaining the process at: http://www.portaldaparticipacao.rs.gov.br/documentos-sistema/

35
semblies is 30 participants. In these assemblies the participants propose up to 10 projects within the thematic areas that had been previously selected by the regional assemblies and propose additional regional priorities. These councils also elect delegates, one for every 30 participants.<sup>37</sup> These meetings are mandated by the law to last at maximum two hours and a half.

In the third phase all municipal delegates meet in regional forums (Fóruns Regionais da ParticipaçãoPopular e Cidadã). The regional forums are augmented by the members of COREDES, members of the regional coordination commission and by the regional representatives of the PPA participativo. These assemblies take the input of the previous engagement processes and construct the ballot. More precisely they identify ten projects for the first field of the ballot. These projects have already a specific monetary value. Then they identify five regional priorities. These priorities do not have a monetary value and will compose the second section of the ballot. Then two representatives are elected. These representatives will participate in the Fórum Estadual da Participação Popular e Cidadã (the state level participatory forum). The latter forum contains also representatives of COREDES, of participants in the PPA participativo, the state level coordinating assembly and the state government. The ballot is further revised by the state bureaucracy that eliminates projects that are not feasible.

In the fourth phase the entire population that has an electoral certificate is invited to vote. In 2012 the vote was on the July 4. It lasts one day in the traditional face-to-face format, and three days online.

In the fifth and final phase the Fórum Estadual da Participação Popular e Cidadã uses the results of the vote to define the budget. This forum also continues to monitor the implementation of the projects together with each regional COREDES.

> 37 One additional delegate is elected if there is a residual of more than 15 people.

36