# Evaluating E-Participation Institutional Design. A Pilot Study of Regional Platforms in Russia

Andrei V. Chugunov, Yury Kabanov

<figure><img src="image_1.png" /></figure>

## To cite this version:

Andrei V. Chugunov, Yury Kabanov. Evaluating E-Participation Institutional Design. A Pilot Study of Regional Platforms in Russia. 10th International Conference on Electronic Participation (ePart), Sep 2018, Krems, Austria. pp.13-25, 10.1007/978-3-319-98578-7_2 . hal-01985607

HAL Id: hal-01985607

https://inria.hal.science/hal-01985607

Submitted on 18 Jan 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Evaluating E-Participation Institutional Design. A Pilot Study of Regional Platforms in Russia

Andrei V. Chugunov¹ and Yury Kabanov²

¹ ITMO University, St. Petersburg, Russia
chugunov@corp.ifmo.ru

² National Research University Higher School of Economics, St. Petersburg, Russia
ykabanov@hse.ru

**Abstract.** The paper presents an attempt to develop an e-participation evaluation technique that considers institutional design, compatible with large- and small-N analysis, as well as useful for policy-makers. Based on the new institutionalism and previous research, we assess the development of *access, embeddedness and control* features of e-participation. The framework is tested on 85 Russian regional e-participation portals, followed by the analysis of factors that might explain the variation. Possible applications and future research are also discussed.

**Keywords:** E-Participation, Evaluation, Institutional Design, Russian Regions

## 1 Introduction

Why does e-participation look as it does, and what factors move the innovation forward? One answer is proposed by the *explaining* perspective, that is “regarded as providing an account of a phenomenon on the basis of an outsider’s perspective” [28]. In fact, quantitative analysis of e-government / e-participation has provided us with a rich account on political, socio-economic and technological factors that drive innovations [19, 23, 34, 36], but the key problem here is the one of measurement. The *UN E-Participation Index* and similar techniques have been criticized for measuring mostly technological aspects and ignoring the context [7, 11, 12, 24]. The *understanding* perspective, on the contrary, views e-tools development as a process of institutional building and adaptation, with actors’ choices and actions in a certain context [28]. It goes deeper into detail but lacks a large-N comparative perspective. Reconciliation of both approaches is hard but necessary to evaluate e-participation comprehensively.

We suggest that the first step here is the shift of assessment techniques from technological readiness to the internal configuration of e-participation tools themselves. Various features of e-participation can be viewed as regulators of citizens and officials’ behavior that basically form its *institutional design*. The latter, on the one hand, is a result of actors’ choices within a certain context, on the other hand, is resulted in outcomes related to democracy and decision-making. This view corresponds to the new institutionalism [13], which is becoming more popular in the area [14, 15].
In this paper we propose a measurement that, we suppose, fit the idea outlined above. Our goal is to exemplify the possibility of quantitative evaluation of e-participation institutional design features. We present a pilot study of the Russian regional e-participation portals' institutional design, carried out in 2017. We propose a theoretical framework of such assessment, operationalize the dimensions and conduct a quantitative evaluation of 85 regional portals of Russia. Finally, we discuss the possible factors explaining the variation, outlining problems and future directions of research.

# 2 E-Participation Institutional Design Evaluation

## 2.1 Related Work and Analytical Framework

E-Government / E-Participation studies have offered a variety of evaluation methodologies so far. Some of them go deeper than *the UN Index* to look at the qualitative aspects of e-participation arenas tools [37, 38]. The established practice is checking the availability of the preset website features, but the questionnaires differ greatly upon theoretical and practical underpinnings [39]. Nowadays more research emphasizes the impact of design on e-participation success, since the way a website is organized may foster or hinder participation [35]. The registration and authentication rules do matter [5, 25], as well as the user-friendliness of design and the responsiveness of the officials [20].

E-participation must also give necessary incentives to decision-makers and be incorporated well into the “the daily routines of political processes along the various stages in the policy life-cycle” [32: 145]. Hence an e-tool should not be considered apart from the level of its institutionalization into the decision-making system [4]. The development of participatory design and its evaluation is becoming an imperative of e-participation evaluation [20], going beyond *perceived usefulness* and *ease of use* to consider “procedural and institutional context” [21: 22]. As technology develops, current issues are to be considered, e.g. mobile and social media dimensions [26].

We understand the *institutional design* here broadly as a set of rules that shape the behavior of involved actors [10]. Each feature employed on a website, as well as legal and other norms that connect it to the decision-making, are to be viewed as (1) results of equilibrium that reflects the preference of actors and (2) sources of information for actors on the outcomes of their actions, clarifying their incentives and costs to act [13]. But to assess causes and effects of these institutional configurations, one should first explore the variety of choices available, and link them to the expected outcomes.

There have been several methodologies proposed so far that link institutional design of e-participation to democratic values and effective decision-making. To mention a few, Östling found the features of e-petition portals institutional design (e.g. moderation) that heavily impact their democratic quality in terms of *equality, accountability, freedom and responsiveness* [30]. Bryson et. al distinguish several design features that should be met, i.e. *legal requirements, inclusiveness, social justice, public information* etc. and map them to required procedures and measurable outcomes [2]. Another approach, developed by Smith, suggests that democratic innovations should reflect the values of *inclusiveness, popular control, considered judgment and transparency* that allow citizens to participate and scrutinize authorities [33].
Based on these findings, we suggest that the institutional design of e-participation should meet the following crucial requirements: (1) *access*, i.e. the incentives for citizens to be actively involved in e-participation processes; (2) *embeddedness*, i.e. the ways e-participation activities are entrenched into the bureaucratic process and political system, incentivizing decision-makers; (3) *control*, i.e. institutions that provide transparency and public control over bureaucratic performance.

In case of *access*, e-participation should, first, lower the barriers for citizens to engage, as it was initially meant to be working [27]. However, there is a plethora of evidence that digital and participatory divides remain persistent hurdles [1, 31]. We suppose that the problem of *access* should not be considered a mere exogenous factor, like the Internet-penetration, skills and human capital, but it needs to be dealt internally by designing inclusive e-participation [29]. Potential participants are to be provided with on-site support, FAQ and search facilities to start. Secondly, e-participation should also expand its reach using new technological affordances, like the social media or smartphones [22]. Thirdly, e-participation tools should not motivate tokenism on the first place, but to stimulate constant engagement and communication.

But this is not enough. Many e-participation initiatives have been criticized as incapable of brining citizens into decision-making, remaining consultative, if not symbolic structures [6]. While civic activism online is growing and does impact policies in many respects, this mostly cannot be considered a success of the formal channels. An effective e-participation institutional design needs to be embedded into the decision-making via regulations bridging networked online activism structures and hierarchical bureaucratic system, converting citizens’ aspirations into the signals taken for actions. Therefore, we expect e-participation to be more effective if there are clear legal regulations, defined workflow algorithms and other features facilitating the work of public officials.

Finally, citizens must have an opportunity to evaluate the way their demands are converted into policy outputs once they disappear in the “black box” [20]. The ability to control the government externally and see if it is responsive or transparent may rise what is called *efficacy* – perceived ability to influence government and be heard [8]. The more successful citizens’ interactions with the government are, the more likely they continue to be involved. At the same time, transparency and control mechanisms can refrain decision-makers from neglecting citizens. This transparency and control can be achieved through reports and evaluations open for public scrutiny.

In general, we suppose the institutional design of e-participation that has the above-mentioned features may provide a correct equilibrium to make the tool working, contributing to human development, democracy and bureaucratic efficiency. It gives positive incentives for citizens to constantly engage with the government, raising their efficacy and waiving costs of participation. At the same time officials also get necessary incentives: *embeddedness* helps to build new tools in the ordinary workflow, while control mechanisms impose costs on non-compliance.

## 2.2 Operationalization of *Access*, *Embeddedness* and *Control*

The abovementioned dimensions need to be operationalized in the way they can be assessed quantitatively, as the availability of different website features and legal norms.
To evaluate e-participation portals, we have checked the availability of the following
elements:

*   **Access:**

- System of registration;

- Pre-moderation of appeals;

- Personal user's account;

- System of rankings or statutes for users of the system;

- Q&A system;

- Feedback and support from developers;

- FAQ facility;

- Search engine;

- Mobile application or / and mobile version;

- Accounts in the social media.

*   **Embeddedness:**

- The regional legislation defining the procedures of citizens' appeals processing;

- The defined algorithm of how appeals are handled;

- The classifier that helps citizens to identify the type of their address when submitting and links it to a certain mode of governmental actions.

*   **Control:**

- The opportunity for citizens to evaluate their experience of interaction with the government on the website;

- The opportunity for users to evaluate other citizens' appeals;

- Official reports on the results of governmental handling of citizens' appeals;

- Openness of citizens' appeals to other users;

- The “public control” function, i.e. the system allows citizens double-checking and reporting if the government has really handled the request properly;

- Maps or GIS systems that may be used for detecting the problem geographically.

# 3 A Pilot Study of Regional E-Participation in Russia

## 3.1 E-Participation in Russian Regions: An Overview of Institutionalization Process

The first attempts to build up e-participation in Russia started in 2002 with the Federal Program "Electronic Russia". In 2006, Federal Law 59 was adopted to regulate the terms and procedures of processing citizens' appeals, but it did not provide an opportunity to do it electronically either. The situation changed in 2010 with a new wave of administrative reform, emphasizing the necessity to introduce the electronic workflow. Then Federal Law 59 was amended to introduce e-appeals. That proposition opened space for regional governmental and non-governmental e-participation platforms.

The next step in e-participation institutionalization in Russia started in 2012, with President Putin's proposition to develop the Internet democracy and the imperative of taking citizens' opinion in decision-making seriously, especially on the municipal and
regional levels.<fnref n="1" /> It became a signal for subnational political elites to implement various feedback mechanisms, public consultations, and special information services. One of the leaders here was Moscow, which introduced an elaborate e-participation platform in 2012, and many regions followed this pattern.

A current stage of e-participation institutional development is connected to another federal initiative, according to which all regions now must have their own e-participation channels. In 2017 a Presidential Decree was adopted that makes all public bodies report to the Presidential Administration on their work with citizens' appeals.<fnref n="2" /> The scheme of e-participation workflow in Russia is presented on fig. 1.

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Institutional Scheme of Regional E-Participation. Source: Authors' compilation of legal documents.</figcaption></figure>

<footnote n="1">¹ Putin V.V. Democracy and Quality of Government. Kommersant. 06.02.2012. URL: http://kommersant.ru/doc/1866753 [in Russian]</footnote>

<footnote n="2">² Decree of President No 171 "On Monitoring and Analysis of Citizens and Organizations' Appeals Results" (17.04.2017).</footnote>
According to the compiled scheme, a citizen can address their claims to both non-governmental and governmental platforms, though it should be noted that only the latter are formalized by the law and hence perceived obligatory for authorities. Submitted complaints are then go to the respective regional authorities that are supervised by the federal agencies, as well as the governor. Once the appeal is processed, the governmental body must report on it to the applicants, as well as submit it to the federal information system that collect all data on regional and federal agencies' compliance and responsiveness. At the same time, while governors are formally key decision-makers in the regions, the influence of the federal regulation is overwhelming. Hence, the system of e-participation in Russia is quite government-centered, i.e. the control mechanisms are based not on the public scrutiny, but on the top-down subordination principles. Considering the so-called power vertical in Russia, a hierarchical system of governance with the Presidential Administration as a vertex [9], those federal requirements have become a serious impetus for regional authorities to introduce and develop e-participation.

It seems, though, that the main goal of those initiatives is not to empower citizens, but to strengthen capabilities of the federal government to control regional ones, and to respond quicker to the potential hotbeds of social tension. The control imperative is clearly seen in successful regional e-participation portals as well [18]. Considering the Russian political system, it appears the only way to make e-participation work for citizens as well, when the vertical control can be a source of responsiveness [3].

## 3.2 Preliminary Results of Evaluation

Despite the overall federal strategy, there might be a substantial variation within 85 regions in the institutional design and its correspondence to the features outlined above. To explore this variety and to assess the validity of this framework to study e-participation, we have conducted a pilot evaluation of the regional e-participation portals in Russia: regional websites that deal with collecting and handling citizens' appeals and complaints to the governmental bodies. Surely, such assessment can be quite rough, but at least it may give an overview for further analysis.

For a pilot survey we have monitored 85 e-participation websites of all regions of Russia, as well as the available regional legislation, according to the operationalization specified in section 2.2. The analysis was carried out in September-November 2017. During the process of monitoring several difficulties were encountered. First, it was not always easy to define the absence or availability of certain features, since all websites have different structures and navigation systems. Some e-participation tools are situated on the main regional governmental portals while the other have their own domains. To assess some of the internal features, like the classifier or user's account, we sometimes had to register into the system. Some websites were in the process of reconstruction, and although a tool might have been available, it didn't work properly.

If a certain feature from the list was available, a region got a score of "1", and "0" in case of absence. All scores were then computed to make an average score for (1) *access*, (2) *embeddedness* and (3) *control*, as well as a total index.
The aggregate results for the index and its components can be found in Table 1, while an interactive map with data for all 85 regions is available online,<fnref n="3" /> as well as the dataset<fnref n="4" />. Overall, the regions are quite different in e-participation institutionalization. The total index mean is 0.5 – which means that on average nearly half of the features we looked for are missing on the website. The most frequent total score is 0.4, which corresponds to about 8 features out of 19. The *embeddedness* component seems, at least on paper, to be the most successful: many regions have adopted specific legislation to formalize the workflow on the portals. However, we could not find any special legal footing for 26 regional e-participation tools, and the classifier of citizens’ appeals is the rarest thing with only 25 subjects having them. Again, we cannot judge whether bureaucracies exactly follow the law if they have one, but at least this is a crucial step towards decreasing bureaucracy discretion.

The *access* component is also performing quite satisfactorily, and few regions do not have any features that stimulate citizens’ engagement. However, the configuration varies deeply. While many regions introduce moderation and registration system, only a few have users’ accounts and rankings that would stimulate more active involvement. Hence, participation instruments still usually act as “complaint boxes”. Also, only 39 regions have mobile apps or versions, while 45 subjects have official accounts in the social media (at least that have links to them on the portals).

The worst situation, as was anticipated, is with the *control* component. The average score here is only 0.3 out of 1, while the mode is even lower. Mostly, complaints and addresses go to the “black box” of government, and only complainants receive the results of their handling. The rest of the citizens are usually unable to see both the appeals themselves and the aggregate reports of how the government acts. Double-checking of governmental activities via public control remains a rare practice. Citizens’ inability to view transparently governmental activities in relation to e-participation put the effectiveness of such mechanisms under serious doubts.

There are certain groups of regions that can be characterized as leaders and outsiders in e-participation institutionalization. Their classification has been done using a cluster analysis (k-means) to form four groups of regions, which are shown in Table 2 with some examples. The first cluster comprise of the least developed regions in terms of their e-participation institutionalization efforts. The most distinctive feature of them is the lack of *embeddedness* of e-participation, since they usually do not have any special regulation or clearly visible algorithm of the workflow. The *access* and *control* dimensions are also weak. On the contrary, the fourth cluster represents the leaders in e-participation development, like Moscow, St. Petersburg and Tatarstan, which have always been taking the highest positions on e-government and e-participation development [4, 18]. The second and third clusters are the most inhabited clusters somewhere in the middle. They are quite close in performance, but the third cluster is lagging because these regions mostly lack proper legal footing. It should be noted that the values are distributed quite normally, which makes them compatible with the widespread quantitative methods, like regression analysis.

<footnote n="3">³ http://qoo.by/47Tx</footnote>

<footnote n="4">⁴ goo.gl/o2kNxW</footnote>
In general, this pilot survey has revealed the opportunity to operationalize and measure our model, grasping the variety of e-participation institutional designs. This opens a way to a further deeper analysis of choices public officials make and outcomes as- sorted designs produce.

<table><caption>**Table 1.** Descriptive Statistics of the E-Participation Institutionalization Index (Russian Regions). Source: Authors' calculations</caption><thead><tr><th></th><th>Cases</th><th>Mean</th><th>Mode</th><th>Standard Deviation</th></tr></thead><tbody><tr><td>Total Index</td><td>85</td><td>0,46</td><td>0,4</td><td>0,19</td></tr><tr><td>Access</td><td>85</td><td>0,52</td><td>0,7</td><td>0,22</td></tr><tr><td>Embeddedness</td><td>85</td><td>0,58</td><td>0,7</td><td>0,34</td></tr><tr><td>Control</td><td>85</td><td>0,3</td><td>0,2</td><td>0,24</td></tr></tbody></table>

<table><caption>**Table 2.** The Classification of Regions Based on K-Means Cluster Analysis. Source: Authors' Calculations</caption><thead><tr><th rowspan="2">Cluster Center Score</th><th colspan="4">Cluster</th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th></tr></thead><tbody><tr><td>Access</td><td>0,31</td><td>0,51</td><td>0,56</td><td>0,82</td></tr><tr><td>Embeddedness</td><td>0,05</td><td>0,74</td><td>0,21</td><td>1</td></tr><tr><td>Control</td><td>0,14</td><td>0,25</td><td>0,31</td><td>0,76</td></tr><tr><td>Number of cases</td><td>17</td><td>47</td><td>10</td><td>11</td></tr><tr><td>Examples</td><td>Ivanovo, Kaluga, Ryazan, Sverdlovsk obl.</td><td>Tyumen, Yaroslavl, Belgorod obl.</td><td>Kalmykia, Dagestan, Komi, Smolensk obl.</td><td>Moscow, St. Petersburg, Tatarstan obl.</td></tr></tbody></table>

### 3.3 Exploring the Variety of Designs

We claim that our measurement is not a mere artifact of technological development, but an estimation of how e-participation is connected to the democratic and effective decision-making process. The deep analysis of causes and effects of the institutions we have explored is the next step we need to take during testing the validity of measurements. However, to preliminary assess the findings we run a correlation analysis of the indices we got with the variables that are commonly used to explain e-government and e-participation performance in Russia and worldwide [19, 23, 34, 36], namely the level of technological and human development, as well as the effectiveness of bureaucracy. We use the most current available data from the Russian Statistical Service (Rosstat). The regional technological development is operationalized by the Internet penetration rate (2016), the human development is assessed by the level of higher education (2010), the average income (2016) and the share of urban population (2016). The effectiveness of bureaucracy is the most difficult to approach, but we use the share of civil servants in the regional population (2015), which proves a good proxy for effectiveness in some
studies [19]. The results, based on the Pearson's correlation analysis, are shown in Table 3. The closer the score to 1, the stronger is the association between the variables, significance is measured at the 0,01 level (**) and 0,05 level (*).

Though correlation does not mean causality, it shows that the level of institutional development is significantly and positively associated with the technological and human development of a region, as well as with its bureaucratic performance. It basically suggests that we are close to quantifying the phenomenon we claim to measure. Technologically advanced, urbanized and less bureaucratized regions are more likely to innovate towards more accessible and transparent e-participation: in this regards, e-participation, citizens' empowerment and good governance may reinforce each other. Innovation might be a function of citizens' pressures, civil society, human capital and bureaucratic performance. At the same time, not all components seem to be associated with this dynamic, especially the *embeddedness*, making us think of agency rather structural factors. For instance, as has been shown previously, the successful institutionalization of e-participation in St. Petersburg was possible to a larger extent due to the policy entrepreneurship of government officials and strong political will of the governor himself [18]. The control imperative of the power vertical should also be considered as a factor, and we may hypothesize that the more region is dependent on the federal support, or the weaker it is vis-à-vis the national government, the more efforts are made to embed e-participation as a formal procedure. This calls for a research perspective that combines quantitative assessment with in-depth case study.

<table><caption>**Table 3.** The Results of the Correlation Analysis. Source: Authors' calculations</caption><thead><tr><th></th><th>Total Index</th><th>Access</th><th>Embeddedness</th><th>Control</th></tr></thead><tbody><tr><td>Internet Penetration</td><td>,378**</td><td>,427**</td><td>,190</td><td>,253*</td></tr><tr><td>Higher Education</td><td>,409**</td><td>,402**</td><td>,234*</td><td>,337**</td></tr><tr><td>Urban Population</td><td>,312**</td><td>,311**</td><td>,202</td><td>,229*</td></tr><tr><td>Average Income</td><td>,207</td><td>,235*</td><td>,062</td><td>,116</td></tr><tr><td>Bureaucracy</td><td>-,293**</td><td>-,256**</td><td>-,149</td><td>-,290**</td></tr><tr><td>Effectiveness<sup>(1)</sup></td><td></td><td></td><td></td><td></td></tr></tbody></table>

Note: ** - Pearson correlation is significant at the 0,01 level (2-tailed), * - Pearson correlation is significant at the 0,05 level (2-tailed); (1) – the inverse relationship should be read the way that the less bureaucratized a region is the most effective.

## 3.4 Discussion: Limitations and Future Research

Our findings contribute to the studies that emphasize the importance of institutional context for e-participation performance and call for careful designing e-participation platforms to ensure intended outcomes. We provide a framework and a tool that can measure quantitatively the variance of intuitional features employed, at the same time discerning aspects for deeper qualitative study. However, the estimation is quite rough so far and needs further improvement.
The first major limitation of the proposed framework is that it neglects the role of informal institutions, that is, unwritten but observed rules of behavior, which are no less important, as the new institutionalism suggests. Informal norms are usually deeply embedded and may conflict with formal institutional innovations to totally subvert their positive effects to maintain status quo [17]. For instance, corruption and informal links that structure bureaucratic behavior may persist regardless of how successful e-participation arrangements seem on paper and on websites.

The second limitation is quite a legalist understanding of *embeddedness* as a formal inclusion of e-participation into the normatively defined workflow, which might indeed be quite far from the reality, influenced by political power relationships. The third limitation is the so-called design-reality gap that is observed in e-projects, especially in the developing countries. Although e-participation might be designed the way it properly incentivizes citizens and bureaucracy, it does not guarantee its successful implementation due to managerial, financial and human factors [16]. We may theorize and speculate on how the institutional design foster e-participation, but we cannot be sure about its outcomes until we empirically test its effects.

The future research may be thus related to minimization of the abovementioned drawbacks. First, we need to elaborate our technique by purifying the operationalization of *access, embeddedness and control*. New measurements of inclusiveness may be included, like the availability of features that enable participation for people with disabilities, provide incentives to discussion, deliberation and community building. A more thorough view on the legislation and algorithms would also contribute to the estimation. Also, we plan to measure not only the availability of different institutional features, but also the quality of them as well. The index can be normalized by the inclusion of corruption, democracy, and civil rights assessments to assess the importance of informal institutions. Alternatively, surveying public officials and citizens on their experience with e-participation tools may be worthwhile. Another fruitful way to assess *control* might be to run the experiment that would measure the true pace and quality of government officials' responses to citizens.<fnref n="5" /> Secondly, we may expect this index to become an empirical data in the analysis of factors that drive institutional innovation. In section 3.4 we have stated a promising possibility for such quantitative research, and specification of explanatory models would be useful.

# 4 Conclusion

The proposed framework, as well as the results of the pilot study, needs to be updated theoretically and empirically to overcome the limitations mentioned in the previous section. However, some conclusions can be drawn.

First, the framework and evaluation technique presented can be used both for large-N explaining perspective, as data is compatible with statistical analysis, and case-based or small-N understanding perspective that is a valuable to explore internal factors of e-participation *access, embeddedness and control* dimensions development. The value of

<footnote n="5">⁵ We thank an anonymous reviewer for this insightful suggestion.</footnote>
the framework, we argue, is in its new institutionalism core, since the paradigm is pop-
ular with various scientific fields (politics, sociology, economics etc.), which contrib-
utes to the multidisciplinary mixed-method approach. Furthermore, we have tried to
move away from the stage-model to more positivist view of e-participation here and
now. Operationalization has taken into account the recent development of e-participa-
tion, like the social media and mobile technologies, as well as a broader look on legal
documents that back e-participation, going beyond websites.

Secondly, our pilot study of Russia, currently being institutionalized e-participation instruments, has shown that despite an overall federal strategy and hierarchical imperatives of the “power vertical”, regions perform a substantial variety of outcomes. The key problem seems to be the lack of the control from citizens and governmental transparency, which puts a question, quite rhetorical, if citizens or federal authorities – are the key beneficiaries of innovations. However, the institutional innovations continue which opens space for more active civic engagement in policy-making. The preliminary correlation analysis has revealed some structural factors that might explain the variation (like socio-economic conditions, technological readiness and bureaucratic efficiency), but the role of agency should be explored as well.

Finally, the proposed framework and measurement, we suppose, can be refined and
adapted to study other cases, especially in case of subnational (regional or municipal)
e-participation development. They can be used not only for scientific purposes but em-
ployed by policy-makers as a benchmarking tool to see where they lag behind. In gen-
eral, we hope that the paper will open further discussion on the role institutions play in
contemporary e-participation development.

**Acknowledgements**

The research is conducted with the support of the Russian Science Foundation grant №
18-18-00360

**References**

1. Best, S.J., Krueger, B.S.: Analyzing the representativeness of internet political participation. Polit. Behav. **27**, 183–216 (2005). doi: 10.1007/s11109-005-3242-y

2. Bryson, J. M., Quick, K. S., Slotterback, C. S., Crosby, B. C.: Designing public participation processes. Pub. Adm. Rev., **73**, 23-34 (2013). DOI: 10.1111/j.1540-6210.2012.02678.x

3. Chen, J., Pan, J., Xu, Y.: Sources of authoritarian responsiveness: A field experiment in China. Am. J. Pol. Sci. **60**, 383-400 (2016). DOI: 10.1111/ajps.12207

4. Chugunov, A. V., Kabanov, Y., Misnikov, Y.: Citizens versus the government or citizens with the government: a tale of two e-participation portals in one city-a case study of St. Petersburg, Russia. In: Proceedings of the 10th International Conference on Theory and Practice of Electronic Governance, ICEGOV, pp. 70-77, ACM (2017). DOI: 10.1145/3047273.3047276

5. Chugunov, A.V., Kabanov, Y., Zenchenkova K.: Exploring Regional Variance in Use. In: Tambouris E. et al. (eds) Electronic Participation. ePart 2016. Lecture Notes in Computer Science, vol 9821, pp. 109-122, Springer, Cham (2016). DOI: 1007/978-3-319-45074-2_9
6. Coleman, S.: Can the Internet Strengthen Democracy. Polity (2017).

7. Coursey, D., Norris, D. F.: Models of e-government: Are they correct? An empirical assessment. Pub. Adm. Rev., 68, 523-536 (2008). DOI: 10.1111/j.1540-6210.2008.00888.x

8. Finkel, S.E.: Reciprocal effects of participation and political efficacy: A panel analysis. Am. J. Pol. Sci., **29**, 891-913 (1985). DOI: 10.2307/2111186

9. Gel'man, V., Ryzhenkov, S.: Local regimes, sub-national governance and the 'power vertical' in contemporary Russia. Europe – Asia Studies, **63**, 449-465 (2011). DOI: 10.1080/09668136.2011.557538

10. Goodin, R.E. (Ed.): The theory of institutional design. Cambridge University Press (1998)

11. Grönlund, Å.: Connecting egovernment to real government-the failure of the UN eparticipation index. In: Janssen M., Scholl H.J., Wimmer M.A., Tan Y. (eds) Electronic Governance. EGOV 2011. Lecture Notes in Computer Science, vol 6846, pp. 26-37, Springer, Berlin, Heidelberg (2011). DOI: 10.1007/978-3-642-22878-0_3

12. Grönlund, Å.: ICT is not participation is not democracy-eParticipation development models revisited. In: Macintosh A., Tambouris E. (eds) Electronic Participation. ePart 2009. Lecture Notes in Computer Science, vol 5694, pp. 12-23, Springer, Berlin, Heidelberg (2009). DOI: 10.1007/978-3-642-03781-8_2

13. Hall, P. A., Taylor, R. C.: Political science and the three new institutionalisms. Political Studies, **44**, 936-957 (1996). DOI: 10.1111/j.1467-9248.1996.tb00343.x

14. Hassan, S., Gil-Garcia, J. R.: Institutional theory and e-government research. In: Handbook of research on public information technology, 349-360 (2008)

15. Heeks, R., Bailur, S.: Analyzing e-government research: Perspectives, philosophies, theories, methods, and practice. Gov. Inf. Q. **24**, 243-265 (2007). DOI: 10.1016/j.giq.2006.06.005

16. Heeks, R.: Most eGovernment-for-development projects fail: how can risks be reduced? Manchester: Institute for Development Policy and Management, University of Manchester, vol. 14 (2003).

17. Helmke, G., Levitsky, S.: Informal institutions and comparative politics: A research agenda. *Perspectives on Politics*, **2**, 725-740 (2004). DOI: 10.1017/S1537592704040472

18. Kabanov, Y., Chugunov, A.V.: Electronic "Pockets of Effectiveness": E-governance and Institutional Change in St. Petersburg, Russia. In: Janssen M. et al. (eds) Electronic Governance. EGOV 2017. Lecture Notes in Computer Science, vol 10428, pp. 386-298, Springer, Cham (2017). DOI: 10.1007/978-3-319-64677-0_32

19. Kabanov, Y., Sungurov, A.: E-Government Development Factors: Evidence from the Russian Regions. In: Chugunov A., Bolgov R., Kabanov Y., Kampis G., Wimmer M. (eds) Digital Transformation and Global Society. DTGS 2016. Communications in Computer and Information Science, vol. 674, pp. 85-95, Springer, Cham (2016). DOI: 10.1007/978-3-319-49700-6_10

20. Kim, S., Lee, J.: E-participation, transparency, and trust in local government. Pub. Adm. Rev., **72**, 819-828 (2012). DOI: 10.1111/j.1540-6210.2012.02593.x

21. Kubicek H., Aichholzer, Closing the Evaluation Gap in e-Participation Research and Practice. In: Aichholzer G., Kubicek H., Torres L. (eds) Evaluating e-Participation. Public Administration and Information Technology, vol 19, pp. 11-45, Springer, Cham (2016). DOI: 10.1007/978-3-319-25403-6_2

22. Lacigova, O., Maizite, A., Cave, B.: eParticipation and Social Media: a Symbiotic Relationship? European Journal of ePractice, **16**, 71-76 (2012).

23. Lee, C. P., Chang, K., & Berry, F. S.: Testing the development and diffusion of e-government and e-democracy: A global perspective. Pub. Adm. Rev. **71**, 444-454 (2011). DOI: 10.1111/j.1540-6210.2011.02228.x
24. Lidén, G.: Technology and democracy: validity in measurements of e-democracy. Democratization, **22**, 698-713 (2015). DOI: 10.1080/13510347.2013.873407

25. Lindner, R., Riehm, U.: Broadening Participation Through E-Petitions? An Empirical Study of Petitions to the German Parliament. Policy & Internet, **3**, 1-23 (2011). DOI: 10.2202/1944-2866.1083

26. Macintosh A., Coleman S., Schneeberger A.: eParticipation: The Research Gaps. In: Macintosh A., Tambouris E. (eds) Electronic Participation. ePart 2009. Lecture Notes in Computer Science, vol 5694, pp. 1-11, Springer, Berlin, Heidelberg (2009). DOI: 10.1007/978-3-642-03781-8_1

27. Macintosh, A.: Characterizing e-participation in policy-making. In: Proceedings of the 37th Annual Hawaii International Conference on System Sciences, IEEE, pp. 1-10 (2004). DOI: 10.1109/HICSS.2004.1265300

28. Meijer A., Bekkers V.: A metatheory of e-government: Creating some order in a fragmented research field. Gov. Inf. Q. **32**, 237-245 (2015). DOI: 10.1016/j.giq.2015.04.006

29. Olphert, W., & Damodaran, L.: Citizen participation and engagement in the design of e-government services: The missing link in effective ICT design and delivery. J. of the Assoc. for Inf. Sys., **8**, 491-507 (2007)

30. Östling, A.: How democratic is e-participation? In: Conference for E-Democracy and Open Government, CeDEM11, Proceedings of the International Conference for E-Democracy and Open Government, 5-6 May 2011, Danube University Krems, Austria, pp. 59-70 (2011)

31. Scheerder, A., van Deursen, A., van Dijk, J.: Determinants of Internet skills, use and outcomes: A systematic review of the second- and third-level digital divide. Telematics and informatics, **34**, 1607-1624 (2017). DOI: 10.1016/j.tele.2017.07.007

32. Scherer, S., Wimmer, M.A.: Reference Framework for E-participation Projects. In: Tambouris E., Macintosh A., de Bruijn H. (eds) Electronic Participation. ePart 2011. Lecture Notes in Computer Science, vol 6847, pp. 145-156, Springer, Berlin, Heidelberg (2011). DOI: 10.1007/978-3-642-23333-3_13

33. Smith, G.: Democratic innovations: Designing institutions for citizen participation. Cambridge University Press (2009)

34. Tolbert, C. J., Mossberger, K., & McNeal, R.: Institutions, policy innovation, and E-Government in the American States. Public administration review, **68**(3), 549-563 (2008). DOI: 10.1111/j.1540-6210.2008.00890.x

35. Wright, S., Street, J.: Democracy, deliberation and design: the case of online discussion forums. New Media & Society, **9**, 849-869 (2007). DOI: 10.1177/1461444807081230

36. Zhang, H., Xu, X., & Xiao, J.: Diffusion of e-government: A literature review and directions for future directions. Gov. Inf. Q., **31**, 631-636 (2014). DOI: 10.1016/j.giq.2013.10.013

37. Tambouris, E., Liotas, N., & Tarabanis, K.: A Framework for Assessing eParticipation Projects and Tools. In: System Sciences, 2007. HICSS 2007. 40th Annual Hawaii International Conference, IEEE, pp. 90-90 (2007). DOI: 10.1109/HICSS.2007.13

38. Garcia, A. C. B., Maciel, C., & Pinto, F. B.: A quality inspection method to evaluate e-government sites. In: International Conference on Electronic Government, IFIP EGOV, 2005, Springer, Berlin, Heidelberg, pp. 198-209 (2005). DOI: 10.1007/11545156_19

39. Panopoulou, E., Tambouris, E., & Tarabanis, K.: A framework for evaluating web sites of public authorities. Aslib Proceedings, **60** (5), 517-546 (2008). DOI: 10.1108/00012530810908229