# An Intercountry Survey of Participatory Practices Used for Open Government Partnership National Action Plan Development

Athanasia Routzouni, Athanasios P. Deligiannis, Vassilios Peristeras, Stefanos Gritzalis

► To cite this version:

Athanasia Routzouni, Athanasios P. Deligiannis, Vassilios Peristeras, Stefanos Gritzalis. An Intercountry Survey of Participatory Practices Used for Open Government Partnership National Action Plan Development. 18th International Conference on Electronic Government (EGOV), Sep 2019, San Benedetto del Tronto, Italy. pp.82-93, 10.1007/978-3-030-27325-5_7. hal-02445794

HAL Id: hal-02445794

https://inria.hal.science/hal-02445794

Submitted on 20 Jan 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# An intercountry survey of participatory practices used for Open Government Partnership National Action Plan development

Athanasia Routzouni<fnref n="1" />*[0000-0002-7120-1067]✉, Athanasios P. Deligiannis<fnref n="2" />*[0000-0003-4998-750], Vassilios Peristeras<fnref n="3" />[0000-0002-4889-3333] and Stefanos Gritzalis⁴[0000-0002-8037-2191]

**Abstract.** This paper maps the participatory process applied during the development of open government commitments by twenty-nine Open Government Partnership countries. It investigates the role and perceived value of e-participation practices and their relationship to inclusive interaction with stakeholders. The insights of the paper on the perceived value of the tools and methods used and the main impediments to the open government commitment creation process may help governments more effectively design their public participation efforts and increase the level of civic engagement in policy making. Although the results of this analysis derive from activities related to open government reforms, they may also be applicable to introducing participatory public policy formulation approaches in a broader range of policy domains.

**Keywords:** open government, public policy co-creation, e-participation, Open Government Partnership.

**Track:** General E-Government & Open Government

## 1 Introduction

This paper attempts to map participatory practices that are used by governments when designing public policy, specifically open government commitments. It analyzes public participation practices and tools that have been applied by twenty-nine governments that were active in the Open Government Partnership (OGP) during the development of National Action Plans (NAPs) with specific commitments on open government issues.

The presented practices have been applied to support an effective dialogue between government and civil society for policy formulation. Successful ones can be used as a tool to strengthen public accountability and transparency in government operations in

<footnote n="1">¹,⁴ University of the Aegean, Karlovassi, Samos 83200, Greece</footnote>

<footnote n="2">² International Hellenic University, Thessaloniki 57001, Greece</footnote>

<footnote n="3">³ European Commission, DG Informatics | International Hellenic University, Thessaloniki 57001, Greece</footnote>

<footnote>* These authors contributed equally to the work</footnote>

<footnote>nROUTZOUNI@AEGEAN.GR</footnote>
an era characterized by an erosion of trust in government [1]. The analysis of the perceived value of the applied practices along with the identification of the main impediments to the process may also help governments more effectively design their relevant public participation processes and increase the level of public participation in policy making.

Although the results of this analysis derive from processes related to open government reforms, they may also be applicable to introducing participatory public policy formulation approaches in a broader range of policy domains.

## 1.1 The concept of participation in public policy making

Embedding the principle of participation in public policy making is not a new concept. Arnstein [2] introduced in 1969 the “ladder of citizen participation” outlining different levels of citizen participation ranging from pretextual to truly meaningful. Medimorec [3] added a level, “codetermination”, in which citizens co-decide, usually with politicians and administrators. The concept of participation evolved to Manzini’s “map of participation” [4] which further describes the involvement of different actors in the design process along with their interaction. In light of research that points to a loss of social capital [5] combined with a decline in trust of government [6], civic engagement is seen as a necessary prerequisite and central component of vital democracies.

The most common approaches to citizen engagement include: electoral approaches (running for office, volunteering at campaigns), legislative and administrative information exchange approaches (public hearings), civil society approaches (participation via volunteer organizations, social clubs, and other forms of association) and deliberative and consensus-based approaches (joint action across sectors of society, classes of people, or types of individuals) [7].

An open, participatory approach to government is suggested to restore legitimacy to government and improve trust in its operation by enhancing accountability [8, 9] and attenuate corruption [10]. By consulting with citizens, government can improve transparency, access and service quality [11].

Enabling meaningful civic participation is seen as a fundamental requirement of well-functioning democracies and should be a priority for governments [12]. The OECD recognizes three levels of participation: information provision, which although a one way flow from the government to a two-way interaction that enables citizens to provide feedback; and active citizen participation, in which citizen's engagement is requested for defining and shaping policies leaving the final decision under the responsibility of the government [14, 15].

An alternative citizen participation spectrum is offered by the International Association for Public Participation (IAP2) [16] with five levels: 1) Inform, which entails providing the public with the necessary information on a given issue; 2) Consult, describing efforts at getting simple feedback from the public; 3) Involve, which describes efforts to engage with the public in more interactive ways to more deeply address their concerns; 4) Collaborate, which entails a meaningful partnership with the public in each aspect of the decision making process utilizing co-creation methods; and 5) Empower, a stage that includes co-creation activities and puts the final decision-making in the
hands of the public as well [17]. The IAP2 public participation spectrum [18] is used
by OGP to classify the quality of public participation.

1.2 E-participation in public policy making

There is significant evidence that the use of Information and Communications Tech-
nology (ICT) can expand opportunities for civic engagement and increase opportunities
for people to participate in decision-making processes and service delivery [19]. For
these purposes e-participation can be defined as the “process of engaging citizens
through ICTs in policy, decision-making, and service design and delivery to make it
participatory, inclusive, and deliberative” [20].

According to OECD guidelines [21] e-participation is dependent on three main con-
ditions: (i) a clear focus on policies, decisions and governance practices to align them
with to people’s needs; (ii) particular attention to the means of interaction – communi-
cation channels should be readily made available for people to offer their views and
communicate among themselves as peers and with public authorities as partners; and
(iii) clear focus on the characteristics of the interaction process between citizens and
government to ensure the quality and legitimacy of e-participation efforts [22].

1.3 The participation principle in formulating open government public policies

Open and participative governance has evolved into a central pillar in the effort to modernize public administration at a global level [23]. Formulating public policies on empowering citizens, enhancing transparency, promoting integrity and accountability are central to government efforts for administrative reforms. In parallel, many international, European and national initiatives assist governmental efforts to promote horizontal government policies on Open Government and Innovation [24].

Public policy formulation is part of the pre-decision phase of public policy making. This phase involves identifying a range of policy alternatives and specifying policy tools and solutions to address the challenges within the public policy agenda [25].

One of the most notable initiatives to support the global open government agenda is the Open Government Partnership (OGP), a multi-stakeholder platform launched in 2011, which in 2019 brings together 79 countries that have committed to making their governments more open and accountable [26]. Recognizing that the process of formulating open government policies involves a wide network of stakeholders and policy-makers, OGP promotes effective collaboration between governments and civil society organizations in open government policy making [27].

Each OGP participating country delivers bi-annually a National Action Plan (NAP) [28] which is the product of a co-creation effort between the government and civil society to develop concrete, time-bound and measurable open government commitments. Engaging civil society, citizens, and other stakeholders is considered a core element throughout the developing, implementing, monitoring and reporting processes of the action planning cycle [29].
The action plan evaluation process is comprised of two activities; progress reports produced by country local researchers working for the Independent Reporting Mechanism of the OGP and self-assessment reports produced by the participating governments. OGP's Participation and Co-creation Standards set out requirements for engaging civil society, citizens, and other stakeholders throughout the OGP process. These standards also guide the content to be included in Self-Assessment Reports and include feedback mechanisms, space for co-creation, and facilitating a multi-stakeholder forum [30].

OGP also uses the International Association for Public Participation's (IAP2) “Participation Spectrum” to define the levels of citizen participation in developing an open government action plan. As already mentioned, the spectrum's participation levels are: Inform, Consult, Involve, Collaborate and Empower — government and civil society make joint decisions [31].

## 2 Methodology

The methodology of the present study includes analysis of the responses to a survey questionnaire which collected information on practices and tools applied during the development of OGP National Action Plans. These responses have also been compared to the results of the assessment made by the OGP Independent Reporting Mechanism on the levels of participation during action plan creation, as well as indicators such as the IAP2 spectrum for each country.

### 2.1 Participants

The participants of this study were members of the official network of National Points of Contact (PoC) in OGP participating countries. In each participating government, the PoC is the person responsible for coordinating the government's activities in relation to OGP [32]. PoC's are working-level counterparts to a ministerial-level representative. They are direct advisors to the national representative of each country to OGP and are usually drawn from the civil service of each country. During the development of a National Action Plan, the PoC engages with civil society and other stakeholders and co-operates with Ministries and other government organizations involved in the agenda that emerges during the co-creation process. A PoC also engages with OGP Support Unit to receive information on the OGP Participation and Co-creation Standards and on international best practices that could be applicable to the national policy agenda.

Twenty-nine PoCs responded to the questionnaire which corresponds to 36.7% of the total OGP countries in 2018. The countries represented are: Argentina, Armenia, Brazil, Chile, Colombia, Finland, France, Germany, Greece, Israel, Italy, Jordan, Lithuania, Luxembourg, Malawi, Malta, Mongolia, Nigeria, Panama, Paraguay, Philippines, Republic of North Macedonia, Romania, Serbia, Sierra Leone, Slovakia, the Czech Republic, the Netherlands and Ukraine.
## 2.2 Research sources and procedures

### Survey questionnaire on the co-creation of OGP National Action Plans

An original electronic survey questionnaire based on the OGP Participation and Co-creation Standards [33] and recent literature on co-creation methods and tools [34-37] was created to examine the co-creation approaches that OGP participating countries have applied to develop open government National Action Plans. A pilot study was conducted to verify the validity of the structure and content of the questionnaire. Five officials from OGP, working for the Support Unit and the Independent Reporting Mechanism, examined the questionnaire. The comments provided were taken into consideration for the final version. The online questionnaire was subsequently distributed to the official network of the PoCs in all 79 OGP participating governments.

The questionnaire explored the latest NAP creation period (2016-2018) and aimed to collect information related to the following key factors of the co-creation process: who initiated the co-creation process within government; the profile of participating stakeholders and the level of their contribution; the process objectives in each government; the characteristics of the engagement approach; the co-creation methods and tools employed; the number of participants in the co-creation activities; the impediments to the process; and the perceived contribution of the co-creation process on improving the completion rate of the Action Plan reforms.

### Use of e-participation methods and tools in the co-creation process

The questionnaire also aimed to identify ICT-supported methods or tools that were used in the co-creation process. Specifically: use of a virtual platform for stakeholder engagement; use of ICT-supported co-creation methods (including social network analysis, online public consultations); use of ICT-supported co-creation tools (including consultation platforms, social media accounts, mobile contribution application, web discussion forums, online surveys); and value of including face to face meetings versus virtual / distance approaches in the co-creation process.

### Comparisons between questionnaire responses and OGP assessment results

OGP evaluates country performance on their action plan creation process using both its Independent Reporting Mechanism (IRM) as well as other openly available indicators. OGP then publishes independent progress reports for each participating country via the IRM. For each National Action Plan, the report summarizes the assessment of the development process and the country's progress in fulfilling key open government principles. OGP also releases, in open formats, the data underlying these reports [38, 39]. OGP data include an assessment of how each country has implemented the OGP consultation guidelines and to what extent each government works together with the civil society to develop the National Action Plan. OGP has published co-creation process assessment data for eighteen out of the twenty-nine countries that filled-in the OGP Co-Creation Questionnaire. The responses have been compared to the following indicators derived from already available OGP assessment data: *Level of public influence* using
IAP2 spectrum; "Open participation" or "Invitation only" approach during consultation; and Consultation steps followed via six indicators that represent the main steps to be followed during Action Plan development (Availability of timeline prior to the consultation; Adequate advance notice; Awareness raising activities; Online consultation; In person consultations with the national community; Online publication of a summary of the received comments).

# 3 Results

## 3.1 Analysis of the survey questionnaire responses

According to the responses to the questionnaire, the surveyed PoCs indicate that in 55 percent of the cases the government itself initiated the Action Plan development process while in 7 countries (24%) the process has been initiated by the government together with the Civil Society. In the rest of the cases (6 countries, 21%), the process was started by a multi-stakeholder forum.

The top identified reasons for using co-creation methodologies are related to strengthening the role of the stakeholders in the process (Engage stakeholders 97%; Increase trust between stakeholders 83%; Identify stakeholder needs 72%; Create a sense of ownership with stakeholders 66%). Raising awareness and communication have also been important factors in the process (Raising the level of awareness on different topics 59%; Improving communication processes 56%). It is worth noting that only one country replied that the required OGP co-creation criteria have been a reason for using co-creation methodologies in the process.

A significant percentage of countries (41.4%) have involved more than 100 participants, while 37.9% have involved more between 31 and 100 participants and 17.2% involved smaller groups of 1-30 people. The participating PoCs (72%) stated that civil society had a high contribution during the co-creation phase (in a scale from 1.Very much to 5.Not at all, 1: 48%; 2: 24%; 3: 24%; 4:3%, 5:0%).

The majority of participating PoCs (69%) stated that the co-creation process improves the content of the action plan (in a scale from 1.Very much to 5.Not at all, 1: 55%; 2: 14%; 3: 17%; 4:7%, 5:7%). More than half of the participants (58%) agreed that the co-creation process also improves the level of completion of the National Action Plan reforms (in a scale from 1.Very much to 5.Not at all, 1: 24%; 2: 34%; 3: 28%; 4:10%, 5:3%).

In terms of involving specific stakeholder groups, all PoCs (100%) responded that they usually engage civil society organizations and government departments. PoCs also involve academics (72%), citizens (55%), as well as the private sector and subnational government representatives (both groups at 72%). Parliament (41%) and the OGP Support Unit (38%) were less frequently involved while political parties have been totally absent from the co-creation process in all 29 countries.

Less than half of the countries (38%) have used e-participation platforms to involve stakeholders in the co-creation process. Except for one country that used a financial award scheme for the most active participating civil society organization, no rewards were used for co-creation participants.
The most frequently used co-creation methods include: SWOT workshops (43%), strategic roadmaps (29%), Living Labs (29%), appraisal interviews (14%), Personas (14%) and social network analysis (14%). To support the co-creation methods, the following tools have been primarily used by the countries included in the survey: Social media (83%), Crowd-mapping (35%), Mobile contribution applications (26%), Mind maps / Collaboration maps (17%), Issue cards (13%) and Motivational matrices (13%).

The participating PoCs indicated that the main impediments in the co-creation process have been the limited resources (66%) and the lack of time (52%) on the part of stakeholders as well as the difficulty in engaging stakeholders (38%). Other impediments include discontinuities caused by government changes and the electoral cycle (28%), difficulties in achieving consensus (21%) and a lack of trust in the effectiveness of the co-creation process (17%).

The OGP PoCs have also been asked to assess the significance of key factors in the co-creation process. The existence of permanent structures and processes, specifically the existence of permanent government agencies responsible for co-creation as well as the existence of a permanent stakeholder forum have been highlighted as key factors for the effectiveness of the process as shown in Figure 1.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Assessment of key factors in the co-creation process (Five-point Likert scale: 0=not important, 4=very important)</figcaption></figure>

Almost a quarter of countries introduced ICT-supported methods in the OGP action plan co-creation process (24%). The main e-participation methods used are online public consultations and social network analysis. Less than half of the countries (38%) have used virtual platforms to engage with stakeholders but more countries have used e-participation tools (69%) in the co-creation process. The main e-participation tools used are social media (68%) and mobile contribution applications (21%). Some PoCs have also reported the use of web discussion forums (4%), online surveys (4%) and consultation platforms (4%).

PoCs report that use of a virtual engagement platforms does not make a significant contribution to NAP content quality. The countries that used a virtual platform, (38% of the total) reported 1: 55%; 2: 9%; 3: 18%; 4:9%, 5:9% while the rest reported 1: 56%; 2: 17%; 3: 17%; 4:6%, 5:6% (in a scale from 1.Very much to 5.Not at all).

Adding face to face meetings during the co-creation process is reported by most PoCs (86%) as being of importance, more so than including virtual approaches in the co-creation process (55%).
## 3.2 Comparing questionnaire responses and OGP assessment results

Out of the 29 countries participating in this study, 18 participating countries have thus far been up assessed by OGP under the five levels of the IAP2 spectrum. They are classified in Table 1:

<table><caption>**Table 1.** IAP2 Level of public influence in the co-creation process of OGP countries</caption><thead><tr><td>IAP2 level</td><td>Countries</td></tr></thead><tbody><tr><td>Inform</td><td>No countries</td></tr><tr><td>Consult</td><td>Jordan, Lithuania, Malawi, Malta, Netherlands, Serbia</td></tr><tr><td>Involve</td><td>Armenia, Czech Republic, Italy, Sierra Leone</td></tr><tr><td>Collaborate</td><td>Brazil, Chile, North Macedonia, Mongolia, Paraguay, Romania, Ukraine</td></tr><tr><td>Empower</td><td>Greece</td></tr></tbody></table>

Comparing IAP2 levels with the use of e-participation tools reveals that of those countries belonging to lower IAP2 levels (Consult and Involve) half (50%) have used e-participation tools while 75% of the countries in the upper IAP2 levels (Collaborate and Empower) have done the same.

OGP PoCs have been asked to indicate whether the co-creation process improves the content of the Action Plan<fnref n="1" />. Half of the countries (50%) belonging to the lower IAP2 levels (Consult and Involve) stated that the process improves the content of the Action Plan (in a scale from 1. Very much to 5. Not at all, 1: 40%; 2: 10%; 3: 30%; 4:10%, 5:0%). A higher percentage (75%) of the countries belonging to the upper IAP2 levels (Collaborate and Empower) agreed that co-creation improves the content of the Action Plan (1: 50%; 2: 25%; 3: 13%; 4:0%, 5:13%).

When asked if the co-creation process improves the level of completion of the included commitments, 40% of countries in the lower IAP2 levels (Consult and Involve) responded positively (1: 30%; 2: 10%; 3: 50%; 4:10%, 5:0%). More countries (63%) of the ones belonging to the upper IAP2 levels (Collaborate and Empower) have done the same (1: 13%; 2: 50%; 3: 13%; 4:13%, 5:13%).

PoCs have also been asked to assess the contribution of civil society organizations during the co-creation phase. Half of the countries (50%) belonging to the lower IAP2 levels (Consult and Involve) agreed that CSO's contributed highly (1: 30%; 2: 20%; 3: 50%; 4:0%, 5:0%). Most of the countries (88%) of the countries belonging to the upper IAP2 levels (Collaborate and Empower) agreed on the significant contribution of the CSO's in the process (1: 25%; 2: 63%; 3: 0%; 4:13%, 5:0%).

Regarding possible impediments to the co-creation process, half of the countries (50%) that are classified under the upper levels of public influence (Collaborate, Empower) have identified “Difficulties in achieving consensus” as an impediment to their

<footnote n="1">¹ All the results reported in this section of the paper are on a scale from 1: Very much to 5: Not at all, unless otherwise specified in the text.</footnote>
process. On the other hand, no countries (0%) from the group of countries belonging to the lower IAP2 levels (Consult and Involve) recognized this impediment.

In terms of accessibility of the stakeholders to the consultation process, the 18 par-
ticipating countries that have been assessed by OGP are classified under two relevant
categories as follows: *Open Participation*: Armenia, Brazil, Chile, Greece, Italy, Lith-
uania, North Macedonia, Paraguay, Romania, Serbia, Sierra Leone and Ukraine, while
*Invitation Only*: Czech Republic, Jordan, Malawi, Malta, Mongolia, Netherlands.

For countries that limited the process to invited stakeholders, the perceived contri-
bution of the civil society organizations is lower compared to countries that applied
open consultation procedures (responses for open participation are 1: 33%; 2: 50%; 3:
17% and for invitation only 1: 17%; 2: 17%; 3: 50%; 4:17%).

Out of the countries responding to the survey that have been classified by the OGP (18 countries) on their completion of the prescribed six consultation steps ( timeline availability; advance notice; awareness raising; online consultation; in person consultations; summary of comments) most (67%) have followed all six.<fnref n="2" />

According to the OGP assessment, all countries have performed in-person consulta-
tions. ( timeline availability: 78%; advance notice: 94%; awareness raising: 72%; online
consultation: 83%; in-person consultations: 100%; summary of comments: 78%).

4 Discussion

According to the paper’s key findings, a participatory, open, public policy design pro-
cess is usually initiated by the government itself with a view to strengthen the role of
the stakeholders in the process. Governments usually engage with stakeholders that
represent civil society organizations, government departments and academic institu-
tions while parliaments and political parties are less involved in the process. Most coun-
tries decide to design consultation procedures that are open and accessible, instead of
limiting the process to invited stakeholders. They manage to reach the fourth highest
level (Collaborate) in the IAP2 spectrum of public participation which points to a
meaningful partnership with between stakeholders in the policy formulation and deci-
sion-making process.

Governments recognize the effective contribution of civil society organizations dur-
ing the co-creation phase, especially in countries with a high IAP2 level of public par-
ticipation. Some governments do introduce virtual platforms for stakeholders’ engage-
ment. However, it is not perceived as a practice that increases the effectiveness of the
co-creation process or the quality of the process outputs.

Governments apply various co-creation approaches but the wide inclusion of e-participation practices is comparatively low. E-participation is enabled mainly through online public consultations and social network analysis. E-participation tools used

<footnote n="2">² The full classification is as follows: 1 step: Malta; 2 steps: Malawi; 3 steps: Jordan, Lithuania; 4 steps: Netherlands; 5 steps: Czech Republic; 6 steps: Armenia, Brazil, Chile, Greece, Italy, North Macedonia, Mongolia, Paraguay, Romania, Serbia, Sierra Leone, Ukraine.</footnote>
mainly include consultation platforms, social media, mobile contribution applications, web discussion forums and online surveys. It is worth noting that e-participation tools are mostly used in co-creation processes in countries that have reached a significant IAP2 level.

Co-creation processes usually favor in-person meeting approaches such as SWOT workshops, strategic roadmaps and Living Labs. These processes might be assisted by ICT participation tools such as crowd-mapping, mind maps and issue cards but they usually require the participants to be present in the same physical setting.

Governments, especially in countries with a comparatively mature participatory process as indicated by their IAP2 level, consider that co-creation can highly improve the contents of the action plan and the level of commitment completion. Additionally, the existence of permanent structures and processes in the government along with the involvement of a permanent stakeholder forum are considered key factors for the effectiveness of co-creation processes.

Although virtual approaches are considered effective co-creation processes, governments give higher priority to face-to-face interaction with stakeholders to maximize the effectiveness of the process. In a similar vein, limited human resources and lack of time are considered key impediments to such processes – not lack of ICT tools.

Establishing central agencies that are responsible for citizen feedback and participatory process adoption and co-ordination can improve feedback quality, relevance and utility and significantly aid meaningful citizen participation. Participation efforts themselves could be submitted to stakeholders’ review and examination and perhaps be co-created through citizen feedback. In some cases, meaningful engagement with citizens could suffer by the use if certain ‘arm’s length’ ICT tools such as surveys, quick polls and others.

The key findings of the paper provide insights into the public participation practices used by governments when designing public policy in an open government setting. They point to a realization by government officials involved in supporting open government commitments and policies that citizen participation is understood not simply as an e-government or e-participation project but as an equal access and direct democracy issue. In such a scenario, meaningful participation and inclusive interaction with stakeholders is considered more important than the availability of ICT tools.

## 4.1 Limitations and proposed future research

An analysis of the remaining OGP countries could address limitations arising from the fact that OGP had completed their assessment for a subset of the participating countries in this survey. This could improve conclusions based on comparing data obtained from survey responses and OGP assessment. By the end of 2019 OGP should officially publish all the evaluation reports for the relevant NAPs. These will provide further insights into the effectiveness of the Action Plan implementation in all twenty-nine countries. Future work could focus on further analyzing the data underlying these reports to identify possible associations between National Action Plan development practices and qualitative factors related to the completion and effectiveness of the Action Plan implementation. A follow-up study targeting the remaining OGP countries could also address
the fact that of the twenty-nine countries that responded to the survey the representation of European states is comparatively high (52% of European OGP members, 32% of OGP countries in the Americas, 27% of Asia-Pacific countries and 14,3% of the African OGP countries responded).

## References

1. Janssen, M., Charalabidis, Y., Zuiderwijk, A.: Benefits, adoption barriers and myths of open data and open government. Information systems management, 29(4), 258-268 (2012).

2. Arnstein, S. R.: A Ladder Of Citizen Participation. Journal of the American Institute of Panners, 35(4), 216-224 (1969).

3. Medimorec, D., Parycek, P., Schossböck, J.: Vitalizing Democracy through E-participation and Open Government: An Austrian and Eastern European Perspective. Bertelsmann Stiftung, 14 (2011).

4. Manzini, E.: Design, when everybody designs: An introduction to design for social innovation. MIT press (2015).

5. Putnam, R. D.: Tuning in, tuning out: The strange disappearance of social capital in America. PS: Political science & politics, 28(4), 664-683 (1995).

6. Nye, J. S., Zelikow, P., King, D. C.: eds. Why people don't trust government. Harvard University Press (1997).

7. Cooper, T. L., Bryer, T. A., Meek, J. W.: Citizen-centered collaborative public management. Public Administration Review 66: 76-88 (2006).

8. Meijer, A. J., Curtin, D., Hillebrandt, M.: Open government: connecting vision and voice. International review of administrative sciences, 78(1), 10-29 (2012).

9. Janssen, M., Charalabidis, Y., Zuiderwijk, A.: Benefits, adoption barriers and myths of open data and open government. Information systems management, 29(4), 258-268 (2012).

10. Bertot, J. C., Jaeger, P. T., Hansen, D.: The impact of polices on government social media usage: Issues, challenges, and recommendations. Government information quarterly, 29(1), 30-40 (2012).

11. Hilgers, D., Ihl, C.: Citizensourcing: Applying the concept of open innovation to the public sector. International Journal of Public Participation, 4(1) (2010).

12. Peña-López, I.: Citizens as Partners. OECD Handbook on Information, Consultation and Public Participation in Policy-Making, OECD, Paris (2001).

13. Medimorec, D., Parycek, P., Schossböck, J.: Vitalizing Democracy through E-participation and Open Government: An Austrian and Eastern European Perspective. Bertelsmann Stiftung, 14 (2011).

14. Macintosh, A.: Characterizing E-Participation in Policy-Making. Proceedings of the Thirty-Seventh Annual Hawaii International Conference on System Sciences (HICSS-37), January 5-8 (2004).

15. Ahmed, N.: An overview of e-participation models. UN Department of Economic and Social Affairs UNDESA (2006).

16. International Association for Public Participation. IAP2 spectrum of public participation. (2007).

17. Wimmer, M. A.: Ontology for an e-participation virtual resource centre. Proceedings of the 1st international conference on Theory and practice of electronic governance (pp. 89-98) ACM (2007).

18. OGP Website, OGP Co-creation Standards, https://www.opengovpartnership.org/ogp-participation-co-creation-standards, last accessed 2019/03/07.
19. United Nations: E-Government Survey 2016. New York: UN (2016).

20. United Nations: E-Government Survey 2012. New York: UN (2012).

21. Peña-López, I.: Citizens as Partners. OECD Handbook on Information, Consultation and Public Participation in Policy-Making, OECD, Paris (2001).

22. Macintosh, A.: European e-participation Summary Report. European Commission - Information Society and Media DG, 30. http://europa.eu/information_society (2009).

23. Yu, H., Robinson, D. G.: The new ambiguity of open government. UCLA L. Rev. Discourse, 59, 178 (2011).

24. Piotrowski, S. J.: The “Open Government Reform” movement: The case of the open government partnership and US transparency policies. The American Review of Public Administration, 47(2), 155-171 (2017).

25. Sidney, Mara S.: Policy formulation: design and tools. In: Handbook of public policy analysis, Routledge, p. 105-114 (2006).

26. OGP Website, About OGP, https://www.opengovpartnership.org/about/about-ogp, last accessed 2019/03/15.

27. Frey, L.: Open government partnership four-year strategy 2015–2018, https://joinup.ec.europa.eu/document/open-government-partnership-four-years-strategy-2015-2018 (2014), last accessed 2019/03/15.

28. OGP Website, OGP Co-creation Standards, https://www.opengovpartnership.org/ogp-participation-co-creation-standards, last accessed 2019/03/07.

29. OGP Website, OGP Articles of Governance, https://www.opengovpartnership.org/documents/current-articles-of-governance-pdf, last accessed 2019/03/07.

30. OGP Website, OGP Co-creation Standards, https://www.opengovpartnership.org/ogp-participation-co-creation-standards, last accessed 2019/03/07.

31. OGP Website, OGP Participation and Co-Creation toolkit, https://www.opengovpartnership.org/sites/default/files/OGP_Participation-CoCreation-Toolkit_20180509.pdf, last accessed 2019/03/07.

32. OGP Website, OGP Support Unit: Government Point of Contact Manual. https://www.opengovpartnership.org/sites/default/files/OGP_POC-Manual_2017_EN.pdf, last accessed 2019/03/06.

33. OGP Website, OGP Co-creation Standards, https://www.opengovpartnership.org/ogp-participation-co-creation-standards, last accessed 2019/03/07.

34. De Koning, I.J.C., Crul, R. M, Wever, R.: Models of co-creation, Paper No. 31, TU Delft, The Netherlands, http://www.ep.liu.se/ecp/125/022/ecp16125022.pdf (2016).

35. Spagnoli, F., van der Graaf S., Brynskov M.: The Paradigm Shift of Living Labs in Service Co-creation for Smart Cities: SynchroniCity Validation. In: Lazazzara A., Nacamulli R., Rossignoli C., Za S. (eds) Organizing for Digital Innovation. Lecture Notes in Information Systems and Organisation, vol 27. Springer, Cham (2019).

36. Zolotov, M. N., Oliveira, T., Casteleyn, S.: E-participation adoption models research in the last 17 years: a weight and meta-analytical review. Computers in Human Behavior, 81, 350-365 (2018).

37. Porwol, L., Ojo, A., Breslin, J. G.: An ontology for next generation e-Participation initiatives. Government Information Quarterly, 33(3), 583-594 (2016).

38. OGP Website, IRM Frequently Asked Questions. https://www.opengovpartnership.org/sites/default/files/ogp_irm_04FAQs.pdf (n.d.), last accessed 2019/03/17.

39. OGP Website, Explorer and IRM data, https://www.opengovpartnership.org/about/independent-reporting-mechanism/ogp-explorer-and-irm-data (n.d.). View of the OGP process data-set last accessed 2019/02/07.