metadata: authors: 0: author: Ladegaard,Peter Farup
1: author: Lundkvist,Petter Jonatan
2: author: Kamkhaji,Jonathan Camillo
count: World
docna: 0: docna: Giving Sisyphus a helping hand : pathways
            for sustainable RIA systems in developing countries
docty: Policy Research Working Paper
subtopic: Business Environment,International Trade and Trade Rules,Investment and Investment Climate,Adaptation to Climate Change
teratopic: Environment,Macroeconomics and Economic Growth,Private Sector Development,International Economics and Trade
abstracts: cdata!: Regulatory impact assessment is a tool
            used by governments to support evidence-based and
            coordinated policy making. This paper contributes to the
            debate on regulatory impact assessment in developing
            countries by addressing the lack of a systematic account of
            reforms, and the lack of a comprehensive explanatory account
            of reform outcomes. The study first maps developing
            countries' regulatory impact assessment reforms between
            2001 and 2016. In total, 60 reforms are identified. Reform
            design is analyzed by measuring adherence to six
            internationally recognized "good practices." The
            study then assesses whether the reforms -- two years or more
            after they were launched -- led to functioning regulatory
            impact assessment systems. Of the 60 reforms, 20 led to
            functional systems within two years of the conclusion of the
            reform. Three reforms were too recent to be assessed. The
            study shows that adherence to good practices is a necessary
            but not sufficient condition for early success. Among the
            six good practices, two are shown to be particularly
            decisive for the success of regulatory impact assessment
            reforms, namely, formal integration of regulatory impact
            assessment in policy making and the presence of a regulatory
            oversight unit. The second part of the study analyzes
            regulatory impact assessment reforms that did not produce
            early success despite adhering to good practices. The study
            tests several hypothetical reasons for slow developments. It
            points to the importance of donor flexibility and patience
            and the need for building reform constituencies inside and
            outside government. The traditional orthodoxy of regulatory
            impact assessment reforms as an extension of red tape
            reduction is challenged. The paper finally presents several
            possible policy implications of the findings.
repnme: repnme: Giving Sisyphus a helping hand : pathways
            for sustainable RIA systems in developing countries
display_title: Giving Sisyphus a helping hand :
            pathways for sustainable RIA systems in developing countries
listing_relative_url: /research/2018/03/29730851/giving-sisyphus-helping-hand-pathways-sustainable-ria-systems-developing-countries
topicv3: Trade m1326924 698,Investment Climate(IC) m1327235 401,Climate Change Adaptation m1327767 574,Regulatory Impact Assessment m1503292 2645,Business Inspections, Licensing and
            Permits m1504624 2531
publishtoextweb_dt: 2018-03-19T00:00:00Z
historic_topic: Environment,Macroeconomics and Economic Growth,Private Sector Development,International Economics and Trade
no_of_pages: 41
author_keys: 0: author_key: 795277
1: author_key: 989155
2: author_key: 1271985
model_extractions: WPS8367 Policy Research Working Paper 8367 Giving Sisyphus a Helping Hand Pathways for Sustainable RIA Systems in Developing Countries _Peter Ladegaard Petter Lundkvist Jonathan Kamkhaji_ Macroeconomics, Trade and Investment Global Practice & Governance Global Practice March 2018
Policy Research Working Paper 8367

# **Abstract**

Regulatory impact assessment is a tool used by governments to support evidence-based and coordinated policy making. This paper contributes to the debate on regulatory impact assessment in developing countries by addressing the lack of a systematic account of reforms, and the lack of a comprehensive explanatory account of reform outcomes. The study first maps developing countries’ regulatory impact assessment reforms between 2001 and 2016. In total, 60 reforms are identified. Reform design is analyzed by measuring adherence to six internationally recognized “good practices.” The study then assesses whether the reforms—two years or more after they were launched—led to functioning regulatory impact assessment systems. Of the 60 reforms, 20 led to functional systems within two years of the conclusion of the reform. Three reforms were too recent to be assessed. The study shows that adherence to good practices is a necessary but not sufficient condition for early success. Among the six good practices, two are shown to be particularly decisive for the success of regulatory impact assessment reforms, namely, formal integration of regulatory impact assessment in policy making and the presence of a regulatory oversight unit. The second part of the study analyzes regulatory impact assessment reforms that did not produce early success despite adhering to good practices. The study tests several hypothetical reasons for slow developments. It points to the importance of donor flexibility and patience and the need for building reform constituencies inside and outside government. The traditional orthodoxy of regulatory impact assessment reforms as an extension of red tape reduction is challenged. The paper finally presents several possible policy implications of the findings.

This paper is a product of the Macroeconomics, Trade and Investment Global Practice and the Governance Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank. org. The authors may be contacted at plundkvist@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Giving Sisyphus a Helping Hand Pathways for Sustainable RIA Systems in Developing Countries

Peter Ladegaard Petter Lundkvist Jonathan Kamkhaji

**JEL classification codes** : D02, H70, K23, L50

**Keywords** : Regulatory Impact Assessment, Regulatory Governance, Government Reform, Reform Design
# Table of Contents

|Acknowledgments ............................................................................................................................................................... 3|
|---|
|1.<br>Introduction ................................................................................................................................................................. 4|
|2.<br>Analytical framework and research design ......................................................................................................... 7|
|How many developing countries have launched RIA reforms? ............................................................................. 7|
|What are the main design features of RIA reforms in developing countries? ................................................... 8|
|Were the RIA reforms successful or not? Identifying operational RIA systems .................................................. 8|
|Can particular design features be associated with successful and unsuccessful RIA reforms? ....................... 9|
|Can we identify key factors causing certain RIA reforms to be unsuccessful? ................................................ 10|
|3.<br>Findings ..................................................................................................................................................................... 13|
|RIA Reforms in Developing Countries ...................................................................................................................... 13|
|Case studies ................................................................................................................................................................. 18|
|Uganda (2004-2007) ........................................................................................................................................... 18|
|Kenya (2009-2013) .............................................................................................................................................. 20|
|Cambodia (2008 - ongoing)................................................................................................................................ 21|
|Botswana (2013-2014) ........................................................................................................................................ 23|
|Cross-cutting observations .................................................................................................................................... 26|
|4.<br>Conclusions and policy implications .................................................................................................................... 30|
|5.<br>Bibliography ............................................................................................................................................................ 33|
|Annex 1: Data points for mapping of RIA reforms and methodology for scoring ............................................. 36|
|Annex 2: Semi-structured survey questionnaire ......................................................................................................... 38|

2
# Acknowledgments

This project has been carried out under the World Bank's Good Regulatory Practice (GRP) program funded by USAID and DFID. The GRP Program is a multi-year initiative to help governments in transition and developing countries enhance the quality of regulatory regimes and their outcomes and put in place effective, transparent, accountable, and consultative reform processes that assist in reform prioritization, design, and implementation.

This report was prepared by a team of analysts from the World Bank led by Peter Ladegaard, global lead for Regulatory Policy and Management in the World Bank. The main authors include Peter Ladegaard, Petter Lundkvist and Jonathan Kamkhaji (Consultant). The team is grateful to Professor Claudio Radaelli for his useful comments during project design.

Detailed comments on drafts of this report were provided by Scott Jacobs, Andreja Marusic, Valentina Saltane, Alfredo Gonzalez Briseno, and Stephen Rimmer.

Carolyn Ndawula (World Bank), Abubakar Moki and Frank Kasajja (Government of Uganda) provided useful assistance in the organization of the Uganda field study. The Cambodia field study was supported by Khy Touk (consultant), with overall backing from Leah April, Marong Chea, Lyden Kong, Linna Ky, Lan Van Nguyen, Sokbunthoeun So, and Vannara Sok in the World Bank’s Phnom Penh office.

The team is grateful for input received during the mapping of RIA reforms, notably from Lorenzo Allio, Pedro Andres Amo, Shihab Ansari Azhar, Om Bhandari, Genevieve Boyreau, Dobromir Christow, Chris Darroll, Francesca de Nicola, Senka Eminagic, Ernesto Franco-Temple, Gabriel Goddard, Michelle Gomes Souto, Maya V. Gusarova, Birgit Hansl, Mombert Hoppe, Kaliza Karuretwa, Amina Khaled El Zayat, Andreja Marušić, Davit Melikyan, Christopher David Miller, Sagita Muco, Konesawang Nghardsaysone, Nhan Thi Thu Nguyen, Peter Nuamah, Madina Nurmatova, Serhiy Osavolyuk, Eugeniu Osmochescu, Andrii Palianytsia, Mikhail Pryadilnikov, Dmitry Pyatachenko, Sabah Rashid, Stephen Rimmer, Tarik Sahovic, Saysanith Vongviengkham, Alex Sienaert, and Fanny Weiner.

3
# 1. Introduction

Over the last few decades, Regulatory Impact Assessment (RIA) has emerged as a key instrument to support evidence-based and coordinated policy-making. The systematic use of RIA is recognized as a key means to improve the efficiency, transparency and accountability of decision making (OECD 2015). RIA typically also encourages several “good governance” features, and contribute to a better business-enabling environment (Ladegaard 2005; OECD 2008). Starting in the US and the UK in the 1980s, RIA gained popularity and traction in the second half of the 1990s. By 2015, all 34 members of the OECD at the time reported to have “some form of RIA” in place (OECD 2015, p. 94).

## Box 1. What is RIA?

RIA is a flexible tool that helps governments make better regulatory and other policy decisions based on information and empirical analysis about the potential consequences of government action. The aim of RIA is to ensure that better policy options are chosen by establishing a systematic and consistent framework, including stakeholder consultation, for assessing the potential impacts of government action. A systematic application of RIA, when embedded in the policy process, trains decision-makers to ask and answer targeted questions, at the beginning of the policy cycle, about the need for and goals of regulation, and the possible consequences of government action. The many methods used in RIA – including benefit-cost, cost-effectiveness, and least-cost tests, and partial tests such as administrative burden and small-business tests – are means of giving order to complex qualitative and quantitative information about the potential effects of regulatory measures.

The final products of this systematic process of analysis are self-contained documents called RIAs (or Regulatory Impact Statements, RISs). The RIAs deal with substantive policy issues, are read and utilized by decision makers, and normally available for public scrutiny and subject to the evaluation (often expost, but increasingly so in parallel with the drafting phase) of an independent oversight authority. Although methodologies vary across countries, a number of key elements are common. An RIA normally includes:

- Problem definition and justification for regulatory action

- Data, on which the analysis is based

- Results of the consultation with affected parties and stakeholders

- Identification of a number of feasible policy options

- Estimation and comparison of the different (qualitative and quantitative) impacts of each policy option

- Selection of a preferred option accompanied by a prospective analysis of its implementation and, more recently, providing basis for ex-post review.

There is no single RIA model. The institutional set-up for RIA depends on legal, political, economic and social conditions. However, good practices have been identified internationally, and there are converging patterns of how RIA is institutionalized. The institutional framework for RIA often includes legal provisions which embed RIA in the policy formulation process, and stipulates criteria about its scope of applicability. A governmental body is often tasked with the oversight of the quality of RIA documents and the RIA process.

_<u>Source: Ladegaard and Jacobs (2010).</u>_ Many developing countries have also launched RIA programs, in particular to improve the investment climate. Following a general global trend (and consensus) toward a more systematic use of knowledge and evidence in policy formulation and decision making (Sanderson 2002; Hertin et al. 2009), the last 15 years have seen support to emerging RIA systems becoming an increasingly important element of donors’ and international 4
organizations’ public policy assistance to developing governments. RIA has become part of a regulatory reform agenda oriented not only toward good governance and evidence-based decision making, but more so towards improving the business climate and competitiveness (WBG 2010a; Zhang 2010, Kirkpatrick 2016, p. 381).

Several studies suggest mixed results, but there is no systematic account of RIA implementation in developing countries. Recent years have seen a growing number of studies about RIA in individual, or in small groups of countries, cf. Box 2 below. The studies suggest that global RIA diffusion has not led to the establishment of a single RIA model, and instead shows marked differences between countries. Having said that, the so-called “OECD model” has proven important both for the design of regulatory reforms and for evaluation of success (Zhang 2010). Most importantly, although these studies identify several factors assumed to negatively affect RIA implementation in developing countries, there is still a lack of a comprehensive overview and an explanatory framework that can explain why some RIA reforms in transition and developing countries seem deliver results remarkably fast, whereas others take considerably longer or do not succeed at all. In other words, many factors that can constrain the success of RIA reforms and the establishment of sustainable RIA systems have been identified, but these factors have not been systematized to facilitate explanatory analysis.

Box 2. Recent multi-country studies of RIA in developing countries Penev and Marušić (2011) and Staronova (2010) have looked at RIA reforms in Eastern Europe and the Balkans. Their conclusions on RIA systems in these regions largely corroborate those of the joint EU/OECD SIGMA initiative and point toward scarce and casual application of RIA, markedly different patterns of implementation, and deficiencies in RIA systems, most of all in their institutional dimension. Similarly, studies from Welch (2007), Banya and Waddington (2007), and WBG (2010b) have focused on regulatory reform in East Africa. They find that positive results of RIA reforms are limited, with various different issues hindering the establishment of RIA systems.

Kirkpatrick et al. (2004) conducted a survey covering aspects of RIA systems among civil servants in 99 developing and transition economies. Out of the 40 countries responding to the survey, only 10 had a legal requirement for RIA, while 30 used RIA without explicit legal basis. The survey showed that RIA awareness was fairly satisfactory among the respondents, but that the coverage and methodologies for RIA varied greatly among countries. To evaluate the status of regulatory reform in selected transition economies between 2006 and 2007, Zhang and Thomas (2009) surveyed 11 developing countries on different aspects of regulatory governance, including RIA. They found that application of RIA was “limited and unsystematic” and RIA itself “not integrated into policy decision making”. Zhang (2010) later combined the answers of the two studies, concluding that the reviewed RIA systems were not yet contributing substantially to policy improvements.

Adelle et al. (2014 and 2016) draw on a survey among civil servants from 14 developing and transition economies. The discouraging conclusion is that “of the eight out of fourteen countries included in our survey which did have RIA systems of some form, only one RIA system (Mexico – the only OECD country in the sample) could be considered relatively well institutionalized”.

Intal and Gill (2016a and 2016b) analyzed a sample of member states of the Association of Southeast Asian Nations (ASEAN). The RIA record here seems far better than in previously mentioned regions. The authors still conclude that “[t]he quality of many regulatory reviews under the RIA/RIS system in many of the countries in the Project has been highly mixed, often unsatisfactory. This suggests that such systems have not been very useful and have had little impact on policymaking.” The most important challenges affecting RIA systems revolve around the analytical quality of RIA documents, their usage by political principals and the prominence of legal aspects over substantial ones.

5
The purpose of this study is to create a better empirical understanding of factors contributing to successful _and_ unsuccessful introduction of RIA in developing countries. This interest is prompted by what seems to be diverging implementation patterns of RIA reforms in developing countries. From a practitioner’s point of view, the objective is to identify causal factors that contribute to RIA implementation challenges. With such factors identified, it will be possible to identify mitigating initiatives which can be incorporated early on in the design of RIA reform programs in order to avoid and/or limit the impact of the detected pitfalls. Specifically, the study set out to answer the following questions:

- How many developing countries have launched RIA reforms?

- What are the main design features of RIA reforms in developing countries?

- Were the developing country RIA reforms successful or not?

- Can particular design features be associated with successful and unsuccessful RIA reforms?

- Can we identify key factors causing certain RIA reforms to be unsuccessful?

The report is organized as follows: Chapter 2 describes the analytical framework and research design of the study, and chapter 3 lists the main findings. Chapter 4 summarizes and provides a number of indicative policy implications and recommendations. Subsequent annexes provide further details about the data collection and scoring approach.

6
# 2. Analytical framework and research design

The analytical framework of the study is designed to address a number of key questions about the design, implementation, and dissemination of RIA systems in developing countries. This entails a descriptive snapshot of RIA reforms implemented in developing countries over the last 15 years (2001-2016), an evaluation of the outcomes in terms of functionality of these RIA systems, and an attempt to identify causes for the implementation challenges. A summary of the five key research questions and the associated research design is summarized in Table 1 and elaborated in the subsequent sections.

**Table 1.** Research questions, approach and data sources

|Researchquestion|Approach& Operationalization|Data sources|
|---|---|---|
|**How many developing**<br>**countries have launched RIA**<br>**reforms?**|Identification and template mapping of all RIA reforms<br>launched in developing countries between 2001-2016.|Desk research; local World<br>Bank staff; external experts;<br>http://rulemaking.worldbank.<br>org/ria-documents|
|**What are the main design**<br>**features of RIA reforms in**<br>**developing countries?**|Mapping of all identified RIA reforms according to full or<br>partial observation of six good practices: i) Formal<br>political commitment; ii) Legal integration of RIA in the<br>policy process; iii) Presence of a regulatory oversight<br>body; iv) Presences of RIA guidelines and methodologies;<br>v) Presence of consultation mechanisms; and vi) Presence<br>ofcapacity building andRIApiloting activities.|Desk research; local World<br>Bank staff; external experts.|
|**Were the RIA reforms**<br>**successful or not?**|“Successful RIA reforms” are defined as RIA systems which<br>are functional i) two years after reform launch ii) and<br>which produce and publish RIA statements on a continuous<br>basis.|Desk research; local World<br>Bank staff; external experts.|
|**Can particular design features**<br>**be associated with successful**<br>**and unsuccessful RIA**<br>**reforms?**|Creation of a “RIA good practice index” based on full or<br>partial observation of the above good practices. RIA<br>reform scores from 0-6. Reforms with scores of 3.5 or<br>above categorized as “goodpractice reforms”.|Authors of the study.|
|**Can we identify key factors**<br>**causing certain RIA reforms to**<br>**be unsuccessful?**|Testing of six hypotheses to understand how they can alter<br>the success RIA reforms.|Desk research; four case<br>studies; stakeholder interviews.|

## How many developing countries have launched RIA reforms?

The first step of the study was to identify the number of RIA reforms implemented in developing countries in the last 15 years. The data collection involved the mapping of RIA reforms in developing countries across a range of data points, including period implemented, the role of donors and international organizations, and other factors in the reform environment such as reforms that have overlapped with the RIA reforms. The empirical research for this study does not rely on surveys and questionnaires administered to public officials. It primarily relies on existing larger surveys and databases (especially the World Bank Group’s Global RIA Database and Global Indicators of Regulatory Governance<sup>1</sup> ), contributions by in-country based World Bank staff, as well as expert validation by Word Bank Group staff. The information is triangulated among several secondary sources including: reports by international organizations, academic publications, case studies, governmental reports, and consultant reports, and to some extent also newspaper articles and blog posts. This approach was chosen in part due to the limited response rate as well as validation challenges related to email and postal surveys administered to public officials (see for example Kirkpatrick et al. 2004). The template for information collection can be found in Annex 1.

> 1 This can be accessed at _rulemaking.worldbank.org/ria-documents_ and _rulemaking.worlbank.org._

7
# What are the main design features of RIA reforms in developing countries?

An overall objective of this study is to look at how different design features affect early success (or lack thereof) of reforms. This leads to the question of how different design features can be identified and studied across countries and regions. A natural entry point was to benchmark against the “good practices” as determined by practitioners, experts, and international organizations. There exists, however, a myriad of documents attempting to set the norms, whereas practices selected for this study need to be fit for operationalization and a replicable coding scheme. To arrive at a broadly applicable set of best practices, this study took as a starting point the following RIA/regulatory governance international best practices: OECD (2008) and (2012); OECD/APEC integrated checklist 2005; WBG (2009); and Adelle et al. (2015). OECD (2012) was eventually considered to be setting the bar overly high for a study of a heterogeneous set of countries, and notably looks at RIA’s integration in a broader policy cycle primarily applicable to mature RIA systems of OECD member states. Also, the OECD/APEC integrated checklist was left out since it – despite targeting developing countries – aims to present a set of detailed questions, rather than a group of broad best practices.

WBG 2009, drawing on OECD 2008, spells out the following set of principles to be used in properly designed RIA systems: 1) A formally established RIA policy, with endorsement at a high political level; 2) Integration of RIA in the policy process; 3) Consistency in RIA application; 4) RIA guidelines and methodological requirements; 5) Targeting of RIA efforts; 6) Consultation mechanisms; 7) Data collection strategies; 8) Central oversight of RIA; 9) Capacity in the administration; 10) Oversight of regulatory quality; and 11) Application of RIA to existing, as well as proposed laws. This list is comprehensive and includes some overlapping points. Consequently, using the RIA Light template in the WBG (2009) report and recent critical accounts (Adelle et al. 2015), the long list of principles can be integrated and condensed to six measurable variables, covering, the political commitment, the design features and the practice of RIA, respectively (Box 3).

## Box 3. Six key dimensions of RIA reform

1. Presence of formal political commitment (for instance in a strategic/policy document)

2. Integration in rule-making process (integration of RIA in the policy cycle/policy formulation process through a legal provision) 3. Establishment of a coordination/oversight body/authority 4. Design of RIA guidelines and methodologies 5. Presence of consultation mechanisms 6. Capacity-building activities and early practice/piloting of RIA _Note: Details on the operationalization of the six dimensions can be found in Annex 1. Source: Authors._ The identified RIA reforms were reviewed and assessed according to their full or partial compliance with these “good practice” RIA standards.

# Were the RIA reforms successful or not? Identifying operational RIA systems

Anecdotal evidence suggests that while many RIA reforms enacted in developing countries over the last decade have been successful, many others have not, at least not as quickly as some reform advocates had expected. However, there is no systematic and comprehensive account of RIA reforms’ success rate (or lack thereof). In this light, a key objective of the study was to gauge whether RIA reforms have led to functioning 8
RIA systems. The challenge here is to determine what constitutes a “functioning” RIA system, and by when such functionality could be expected.

Two options were considered for determining the status of RIA systems after introduction. A first alternative was to contrast RIA systems in developing countries with their observation of a number of “good practices” both in terms of internal procedures, transparency measures, citizen engagement, and dissemination of RIAs – possibly leading to the construction of synthetic indicators also for this dimension of analysis. The challenge with this approach would be the degree of subjectivity and discretion associated with selecting and attaching value to certain RIA features.

An alternative pathway was to establish a simple and easily operationalizable threshold criterion to adjudicate whether a country has effectively a (minimally) functioning RIA system in place. This second option appeared more straightforward, promising and commensurate to the degree of reasonable expectations that one should hold, notably for _young_ RIA systems in countries with limited administrative capacity. This study thus measured the status of an RIA system in terms of _the existence of publicly available RIA documents drafted by national officials, at least two years after launch of the RIA reform._ This choice was informed by the assumption that the utility of RIAs is limited if RIA documents are not subject to public access and scrutiny. The study recognizes that evidence-based assessments may be produced and made available for internal ministerial purposes. However, for the purposes of this study we do not consider such practices to equate the existence of an RIA system, which also would have to include transparency and consultation measures.

To gauge if RIA systems were “alive” or not, the study also applied a two-year design and initiation phase from the time the RIA reform was launched. RIA reforms which have been launched within the last two years or less than two years were counted as “too early to call”, regardless if these RIA systems were actively producing RIAs or not. This time gap was established in part because it is well-known that RIA systems take time to develop and launch. But also because many RIA systems, in their first couple of years, are artificially propped up by financial and in-kind support from development partners. The two-year threshold does not reflect a view that RIA systems can be fully designed and operating impeccably two years after launch. However, it reflects an assumption that “chances of survival” are limited if two years into the reform there is no publicly available evidence of activities to improve regulatory policy-making. It should be added that this two-year threshold is not entirely unproblematic. There are a large number of countries, notably in the OECD, that have struggled for longer time periods than two years before finally creating a well-functioning system. The two-year threshold can therefore be considered a very conservative measure of RIA reform success.

# Can particular design features be associated with successful and unsuccessful RIA reforms?

The study also sought to explore if particular design features could be associated with successful and unsuccessful reforms. An indicator of “good practice design” was developed to gauge the extent to which the identified RIA reforms observed good practices along the six RIA dimensions identified in Box 3. Although some studies have suggested that different reform components may carry different weights (Gill and Intal 2016a and 2016b), each of the six dimensions of RIA reform were assigned equal weight. This was done because assigning more weight to “important” best practices could artificially augment the importance of these practices beyond reason, should it be true that they are more prevalent in successful reforms.

Scores assigned to each of the elements above are mainly based on _de jure_ indications about the various reforms, retrievable in publicly available sources. For instance, this allows scoring according to whether RIA or consultation was introduced as a legal requirement, but not (apart from rare instances) whether these obligations were complied with, or whether they were soundly applied in practice, as this would require detailed cast study analysis.

9
Each of the RIA reforms was studied along the dimensions presented in Box 3, and for each of the dimensions a score was provided based on the degree of implementation:

- 0 - absence of the dimension;

- 0.5 - presence of certain aspects of the dimension but not fully implemented;

- 1 - full, or near full presence.

This allowed the creation of an index allowing us to measure the comprehensiveness of the reform design, where each reform is scored on a 0-6 scale. More detailed criteria for scoring on each of the six dimensions is available in Annex 1.

Based on the scores developed above, a minimum for a “good practice” RIA reform score was set. To appraise an RIA reform as _sufficiently well designed_ , it must receive a score of _at least 3.5 out a possible maximum of 6_ . This choice draws upon previous considerations by WBG, namely that: “RIA Light systems in developing countries will have _some of those elements_ , not all of them. [They] essentially provide a menu for developing countries to select RIA components that will work best for them when establishing or strengthening their RIA processes.” (WBG 2009, emphasis added)<sup>2</sup>

# Can we identify key factors causing certain RIA reforms to be unsuccessful?

Finally, we wanted to explore why some RIA reforms, despite having “ticked all the boxes” of good practice design, had not (yet) delivered sustainable RIA systems. The purpose of this part of the study was to articulate reasons for implementation challenges within a structured analytical framework, i.e. a theoretical causal mechanism<sup>3</sup> that can be subject to empirical qualitative testing. The research design consisted of a theorydriven empirical analysis of four case studies. The purpose of the case studies was to test causal or explanatory mechanisms for variations in RIA reforms’ success rates. This step included the construction of theoretical causal mechanisms through the assessment and contextualization of factors already identified in the literature and, most importantly, in the practice of implementing RIA reforms. The intention was then to test these factors (hypotheses) through process tracing (Beach and Pedersen 2013).

The factors constraining implementation are listed in Box 4. The factors comprise a pragmatic collection of anecdotal evidence and suggestions from other case studies. While some of these factors are echoed in the literature, they are primarily drawn from the experiences of practitioners in reform implementation. In other words, to some extent these factors consolidate “popular” explanations shared among RIA advocates and practitioners, which have never been systematically tested.

> 2 It is worth repeating that the best practices measured in this study have been selected based on their predicted importance and ease of use in a larger quantitative study. Nevertheless, there could be other best practices of importance to reform success, as well as nuances of the above list which cannot be easily captured through these indicators. This could for instance include comprehensiveness of the RIA system (very advanced and “heavy” RIA designs are believed to often lead to failures in young RIA systems).

> 3 “[...] Mechanisms [...] are a system that transfers causal forces from X to Y. In the system understanding the focus is on the process whereby causal forces are transmitted through a series of interlocking parts of a mechanism to produce an outcome” (Beach and Pedersen 2013)

10
# Box 4. Implementation Challenges for RIA Reforms

– possible factors and hypotheses

**1. Crowding out** . Competing short-term reforms seen as low hanging fruits, for the political principals can crowd out more time-consuming and challenging RIA reforms. One-off deregulatory and red tape reduction efforts may require less political capital, and ensure more visible and faster results. Consequently, within the given policy-cycle, RIA reforms may not be considered politically attractive compared to other reform options. In the words of Penev & Marušic (2011) on reforms in the Balkans: “RIA implementation lags behind activities related to the simplification of existing legislation because it requires not only political support, but also resources for training, performing adequate RIAs, and the establishment of adequate institutional infrastructure. Investing into a good RIA system does not produce immediate results and is therefore less attractive to the political structures.”

**2. Insufficient adaptation. (“plug and play”).** An often-voiced criticism is that RIA systems in developing countries involve too little adaptation of practices of developed nations “OECD best practices”. Such best practices might be unsuitable and unrealistic in developing countries. For instance, establishing an independent body for RIA coordination may paralyze, delay, or even halt the reform process. Similarly, requirement to use complex analytical methodologies could be detrimental in countries with limited analytical capacity in the administration, and where data are hard to obtain. More than a decade ago, Kirkpatrick and Parker (2004) wrote that “The notion of benchmarking RIA practice against international best practice implies a 'one size fits all approach to policy, which is contrary to the purpose of RIA.”  Despite the many attempts to simplify RIA in developing countries since then, a recent article argues that “we have not yet developed a RIA method that, for most countries, can be practically mainstreamed for the production of relevant and timely policy analysis.” (Jacobs 2016)

**3. Misunderstanding of reform requirements (“Pig-in-a-poke”).** Domestic reform champions may have insufficient understanding of RIA as a long-term governance reform that needs broad stakeholder buy-in. An implementing government may have a basic understanding and ownership of regulatory reform concepts and RIA, but may be unaware of resources required to successfully introduce a RIA system, including the political capital investments and political risks required. This may result in the failure to allocate adequate resources, or to put in place sufficiently empowered institutions. The limited appreciation of RIA reform requirements may in part be due to insufficient clarifications made by external experts and developing partners. A case in point is Greece, which adopted an extremely ambitious RIA program in 2006. Low implementation capacity and a lack of coordinating body resulted in low quality RIAs, if any (Hatzis & Nalpantidou 2007).

**4. Resistance and vested interests.** RIA systems in developing countries may also be difficult to implement because of lack of commitment among civil servants. Unelected civil servants can resist/oppose reform even when there is a binding requirement explicitly mandating RIA in policy formulation. Public officials may resist RIA systems simply because they can change established processes and work patterns, but also because the associated transparency may expose practices which are not ethical or legal. In Australia and New Zealand, it was reported in 2012 that no minister consulted in the preparation of major projects felt that impact assessments had any relevance to their own, or to the Cabinet’s decision making. This was said to be because i) RIAs would reduce individual ministers’ decision-making power; ii) assessments could be of low quality; and iii) RIAs could potentially reveal weaknesses of regulatory proposals in parliament and in the press (Carroll 2014).

**5. Impatient donors.** Some observers and anecdotal evidence also suggest that development partners supporting RIA reform have an overly optimistic view about the time it takes for an RIA system to develop and become sustainable (see for instance Andrews 2013). Consequently, the argument goes, too early withdrawal of financial and technical support may lead to the breakdown of RIA reforms, which may have survived with a slightly longer implementation support from development partners.

**6. “Unhinged”.** This hypothesis suggests that RIA reforms face implementation challenges because they are developed and implemented in a “vacuum”, without adequate linkage to other supportive governance systems such as for example performance management, strategic planning, administrative procedure or freedom of information laws, as well as to the general policy development process (Damonte et al. 2015). This may also indicate the lack of systemic regulatory and policy planning mechanisms which in turn will affect the functioning of RIA, for instance by not allocating time for consultation.

11
It was not expected that any of the factors above by themselves would be able to fully explain why a particular RIA reform did not deliver on its goals. Consequently, this study attempted to capture the dynamic of these factors in the RIA reform context through the country case studies.

The country case studies were based on desk research / secondary sources, followed by field studies. Information was collected through semi-structured interviews with government officials, donors, and other stakeholders for two of the countries. Two of the studies were created relying on official World Bank documents and interviews with World Bank staff.

12
# 3. Findings

## RIA Reforms in Developing Countries

Table 2 shows the key results of the mapping of RIA reforms, outlining that 60 developing country governments launched RIA reforms in the 2001-2016 period.<sup>4</sup> This research also showed that international organizations or bilateral development partners were involved in all of these reform efforts through direct financial support, technical assistance, or other means. In three of the surveyed countries (5%), RIA reforms are within the first two years of implementation, and hence “too early to call”. These reforms are left out from the analysis of whether the reforms were successful or not.

Out of the 57 governments having initiated RIA reforms more than two years ago, 20 (35%) of the RIA systems are functional and operational, whereas 37 (65%) have not succeeded in establishing an operational RIA system. As described in the previous chapter, the threshold for defining an RIA system as functional and operational is that two years after the RIA reform began, Regulatory Impact Statements or documents are produced regularly and available in the public domain. While this criterion is over-simplified and possibly favoring countries with a comparably strong tradition of transparency and freedom of information, it has the clear advantage of relatively easy validation.

Countries’ scores on the “RIA Reform Index” vary in a largely expected pattern. As described in Chapter 2, the project scored all RIA reforms according to their adherence to “good practices”, and developed an “RIA Reform Index” with possible scores from 0-6. A score of 3.5 or higher was set as the minimum threshold for a “well-designed” RIA system. The mean score across all 60 countries is 3.8. For RIA systems which are functional and operational, the average RIA reform score is 5.1. The average RIA reform score for the sample of 37 reforms that did not lead to functioning RIA systems is 3.2. In 22 (37%) of the surveyed countries, the score assigned to RIA reforms is below 3.5. Table 2 summarizes reform periods, good practice adherence, and status for all 60 surveyed countries.

> 4 59 countries plus the subnational entities of the Federation of Bosnia and Herzegovina (FBH), and Republika Srpska (RS) respectively. Due to similarities, these entities were counted as one.

13
**Table 2. Reforms modalities and resulting RIA systems in the 60 developing countries where RIA reforms have been identified as having taken place in the 2001-2016 period. See methodology in Chapter 2.**

|**1. Reforms and scores**||**2. Too early to call?**|**3. Adherence to**|**goodpractices**|**4. RIA sy**|**stem status**|
|---|---|---|---|---|---|---|
|Country / Reform period|Score|Reform initiated within<br>the last twoyears|Not well-<br>designed(<3.5)|Well-designed<br>(≥3.5)|Operational two years<br>after reform initiation|Not operational two years<br>after reform initiation|
|1)Albania(2013)<br>2)Armenia(2008-)|4<br>3||X|X|X<br>X||
|3)Azerbaijan(2015-)|2|X|X|||X|
|4)Bangladesh(2004-2014)|3.5|||X||X|
|5)Belarus(2002-)<br>|2.5<br>||X|||X|
|6)Bhutan(2008-2011)|6|||X|X||
|7)Bolivia(2006)<br>|1<br>||X|||X<br>|
|8)Bosnia Herzegovina(2002-)|4|||X||X|
|8a)BiH – FBH and RS(2012-2015)<br>|5.5|||X|X||
|9)Botswana(2013-2014)<br>10Bil20072011?|3.5<br>25||X|X|X|X|
|)raz(-)<br>11)Bulgaria(2003-2015)|.<br>6|||X|<br>X||
|12)Cambodia(2008-2012/3 –)|5.5|||X|X||
|13)Colombia(2012-)|3||X|||X|
|<br>14)Ecuador(2007-2013,2013-2014)|<br>2||<br>X|||<br>X|
|15)Egypt,Arab Rep.(2008-2012)<br>|5<br>|||X<br>||X<br>|
|16)Georgia(2009-2013,restarted)<br>17)Ghana(2007)|3.5<br>2||X|X||X<br>X|
|<br>18)Guatemala(2015-ongoing)|<br>2|X|<br>X|||<br>X|
|19)Hungary (15years from 1980s)|6|||X|X||
|20)Jamaica(2012)|2||X|||X|
|21)India(2012-)|1.5||X|||X|
|22)Indonesia(2002-2012)|3.5|||X||X|
|23)Iraq (2013-)|1.5||X|||X|
|24)Jordan(2009-2013)|4.5|||X||X|
|<br>25)Kazakhstan(2009-2013,2010-)|<br>4|||X||X|
|26)Kenya(2005-2013)<br>|5<br>|||X<br>||X<br>|
|27)Kosovo(2010-now)|4.5|||X||X|
|28)Kyrgyzstan(2008-)|3||X|||X|
|29)Lao PDR(2012 -2015)|5.5|||X|X||
|30)Lebanon(2009-2012)<br>|2.5<br>||X|||X|
|31)Macedonia(2006-2008)|5.5|||X|X||
|32)Malaysia(2002;2012-2015)|5.5|||X|X||
|33)Mauritania(-2013)|2.5||X|||X|
|34)Mauritius(2015)|2|X|X|||X|
|35)Moldova(2004-2008,2013)|5.5|||X|X||
|36)Mongolia(2011-)|4.5|||X||X|
|37)Montenegro(2010-2012)|5.5|||X|X||
|<br>38)Morocco|<br>2||X|||X|
|39)Nicaragua(2006 - 2011)|2||X|||X|
|<br>40)Palestinian Authority (2007-15)|<br>3.5|||X||<br>X|
|41)Peru(2012-2014)<br>|3<br>||X|||X<br>|
|42)Philippines(2012-2015)|4.5|||X||X|
|43)Romania(2007)|4.5|||X|X||
|44)Russian Fed.(2008-2010)<br>|5.5<br>|||X<br>|X||
|45)Rwanda(2011-2013)<br>46)Serbia(2003-2010)|3.5<br>5.5|||X<br>X|X|X|
|47)South Africa(2005-2012)|2.5||X|||X|
|48)Sri Lanka(2007-2009)|2.5||X|||X|
|<br>49)Tajikistan(2013-)|<br>3.5|||X||<br>X|
|50)Tanzania(2004-2005)|3.5|||X||X|
|<br>51)Thailand(2001-2011)|<br>6|||X|X||
|52)Tunisia(2011-2013/2014)<br>|3.5<br>|||X<br>||X|
|53)Turkey (2005-2008)|4.5|||X|X||
|54)Turkmenistan|1||X|||X|
|<br>55)Uganda(2004-2007)|<br>4.5|||X||<br>X|
|56)Ukraine(2005-)|4.5|||X|X||
|57)Uzbekistan(2008-)|4.5|||X||X|
|58)Vietnam(2007-2010)|5.5|||X|X||
|59)Zambia(2007-2015)|4.5|||X||X|
|**Total**||**3**|**22**|**48**|**20**|**40**|

14
<!-- Start of picture text --> é<br>5<br><!-- End of picture text --> 4 <!-- Start of picture text --> geRasPEL PSEEEBSESEESEEERESELIECEOPLBESLEELEEL®LSSRasPEL PSEEEBSESEESEEERESELIECEOPLBESLEELEEL®LSSPEL PSEEEBSESEESEEERESELIECEOPLBESLEELEEL®LSS PSEEEBSESEESEEERESELIECEOPLBESLEELEEL®LSSELIECEOPLBESLEELEEL®LSSCEOPLBESLEELEEL®LSSOPLBESLEELEEL®LSSLSS S &2 ELALEOO ERG okEOO ERG okO ERG ok ERG ok ok POETcoco L L SEESELOL ELS SEESELOL ELSELOL ELSOL ELS ELSS ELSESOL&SOL&& EES POL ESESSAGS POL ESESSAGS ESESSAGSAGS OL LE FSSROA LE FSSROAROA OOESES FEESOSESES FEESOSOS is<br>& OS EES WSESELE SESE SESEES PR EBS EEES EBS EEES EEES SESSE? & —YESESSEEESESESS<br>“é IEGE FFE WEL GB OF PES SITS Ee TSE SPE ¥ PEG ILE SLE FIL<br>S $ é ao ge ~ a SO “SK<br>RiFSFS §33 ¢—— vEé??<br>& o RS e<br>s$$ §<br>e==<br>a==<br><!-- End of picture text --> geRasPEL PSEEEBSESEESEEERESELIECEOPLBESLEELEEL®LSSRasPEL PSEEEBSESEESEEERESELIECEOPLBESLEELEEL®LSSPEL PSEEEBSESEESEEERESELIECEOPLBESLEELEEL®LSS PSEEEBSESEESEEERESELIECEOPLBESLEELEEL®LSSELIECEOPLBESLEELEEL®LSSCEOPLBESLEELEEL®LSSOPLBESLEELEEL®LSSLSS S &2 ELALEOO ERG okEOO ERG okO ERG ok ERG ok ok POETcoco **L** L SEESELOL ELS SEESELOL ELSELOL ELSOL ELS ELSS ELSESOL&SOL&& EES POL ESESSAGS POL ESESSAGS ESESSAGSAGS OL LE FSSROA LE FSSROAROA OOESES FEESOSESES FEESOSOS is & OS EES WSESELE SESE SESEES PR EBS EEES EBS EEES EEES SESSE? & —YESESSEEESESESS “é IEGE FFE WEL GB OF PES SITS Ee TSE SPE ¥ PEG ILE SLE FIL S $ é ao ge ~ a SO “SK RiFSFS §33 ¢—— vEé?? & o RS e s$$ § e== a==
# pJidde
# Case studies

This section summarizes four case studies, which aimed at identifying the nature and dynamics of factors challenging RIA implementation. This sub-sample of four cases were selected from the sample of 60 RIA reforms identified in the previous chapter as being designed in line with good practice, but where the RIA system is not operational and fully functional at least two years after reform initiation. This approach allows for the study to rule out “work-in-progress” as a reason for the unsatisfactory RIA reform.

**Table 3** . Summary of reform content and design for countries selected for case studies

||**Botswana**|**Cambodia**|**Kenya**|**Uganda**|
|---|---|---|---|---|
|**RIA**|2013‐2014|2008‐2012/3 –|2005‐2013|2004‐2007|
|**reform**||ongoing|||
|**Design**|3.5|5.5|5.0|4.5|
|**score**|||||
|**2016**|RIA not|Reform still|RIA not systematically|RIA not functioning two years|
|**Status**|systematically|ongoing, RIAs are|applied|after reform initiation (but has|
||applied|produced||later restarted)|

The case studies looked particularly at the role of five possible constraints for RIA implementation. In addition to a general overview of the reforms, the study explored if and how the unsuccessful implementation of RIA reforms in the four case studies could be assigned to any of or a combination of the following factors (as per Box 4 on page 11):

- i. Crowding out by quicker and more visible reforms;

- ii. Insufficient adaptation of “OECD best practice” reforms;

- iii. Misunderstanding or misrepresentation of reform requirements;

- iv. Resistance from vested interests, especially public officials;

- v. Impatient donors and short time horizons; and

- vi. Insufficient linkage to other governance tools and systems.

## Uganda (2004-2007)

_The 2004-2007 RIA reform in Uganda was led primarily by the Ministry of Finance, Planning and Economic Development (MoFPED) with subsequent involvement of the Cabinet Office and Ministry of Tourism Trade & Industry. Identified outputs include the establishment of oversight units, technical guidance notes, and a range of capacity-building efforts. During this period the reform was supported by the British Department for International Development (DFID)._

**Policy context.** The origins of the 2004-2007 RIA reform in Uganda can be traced back to 1998, when the Government of Uganda restructured its ministries and departments to improve the processes of policy making (Opio-Lukone 2005), including through the establishment of Policy Analysis Units in each line ministry. The restructuring was fundamentally motivated by business environment factors (ibid.). Between 2000 and 2003, DFID supported a business licensing reform – the “Deregulation Project” – which led to a significant reduction of administrative burdens (see DAI 2007, p.7) and created appetite for broader reform measures. It was noted that “it [the Deregulation Project] successfully built an understanding and awareness of regulatory reform among top-level public officials in Uganda” (DAI 2007 p. 3). As demand grew within the government, DFID agreed to extend their efforts at the end of the Deregulation Project, by creating the Regulatory Best Practice Programme (RBP) aiming to introduce regulatory best practices, particularly RIA.

18
The RIA reform was endorsed by the President of Uganda on multiple occasions in 2004 and 2005 (DAI 2007; WBG 2008), including by distributing a memo to all governmental and non-governmental stakeholders on the importance of RIA, sending circulars to all ministries requiring its implementation in 2003, and authoring the foreword to the 2004 Guide to Good Regulation issued by MoFPED.

**Institutionalization and oversight** . The RBP program retained the Deregulation Project’s focus on business climate, as well as the project’s organizational structure and government leadership (MoFPED). It also maintained the team of international consultants (Bannock Consulting/DAI).

However, the RBP program expanded the number of involved actors. The Steering Committee of the program, supported by consultants, reached out early to all line ministries, parliament, private sector stakeholders, and even to the civil society and the press.

MoFPED hosted the RBP Unit, but an additional RIA unit was established in the Cabinet Office within the Office of the President. Its tasks would entail RIA advocacy across and beyond the government, monitoring of RIA compliance, and coordination and guidance of RIA implementation in line ministries. The RBP Unit was assigned similar, but slightly more strategic/forward-looking and business-oriented objectives. Although the Cabinet Secretariat was considered a relatively weak and irrelevant body at the time, its responsibility in the RIA reform was considered a step in the right direction due to its center-of-government location, comparably high degree of capacity, and appetite for RIA (interviews; DAI 2007; WBG 2008).

In 2005, the Ministry of Tourism Trade & Industry (MoTTI) assumed ownership of the RIA reform. The reason was that RIA was considered a pro-business tool falling within the responsibility of MoTTI, but the decision may also have reflected declining interest from MoFPED (interviews). This change came in the final year of the project and was seen as causing some concerns about the leadership of the reform, as well as requiring capacity-building efforts to be restarted, now in MoTTI (DAI 2207, p. 27). According to one interviewee and to DAI (2007, p. 21), the MoTTI’s RBP Unit was not properly staffed and funded to function properly (two desk officers and a chairman). The RIA Unit of the Cabinet Secretariat also had challenges to fulfill its role due to a small and changing staff (two Policy Analysts).

**Raising awareness and capacity building.** Capacity building activities were significant and systematic. The Cabinet Secretariat partnered with Uganda Management Institute (UMI) in developing curriculum and training strategy, and both institutions visited the UK for a study tour. Between 2004 and 2006, UMI trained some 300 policy analysts, mid-level ministerial officers and private sector stakeholders, while the international consultants and Cabinet Secretariat provided some 50 staff in ministerial leadership positions with training and mentoring on coordination, advocacy and oversight. The newly developed RIA handbook was utilized to familiarize the officers with the RIA template.

However, RIA practice in line ministries did not go much beyond training and RIA pilot studies on a few selected policies at the MoTTI (interviews; DAI 2007, p. 25 and 28). This was partly due to low capacity, but also due to the limited buy-in from leadership in ministries, who allocated insufficient resources.

**Donor support.** As noted above, the Uganda RIA reform starting in 2004 was supported by DFID. DFID’s

support was phased out in January 2007.

**Methodologies and guidelines.** Late 2004 and 2005 saw the selection of an RIA model inspired by the one used in the UK. Interviewees considered the template fit for the purpose but acknowledged that capacity within line ministries was insufficient and would have required extensive training. At the time, the UK’s RIA approach was considered one of the most advanced and demanding among OECD countries.

**Scope.** The 2004-2007 RIA reform did not lead to the establishment of a functioning RIA system in the years immediately following the project. A draft Cabinet Memo prepared to seek approval for the establishment of RIA as a mandatory requirement and embedded in Uganda policy formulation was not tabled for

19
discussion and approval. Moreover, the effort to seek a legal underpinning of the RIA system took place toward the very end of the RBP project and amid political conflicts between the RBP Team Leader and the Commissioner for Law Reform (WBG 2008).

**Key challenges and constraints.** Despite the adherence to many good practices, a number of challenges and constraints contributed to the unsuccessful roll-out RIA in Uganda. Key among these may have been an insufficient appreciation of the governance requirements and time-frame to design and implement a functional RIA system. In the words of an interviewee: “The Government was committed, but not fully aware of the challenges of RIA and RIA sustainability”, prompting diminishing degrees of political support and thereby limited capacity to resist other challenging factors. A key motivation for the project was the GoU’s awareness of the need to secure the benefits from the Deregulation Project or “to embed and institutionalize the achievements of the first phase [deregulation]” (see DAI 2007, p. 45). This came instead of a systemic and long-term overhaul of policy formulation and policy making procedures based upon improved accountability, transparency and stakeholders’ engagement in rulemaking. A closely related factor may have been that the RIA Reform was designed and implemented on the back of the preceding deregulation project, thereby implicitly suggesting that reform dynamics were similar. This was further exacerbated by the transfer of reform leadership to the Ministry of Trade, Tourism and Industry. Complementary factors may have included the relatively complex methodologies chosen from the outset, and the very late timing of the attempt to seek a formalization and integration of RIA in the regulatory policy making process. Because of these factors, political commitment to spearhead and “fight battles” for the RIA Reform faded. This again trickled down to high staff turnover, as well as insignificant staffing and resource allocation to the relevant units responsible for RIA roll-out. After the donor’s funding was phased out, the prospects of adopting a RIA system faded. As stated by a project officer close to the process, “RIA was not enough ‘strategized’ and prioritized by government and Cabinet Secretariat” and there was a “weak transition from deregulation to better regulation which was not fully understood as a reform of policy making procedures.”

# Kenya (2009-2013)

_The reform was led by the Ministry of Trade and Industry and the Ministry of Finance, and pursued in 20092013. Identified outputs of the reform include a technical guidance note, two RIA pilot studies and associated capacity-building efforts. The reform was supported by the World Bank Group._

**Policy context.** RIA was introduced in Kenya as a follow-up measure to the 2005-2007 business licensing reform. The implementation of RIA (and many other reforms) was significantly affected by other political developments in Kenya at the time, most importantly the unrest following the 2007 elections, the negotiations in 2007 of a new constitution, and its referendum in 2010. The result of these challenging political circumstances “[...] was high government priority for certain program components compared to others and not much emphasis in some cases, e.g. business law reform [which was the key legal instrument foreseen to embed RIA].” (WBG/IFC 2012, p. 15). Consequently, active efforts to support the RIA reform were not commenced until late 2009.

**Donor support.** RIA was included in a World Bank Group investment climate program, launched in 2007. Support under the program included RIA pilot studies, guidelines, and limited capacity building.

**Institutionalization and oversight.** To support the above-mentioned licensing reform, the Government of Kenya in 2005 established the Working Committee on Regulatory Reforms for Business Activity in Kenya (WCRR) with a mandate to streamline the regulatory environment for business activity. The WCRR enjoyed cross-governmental buy-in to carry out its mandate: “The two key players – the Permanent Secretaries of the Ministry of Trade and Industry and the Ministry of Finance – were supported by a reform coalition made up of high-level representatives of the Attorney General’s Office, the Kenya Law Reform Commission, and the Investment Promotion Agency.” (Jacobs et al. 2007) In 2006, based on the proposals of the WCRR, the MoF decided to establish and host the Business Regulatory Reform Unit (BRRU). Its main objectives were to

20
permanently keep track of all regulatory and licensing regimes in Kenya and ensure that new regulations, licenses, fees and charges (issued at central or local level) did not create unnecessary burdens on business and met international best practices. As part of this mandate, the BBRU also championed the roll-out of RIA in Kenya. The WCRR’s mandate was not extended after it finalized its review and recommendations related to business licensing.

**Raising awareness and building capacity.** RIA-related awareness raising initiatives and policy dialogue took place from the initial stages of the reform process until 2011. Capacity building and awareness raising events involved 138 stakeholders coming both from the government and from the private sector (WBG/IFC 2012, p. 2). Activities also involved drafting of two pilot RIAs, and associated dialogue with the key ministries involved (Ministry of Finance and Ministry of Housing).

**Methodology and guidelines.** The Business Regulatory Reform Unit (BRRU) developed RIA guidelines and methodologies, aiming at a relatively “light” RIA approach, including a focus on business-related legislation. This material was issued in 2011. The approach and growing prioritization of RIA was reflected in the 2012 government-wide “National Handbook for Public Policy Making”. The Handbook stated that “regulatory policy is one of the key levers of the government’s strategy for economic growth” as well as that “regulatory policy should enhance and improve evidence-based decision making and promote an understanding of the benefits of regulatory reform”. Although the publication of the Handbook represented an important step in terms of increasing formal political commitment to RIA, it did not represent a legal obligation.

**Key challenges and constraints.** Several factors seem to have contributed to the unsuccessful implementation (to date) of RIA in Kenya. Firstly, in a context constitutional crises and devolution, RIA reforms did not have sufficient importance and potential to provide short-term and tangible impact. The 2007 “novelty” of RIA reform was gone by the time more attention could be generated in 2009-2010. Second, compared to other RIA reforms supported by development partners, the support by the key donor (World Bank Group) was very limited, and included only support to pilots and guidelines. Broader support measures, including capacity building and institutional support over a number of years was not provided by other donors. Thirdly, there was never any formal requirement established for ministries to conduct RIA. Without this degree of formalization and enforcement of non-compliance, RIA reforms are unlikely to succeed. And finally, the introduction of RIA on the back-end of a traditional business licensing reform may have limited broader traction and recognition of the reform as a government-wide endeavor to improve evidence-based policymaking.

# Cambodia (2008 - ongoing)

_Cambodia has for many years pursued a commitment to public policy and regulatory reform as a driver of market-led development. Cambodia’s RIA system is well-designed and operational. However almost 10 years after RIA reforms were launched, RIA is still not a formal requirement and is not implemented systematically across the whole government. Supported by the Asian Development Bank (ADB) as well as other development partners, output of reforms to date include RIA technical guidance and training as well as actual RIA statements produced on an ad-hoc basis by more than half of the central government ministries._

**Policy context** . The origins of Cambodia’s RIA reform can be traced back to the Government of Cambodia’s (GoC) 2004 long-term development strategy, followed by a development plan for 2006-2010, which were both committed to good governance and regulatory reform for a better investment climate. Their implementation – supported by the Asian Development Bank (ADB) – helped to raise the government’s awareness of the importance of regulatory reform. This led to a broader program for economic diversification, launched in 2008, within which RIA came to be piloted from 2008 onwards at what is now the Ministry of Mines and Energy (MIME). MIME produced an action plan for mainstreaming the initially “light-touch” RIA across government. These efforts included the preparation of RIA guidelines and steps to making RIA mandatory for the ministry’s own regulations and amendments.

21
**Institutionalization and oversight.** Except for the first RIA pilots in MIME 2008-2010, the broader RIA reform initiated in 2011 was championed by the Prime Minister’s Office’s Economic, Social and Cultural Council (ECOSOCC). ECOSOCC already fulfilled a legislative review function. Several approaches to quality assurance and oversight were considered. The choice fell on a system where the oversight authority would act as facilitator, rather than gatekeeper, during the preparation of RIAs. This was intended to allow for technical advice and support to implementing ministries throughout the drafting stage (and not only after submission of a RIA document). As drafting practices improve in ministries, the ECOSOCC’s Office of Regulatory Impact Assessment (ORIA) is expected to transition to capacity building and gatekeeping. In 2016 ORIA had a staff of 14 desk officers.

**Raising awareness and building capacity.** In 2009 MIME staff participated in a World Bank Group “Network of Reformers” conference in Hanoi. According to interviewees, the event was catalytical and a milestone for raising awareness and bringing political attention to RIA. The MIME SME secretariat subsequently raised awareness across government, and from 2010 also provided RIA training to ministries. Initiatives and support to raise awareness and build analytical capacity have been extensive and multifaceted. The ORIA Unit received training by donor-funded consultants, which enabled them to deliver RIA training independently. All the ministries implementing RIA have established and properly staffed internal RIA units, in each of which up to four desk officers received at least one week of RIA training. More than 120 ministerial officials received technical training and dedicated mentoring sessions were held for permanent secretaries and chairmen of newly established RIA. ORIA and the ADB’s project implementation unit has conducted 52 awareness raising seminars/workshops between 2011 and mid-2016, involving more than 5,800 governmental officials, as well as private sector and civil society representatives (including academics and journalists). Because of this extensive activity of promotion of regulatory reform principles, RIA awareness is considered relatively high. ECOSOCC (and ORIA) has played an important role in championing RIA across government, organizing and delivering awareness raising events, designing and administering RIA training, and drafting guidance for the implementing agencies. Another important capacity building tool was represented by study tours. Four of such study visits were organized by the PIU and ECOSOCC/ORIA: two to Australia, one to the UK, and one to Vietnam. The key feature of these trips was that they exposed the Cambodian officers to the practical aspects of impact analysis, consultation and drafting RIA documents from the perspective of more advanced RIA systems.

**Donor partners.** RIA Reform in Cambodia has been supported mostly and from the outset by the Asian Development Bank (ADB) through the centrally located ECOSOCC, with occasional support from others. Australia’s better regulation office, for example, has provided capacity building.

**Scope of RIA.** The 2008-2010 phase was mainly preparatory, meant to cooperatively outline a systematic plan for RIA introduction across government, and to conduct a small-scale experimental practice/piloting of RIA at the MIME. The scope of the RIA system has gradually expanded, on a voluntary basis. At the start of the broader RIA project in 2011, four ministries participated in piloting RIA. In 2013, another three ministries joined, and in 2015 another six, thereby covering 13 of 20 national government ministries. The RIA process and the oversight of ORIA has been designed to apply to both delegated acts (laws, decrees and subdecrees), and to internal directives or circulars ( _Prakas_ ), but in the latter case the practice of impact assessment and subsequent independent review has proved more challenging.

**Methodologies.** The choice of the Cambodian RIA governance and drafting templates were carefully customized. This was due to the phased and cooperative approach to the reform. The donor-funded consultants prepared in 2012 an informal paper elaborating on the different options for RIA systems and methodologies. A broad policy dialogue was conducted in 2012, involving the consultants, the ECOSOCC/ORIA Unit and representatives of the ministries piloting RIA, which ensured that the selection of RIA design was based on a locally owned process of multilateral consultation and fine tuning.

22
An interesting two-stage RIA methodology involving two different templates for carrying out RIA analyses has been developed at the ministerial level. All proposals (whether decrees or circulars) that may have impacts on business require a Preliminary Assessment Statement (PAS). Proposals that are estimated not to have a significant economic and/or competitive impact do not need to undergo further analysis and, in the case the proposal is a decree or a sub-decree, they are transmitted directly to the Cabinet for consideration upon ORIA review (whereas in the case of _Prakas_ they are directly adopted by the minister –always pending ORIA’s review). If instead the impacts estimated in the PAS are proved to be significant at the socio-economic or competition level, a full Regulatory Impact Statement (RIS) must be drafted. This latter, although based on the same steps and principles as a PAS, requires a more in-depth assessment and is subject to a more careful review by ORIA before approval by the Cabinet or minister.

The suggested methodologies for impact analysis are mainly qualitative but also include basic costeffectiveness analysis, break-even analysis, multi-criteria analysis and simple forms of BCA. It is expected by ECOSOCC and the PIU that the analysis will become more quantitative with practice and improved skills. All interviewees expressed good understanding and ownership of the elements of the ECOSOCC RIA template, and the two-pronged RIA approach matching the rather limited capacity and experience available was highly praised by interviewees.

**RIA production and quality.** 29 PASs/RISs have been submitted and approved by the ECOSOCC/ORIA Unit since 2012 (and 11 more are underway at the time of writing). In theory, all sub-decrees and Prakas emanated by IAs should be subject to the RIA process, but since the system is voluntary, PASs/RIAs are carried out only on acts with impact on business and/or on competition. This selection process typically takes place in the context of legislative action plans devised by ministries. The resulting documents and their quality evaluations are all available online (in Khmer language). Their “quality” varies greatly and according to interviewees, ministries have a preference for PASs and qualitative assessments of impacts due to available analytical capacity and limited funding (since donor funding channeled to ministries through the ECOSOCC are meant to fund only training/capacity building activities and purchase of basic equipment). Nonetheless, all the 29 produced PASs/RISs comply with the template and methodologies established by the 2014 RIA Handbook. Three ministries have even made a further step ahead by drafting internal RIA drafting guidelines/circulars, a clear sign of the strong commitment to and buy-in of the reform.

**Challenges and constraints.** Full rollout through a mandatory whole-of-government RIA issued by the Cabinet/CoM was foreseen for 2017, although many stakeholders considered this target too ambitious. On the other hand, another firm and broadly shared opinion is that RIA has now become a permanent and highly complied element of the Cambodian policy cycle (at least for ministries implementing RIAs). Another central aspect of this positive outlook is the role of ECOSOCC/ORIA Unit which has been officially established, fully operational, properly funded and staffed for six years. It is considered well-located within the government’s structure to fulfill its roles of reform coordinator, champion and oversight authority and enjoys the backing of most line ministries. Nonetheless, more cross-cutting buy-in must be built according to ECOSOCC and the donor. According to interviewees, the key challenges ahead are to ensure broader crossgovernmental buy-in, to improve analytical capacity and skills and to secure long-term budget lines to the RIA system (both to the ECOSOCC and to ministries) once the donor’s funding and TA will be phased out.

# Botswana (2013-2014)

_The reform was led by the Ministry of Trade and Industry, and implemented in 2013-2014. Identified outputs of the reform include an RIA strategy, technical guidance notes, two RIA pilot studies and associated capacity building efforts. The reform was supported by the World Bank Group._

**Policy context.** Botswana’s RIA reform was linked to efforts to improve competitiveness. Despite its lack of economic diversification, Botswana has broadly been considered a success story in terms of economic growth

23
and development, as well as democratic rule and good governance. However, in the aftermath of the economic downturn of the financial crisis in 2008-2010 it was evident that the economy suffered from an overall low competitiveness, heavy dependence on the mining sector, and limited FDI. Challenges in the operating environment were evidenced through a decline in rankings in global comparisons such as the Global Competitiveness Index and Doing Business Indicators.

**Donor support.** The World Bank Group supported Botswana’s RIA Reform as part of a broader technical assistance program focusing on economic diversification. In line with World Bank policy, since Botswana is a middle-income country, the program was fully funded by the Government of Botswana (GoB). The broader program had five main components: Business environment, trade and industrial policies, infrastructure, access to finance, and skills and innovation. Each of the five components were assigned to a specific governmental counterpart/focal point for implementation. As it is common practice in development projects, the subcomponent focusing on RIA fell under the umbrella of business environment initiatives with the Ministry of Trade and Industry (MTI) as the designated counterpart. The business environment component also included a focus on business licensing and “Doing Business” reforms. The two other reform components were launched six months earlier than RIA in the two-year project, meaning that they got a head start in locking in resources and staff.

**Raising awareness and building capacity.** The institutional dialogue between the WBG and MTI began in mid-2013 and led to the issuing of a strategy for the introduction of RIA in Botswana, starting with awareness raising. These activities took place in a context of many policy changes in other and related domains. The Botswana Chamber of Commerce (BOCCIM) and several private sector representatives emerged as vocal RIA champions and advocates, building on earlier advocacy for increased openness in government policymaking. The BOCCIM played an important role in raising the attention of government actors, and for launching a policy dialogue about RIA. An August 2013 event at the Botswana Bureau of Standards was attended by representatives of almost all Botswana ministries and cabinet-level bodies, by officials from independent regulatory authorities, and various representatives of the private sector.

This first step was considered successful by the project team in terms of building capacity for RIA to enable advocacy by a broad set of actors. It was followed by a cross-cutting policy dialogue initiative through individual visits by MTI officials and WBG experts to various ministries and governmental bodies. These meetings were meant to achieve four goals: (i) to cement ownership and awareness of RIA; (ii) to create a coherent group of stakeholders (an “RIA interest group”) which could help shape and steer the RIA reform process along with the MTI; (iii) to explore the willingness of other ministries/independent regulatory authorities to be trained and pilot RIA; and, (iv) for a set of selected institutions, to explore their willingness to take over MTI’s role of focal point and host the to-be RIA oversight and coordination unit.

A range of training and capacity building activities also took place. In total, 30 officers were trained in conducting RIA. Many private sector stakeholders were involved in the consultation stage of the pilots conducted in the context of the training. Participants in two one-week RIA pilot sessions in 2014 were MTI staff and staff from a regulatory authority hosted at the Ministry of Finance and Development Planning, the Non-Banking Financial Institutions Regulatory Authority (NBFIRA). Thus, WBG and the two groups of trainees cooperatively drafted two pilot RIAs based on the selected template through qualitative methodologies. Additional practice of RIA beyond the pilots and training did not take place in ministries and local capacity to deliver RIA capacity building was not developed.

**Methodologies and guidelines.** Technical efforts then focused on developing RIA methodologies and oversight mechanisms. The template developed was based on an “RIA light” model and included (i) Problem definition and rationale for intervention; (ii) Consultation of stakeholders/affected parties; (iii) Impact analysis of alternative policy options; (iv) Impacts on implementation; and (v) Plan for post-implementation monitoring and review. The scarcity of data and the relatively low level of statistical/analytical capacity at

24
the ministerial level led to proposing Multi Criteria Analysis-MCA rather than full Benefit-Cost Analysis as the key methodology for the comparison of the policy options within the RIA process.

**Institutional leadership and oversight.** Following policy dialogue within the “RIA interest group” and with a number of agencies potentially interested in hosting the RIA oversight function, an agreement was reached on the institutional structure of the future RIA system. Many models of oversight, including also private/public partnership like the Dutch Actal, were proposed and considered by the interest group. The agreed framework foresaw an independent unit - the Red Tape Reduction Unit (RTRU). This unit would have been hosted either at the level of a ministry (but not at the MTI which strongly refused to house the unit) or at the level of the Cabinet Office/Office of the President. It would have been composed by a Board with mixed private-public membership and by an Executive Secretariat composed by civil servants. Its main foreseen tasks were to champion and coordinate the implementation of RIA across the government, to issue RIA guidelines for ministries, to oversee RIA documents per the agreed guidelines, to prepare an annual report on RIA implementation and even to perform ex-post reviews.<sup>6</sup>

Once the RTRU would have been established, the expectation was that its Executive Secretariat would have endorsed and used the guidelines and the checklist for coordination and oversight purposes. RIA rollout was planned to be gradual, following a criterion of progressive complexity, also in order to concede enough time to further build capacity at the ministerial level via learning-by-doing and RIA piloting, and RIA was meant to be mandatory only for business-related acts and to be scaled up to the whole legislative production over time.

**RIA production and quality.** Two pilot RIAs of existing regulations were produced, but they were pure training exercises. The trainees provided the input data, experimented with consultation techniques with realworld stakeholders and gave input on the policy options. These exercises showed a general need, common to most developing countries, to foster analytical/quantitative skills within the PA. While the MTI’s mid-level officers that took part in the training showed resistance and distrust toward RIA and evidence-based methodologies, almost the opposite held true for the NBFIRA trainees that showed a level of awareness, commitment and interest well above the average of typical RIA training participants.

**Scope.** The final part of the WBG engagement (second half of 2014) was focused on seeking agreement within the “RIA interest group” on a Cabinet Memorandum intended to provide RIA and the planned RIA Unit (RRTU) with a clear legal mandate. Efforts also focused on transfer of relevant material and guidance to the MTI for it to lead the RIA reform efforts after the World Bank Group support ended. The Memorandum proposing formalization of the RIA system, however, was not submitted to the Cabinet for consideration.

**Key challenges and constraints.** Several political factors contributed to this outcome. Most importantly, general elections were to be held in October 2014. The MTI, having received the material in mid-August 2014, considering the delicate political phase, decided not to push the RIA Memo as a “midnight” law. After the elections, a new minister was nominated at the MTI and RIA was not considered further.

However, a number of preceding factors are likely to have contributed to the limited willingness to formally launch and institutionalize the RIA reform. Key among these factors might have been the choice of lead agency to champion the reform, and the packaging of RIA reforms with business licensing and Doing Business Reforms. Structurally, and in terms of the ministry’s capacity and interest in RIA, allocating the implementation responsibility with MTI may have been decisive for unsuccessful roll-out.

> 6 The proposed scope of the (initial) RIA system was intentionally relatively narrow. The scarcity of data and the relatively low level of statistical/analytical capacity at the ministerial level led to proposing Multi Criteria AnalysisMCA rather than full-blown Benefit-Cost Analysis as the key methodology for the comparison of the policy options. A set of guidelines of good regulatory practice and a checklist for RIA oversight drawn from international best practices were also prepared and approved by the “RIA interest group”.

25
At the time of the launch of the RIA reform, MTI’s reform capacity and attention were already (partially) occupied by other concerns and reform priorities than RIA. The other two reform packages (business licensing and doing business reform) appeared, as candidly admitted by a number of officials close to the process, significantly easier to implement and more readily appropriable than a long-term reform meant to bring about a full-blown overhaul of rulemaking practices across government.

A key task of an RIA reform focal point is to credibly champion the reform and raise awareness, trigger cross-governmental buy-in, and have sufficient authority/power and trust across fellow ministries to be entrusted with regulatory oversight functions. However, the observed level of ownership of RIA reforms by MTI, and its understanding of the related challenges was low. MTI was not itself fully supportive of the proposal for how to institutionalize RIA, and not influential enough to change it, lacking a solid agreement across government for the housing of the to-be focal point (the RRTU).

# Cross-cutting observations

This section explores if and how the implementation challenges described in the four cases can be explained by some of the six general factors identified earlier. These six factors possibly influencing RIA implementation were:

- i) “Crowding Out” by short-term and more visible regulatory reforms

- ii) “Plug and Play”, i.e. insufficient adaptation of OECD country-approaches to RIA

- iii) “Pig-in-a-Poke”, i.e. insufficient understanding by reform champions of RIA reform requirements iv) Resistance from public officials

- v) Impatient donors, i.e. too short timeframe; and

- vi) “Unhinged, i.e. RIA reforms not linked to or leveraged by other public sector reforms.

First and foremost, it seems clear that all six factors play important roles in explaining why RIA reforms not always lead to operational RIA systems: Crowding out, insufficient adaptation, misunderstanding of reform requirements, resistance from vested interests, short time horizons for implementation, and limited linkages to other governance systems all seem relevant and often powerful contributors to why RIA reforms do not always develop as intended. It also seems that these factors are closely connected and mutually reinforcing each other.

A second observation relates to political leadership and institutional anchoring of RIA reforms. The initial rationale and impetus to pursue RIA reform is often closely linked to preceding or parallel regulatory burden reduction reforms. This close connection is very legitimate and in most cases probably strategically sound. However, it also comes with a risk of the RIA reforms being crowded out (factor 1) by faster and more tangible reforms, _and,_ perhaps more importantly, of being “stuck” with reform champions who are not capable or willing to pursue the cross-ministerial coordination and enforcement roles associated with a functioning RIA system. The “crowding out” factor seems closely associated with an insufficient appreciation by RIA reform champions of the requirements and long-term nature of RIA reforms (factor 2). The limited appreciation of RIA reform requirements may in part be due to insufficient clarifications made by external experts and developing partners.

Third, the case studies found some evidence of insufficient adaptation of “OECD best practices” being a negative factor for the RIA roll-out (factor 3). However, compared to other constraints it did not seem to be a major factor for why RIA reforms were never satisfactorily implemented. This is not to say that careful adaptation is not important. However, it is possible that such constraints may manifest themselves more clearly when the RIA system is operational.

Fourth, across the board of al,l the case studies there seemed to be no or only limited integration with or linkage to other public sector reform tools. Instead, RIA reforms are developed and implemented in a “vacuum”, without adequate linkage to other supportive governance systems such as for example 26
performance management, strategic planning, administrative procedure or freedom of information laws, etc. While it is difficult to assess the impact of the counterfactual – i.e. what would have happened if RIA reforms _were_ leveraged by, and integrated with other public management reform tools – it seems likely that the relative disconnect of RIA systems from other governance systems had a negative impact on the reforms’ sustainability and capacity to evolve according to changing circumstances.

A summary of these observations can be found in table 4.

27
Table 4. Overview of RIA reform challenges identified in the four case studies

||**Botswana**|**Cambodia**|**Kenya**|**Uganda**|
|---|---|---|---|---|
|**Factor 1.**<br>**Crowding out. Competing short-term reforms can**<br>**crowd out more time-consuming and challenging**<br>**RIA reforms.**|**RIA was crowded out of a broad**<br>**reform program with short time**<br>**horizon.**<br>The overall reform package was<br>overly broad in relation to time-<br>frame. At the same time, other<br>(more<br>attractive)<br>reform<br>components were already being<br>implemented in MTI when RIA<br>reform started.<br>The importance of this factor is<br>ranked as**STRONG**since the risks<br>could not have been mitigated<br>without fundamental redesign of<br>the program. It is likely that<br>another counterpart than MTI could<br>have provided more attention to<br>RIA, but due to project design and<br>overall government awareness MTI<br>ended upthe only option.||**Crowded out in Ministry and**<br>**among policymakers**<br>Due to its location in the Ministry of<br>Finance, BRRU had to compete with<br>a number of other units for<br>resources and staff and due to<br>weakened<br>support,<br>it<br>was<br>eventually crowded out. Political<br>support at higher levels also saw<br>interest in RIA being crowded out<br>due to political developments in the<br>country.<br>The impact of this factor is itself<br>**STRONG**, and in particular the<br>weakened political support could<br>not have been foreseen.|**Crowded out in ministry**<br>When RIA was located in MoFPED,<br>the ministry had a range of<br>traditional financial<br>responsibilities, leaving limited<br>capacity to steer the reform.<br>This factor has**MEDIUM**impact<br>and could have been predicted<br>and possibly mitigated.|
|**Factor 2.**<br>**Insufficient adaptation (“plug and play”). Too**<br>**little adaptation of practices of developed**<br>**nations**||**Insufficiently adapted training**<br>RIA training was said by some<br>participants not to be sufficiently<br>adapted to needs and to existing<br>capacities<br>in<br>RIA<br>units.<br>The impact of this factor appears<br>**LOW**.||**Overly complex RIA template**<br>RIA modelled after the UK<br>template, considered as one of the<br>most complex in the world.<br>This factor has**MEDIUM**impact<br>and could have been mitigated.|
|**Factor 3.**|**RIA was introduced as part of a**||**RIA promoted as continuation of**|**Business environment focus led**|
|**Misunderstanding of reform requirements (“pig-**<br>**in-a-poke”). Domestic reform champions may**<br>**have a lack of understanding of RIA as a long-**<br>**term governance reform that needs broad**<br>**stakeholders buy-in.**|**larger reform package to improve**<br>**competitiveness,**less so as a tool<br>to improve evidence-based policy-<br>making across government.<br>The importance of this factor in<br>itself is ranked**MEDIUM**since there<br>could potentially have been scope<br>to overcome this challenge||**one-off reform**<br>RIA seen as a continuation of<br>licensing reform, rather than as a<br>comprehensive governance reform.<br>This led to limited understanding<br>for<br>the<br>broader<br>commitment,<br>resources<br>and<br>governance<br>arrangements that would have<br>been necessary for reform success.<br>Few<br>ministries<br>were<br>actively<br>involved.<br>This is a STRONG factor which is<br>the root to many other challenges<br>in the project.|**to wrong project design.**RIA was<br>initiated as a continuation of<br>licensing reform. This resulted in<br>government not fully aware of the<br>complexities of reform, and the<br>degree of commitment required.<br>MoFPED<br>turned<br>out<br>to<br>lack<br>commitment<br>and<br>capacity<br>to<br>implement a broad reform, as<br>evidenced by later change of<br>counterpart.<br>This factor is key to project design<br>and its impact is**STRONG**.|

28
||**Botswana**|**Cambodia**|**Kenya**|**Uganda**|
|---|---|---|---|---|
|**Factor 4.**<br>**Resistance and vested interests. RIA systems in**<br>**developing countries may also fail to take off**<br>**because of lack of commitment among civil**<br>**servants. Public officials may resist RIA systems**<br>**simply because they can change established**<br>**processes and work patterns, but also because**<br>**the associated transparency may expose**<br>**practices which are not ethical or legal.**|**Limited**<br>**acceptance**<br>**across**<br>**government**<br>Since RIA was considered a<br>business environment tool, MTI<br>became the implementing agency.<br>However, they openly resisted<br>taking on required responsibilities.<br>At the same time, bodies at center<br>of government were similarly<br>uninterested<br>due<br>to<br>heavy<br>workload.<br>This factor was key to the reform<br>impediments<br>and<br>impact<br>is<br>**STRONG**.|**Limited awareness and advocacy**<br>**among key constituencies**<br>RIA has failed to build an important<br>constituency outside implementing<br>agencies. The fact that ministries<br>still not embrace RIA leads to it<br>continuously being voluntary.<br>The public-private dialogue is<br>limited, and private sector has not<br>been able to assume a role in<br>advocacy.<br>This factor had a**MEDIUM**impact,<br>and does not challenge the whole<br>program but prevents broader<br>|**RIA priorities gradually faced.**<br>The Institutional structure from the<br>preceding licensing reform was<br>continued for the RIA reform, and<br>then gradually phased out. RIA<br>would have required stronger,<br>better<br>staffed<br>and<br>trained<br>institutions.<br>It is unknown whether RIA was<br>opposed outside MoF, but MoF’s<br>cross-governmental role in RIA<br>oversight was openly opposed.<br>This factor has in itself**MEDIUM**<br>impact and could potentially have<br>|**Resistance from management in**<br>**ministries**<br>There appears to have been<br>pockets<br>of<br>resistance<br>among<br>management in line ministries, who<br>considered RIA a hurdle taking<br>away resources that could be used<br>for other tasks.<br>The impact of this factor is**LOW**<br>and could have been overcome<br>through outreach, as well as<br>stronger oversight.|
|||rollout.|been overcome.||
|**Factor 5.**<br>**Impatient donors. Development partners may**<br>**have an overly optimistic view of the time it takes**<br>**for a RIA system to develop and become**<br>**sustainable. Consequently, too early withdrawal**<br>**of financial and technical support may lead to the**<br>**breakdown of RIA reforms, which may have**<br>**survived with a slightly longer implementation**<br>**support from development partners.**|RIA was implemented within a two-<br>year program, which included a<br>range of components. While the<br>time frame may have been fit for<br>the other components, it appears<br>time was too optimistic for the<br>fundamental changes required for<br>RIA.<br>This was a contributing factor, but<br>has in itself**MEDIUM**impact|RIA was supported through a string<br>of closely connected donor-funded<br>projects over more than a decade.<br>This was not a contributing factor,<br>although impacts of the donor’s<br>possible<br>withdrawal<br>in<br>2017<br>remains to be seen.|||
|**Factor 6.**<br>**“Unhinged”. The RIA reforms are developed and**<br>**implemented in a “vacuum”, without adequate**<br>**linkage to other supportive governance systems.**|No integration with or leverage<br>from other public management<br>reform observed.|No integration with or leverage<br>from other public management<br>reform observed.|BRRU never seems to have been<br>integrated into the ministry’s<br>activities, and was instead<br>considered a unit that competed<br>for resources with the traditional<br>MoF activities.<br>This factor has**MEDIUM**impact<br>and could potentially have been<br>mitigated by the project team.|No integration with or leverage<br>from other public management<br>reform observed.|

29
# 4. Conclusions and policy implications

This study has provided one of the first comprehensive overviews of RIA reforms in developing and transition countries and identified design features that are associated with early success or failure of such reforms.

Over the last 15 years, at least 60 developing countries have initiated RIA reforms, all of which have been supported by development partners in one way or another. Of the 60 reforms, three have been launched within the last two years. These three reforms are not included in the subsequent analysis, since their status may reflect that the systems are still “under construction”, or that a high activity level reflects initial and unstainable boosts of political support and abundant donor funding.

Looking at the 57 RIA reforms that were initiated more than two years ago and applying a very simple “dead or alive” criteria (are RIA statements regularly produced and publicly available two years after the reform’s launch, or not?) the mapping of reforms finds that 20 RIA reforms have already led to operational and functional RIA systems, whereas 37 have not.

Whether a “success-to-failure” rate of 1:2 after only two years is satisfactory, mediocre or poor depends on one’s perspective. If such appraisal is informed by an assumption about RIA reforms as relatively simple and linear, the judgement call is likely to be negative. However, if RIA reforms are considered a relatively complex governance reform requiring a difficult combination of long-term political commitment, coordination across government, technical skills, and willingness to rearrange decision-making processes, the success rate may be very acceptable, possibly even encouraging. Add to this that many RIA reforms which have not yet become operational and systematically applied may do so over a longer time horizon.

The study confirmed that RIA reforms designed in line with generally recognized “good practices” for developing and transition economies are more likely to lead to operational RIA systems than RIA reforms which were not. However, the study also explored how a number of design features seemed to be more important than others. The single biggest difference between RIA reforms which led to operational systems, and reforms which did not, was formal/legal integration of RIA requirements in the policy-making process. In other words, formal/legal requirements for policy-makers to comply with RIA quality-assurance mechanisms, although not necessarily a guarantee for the quality of RIAs, may be the single-most important milestone towards establishing an operational RIA system.

The study also found that there was a negligible difference between operational and non-operational RIA systems in terms of the capacity-building measures that had been undertaken. As a matter of fact, RIA reforms which did _not_ lead to operational RIA systems had a slightly better score on capacity building measures that those reform that led to an operational RIA system. This study could not confirm if this was due to significant variations in the quality of capacity-building measures (which would be a reasonable assumption), or whether it was due to the limited impact of capacity building measures overall.

A very large proportion of the observed RIA reforms were launched as an integral part of regulatory burden reduction reforms, mostly aimed at eliminating and simplifying business licenses. This linkage has traditionally been considered as commonsensical and positive: After having reduced the _stock_ of existing regulations, efforts should logically move to ensuring that the _flow_ of new regulation does not erode the results already made. Findings of the study suggest that this may be an unholy marriage which does not last long. In many cases, the institutional leadership “burden reduction champions” are not the same as those who can promote the good-governance and whole-of-government dimensions of RIA reforms. If organized within the same reform and institutional structure, short-term efforts with highly visible and quantifiable impacts can crowd 30
out longer-term and less quantifiable RIA reforms. Accepting this premise may have important repercussions for how governments and development partners frame and organize RIA reforms.

What policy implications and recommendations can the study provide to transition and developing countries considering to establish an RIA system, and to development partners supporting such efforts? As mentioned above, the study has confirmed the importance of many of the already known good practice preconditions for building an RIA system. For countries to design and implement a successful RIA system, they should focus on establishing such preconditions. However, the study also provides nuances on the factors that can influence the many pathways to RIA. In summary, we point to five areas where changes to the strategic framing of RIA initiatives, compared with how most RIA reforms are developed today, may have the most significant impact:

**1. Recast RIA as part of a long-term plan to improve regulatory quality and evidence-based rulemaking (not just burden reduction for businesses).** Because of regulatory reforms’ close relation to the investment climate, RIA is often launched in the context of regulatory burden reduction reforms. However, evidence-based policy making seen as a simple add-on to business licensing or other oneoff reforms may create wrong perceptions about reform requirements and may embed RIA in suboptimal institutional structures. Our case studies indicate that RIA was crowded out in Botswana when the government prioritized other business environment reforms that were easier to implement and could provide short-term rewards. They also indicated how reforms in Kenya and Uganda stalled when RIA reforms were relying on institutional structures previously used in licensing reforms, with limited adaptation or additional strengthening. RIA is a whole-of-government and systemic tool, which requires coordination and authority beyond single ministries.

**2. Use high-level political support to lock in the RIA reform at an early stage** . Many RIA reforms – especially those which have a longer implementation phase – set out to evolve organically from raising awareness, building reform constituencies, conducting training, pilot studies, developing guidelines, etc. Formalization, i.e. a formal/legal integration in the policy-making process has often been pursued at the very end of the reform cycle, at a time when supporting donor projects were coming to an end, and where the initial political attention has faded. The findings of this study suggest that there may be merit in capitalizing on the strong political commitment often enjoyed at the early stages of RIA reform by seeking a formal/legal integration of RIA in the policy-making process. Legal amendments could be made such that they would only come into force with a certain delay (i.e. 12-18 months), but they would send a credible signal and navigation point for relevant stakeholders.

**3. Establish regulatory oversight bodies to champion RIA reform; functions may initially be focused more on guidance and support than gate-keeping.** As could be seen in the overview of RIA reforms, the presence of an institutional structure that spearheads the reform efforts is a strong predictor of success. A formal structure can help manifest the government’s long-term commitment to RIA, both to internal and external stakeholders. A centralized body can manage roll-out across government and collect and disseminate knowledge over time. A strong gatekeeper function with authority to the oversight unit to “review and reject” RIAs of suboptimal quality may create unproductive adversarial relations between RIA stakeholders and may lead to speculative behavior aimed at avoiding such scrutiny. The case of Cambodia is an example of a reform where training efforts by external experts targeted an oversight body, which has thereafter disseminated this knowledge across government. Although Cambodia envisages a gatekeeper function over time, during rollout its main tasks have been on building capacity and guiding RIA drafting. In the first several years of existence, RIA oversight bodies may therefore be better served to take on support and advocacy roles, as opposed to formal gatekeeper functions.

31
**4. Leverage other public sector reform tools to promote evidence-based rule-making.** RIA should not be considered an activity separate from other dimensions of the policy-making process. The case studies from Kenya and Uganda showed that although the institutions were formally there, they were considered new and foreign entities that stole attention and resources from the regular government work. Efforts should be made to make evidence-based policy making part of the regular government functioning already at an early stage of the reforms. This includes communicating synergies and exchange of information with other functions, such as regulatory enforcement and strategic planning, but also formal integration in the government’s consultation practices and training curriculum. It also entails an increasingly developed system of RIA-specific levers, such as budgetary incentives for developing RIAs, or inclusion of RIAs in departments’ or staffs’ performance agreements. Also, when deciding on and communicating the reform’s targets, it may be useful to look at RIA as one of several components that together make up an “ecology” of government tools (Radaelli et al 2012).

**5. Focus capacity-building efforts on clear targets and on-the-job requirements.** Findings of the study suggest that training and capacity building measures are not a predictor of an RIA reform’s outcome. The suggestion is that investments in RIA training may often provide only limited direct returns to the reform efforts. This finding corroborates many anecdotal observations about donor supported RIA reforms focusing on expensive RIA training, only to have the trained officials move on to other jobs or have limited specific utility of the general training received. The implication for RIA reform may be that training at the early stages of RIA reforms should focus less on broad and generic RIA training, but rather be targeted to specific RIA requirements of the country. It seems important that training efforts are well measured, anchored and led by the reforming country, and that experience and knowledge can feed into further developments and learning. Training will only become efficient and useful to motivate reluctant civil servants if they know they will be rewarded through their ability to master RIA in their daily work.

_Implications for future research._ The process of preparing this study generated new ideas or confirmed existing ones for future research on RIA in developing countries. It also exposed ways in which the methodology applied for this paper could be improved in subsequent updates. First, in identifying factors influencing the success of RIA reforms in developing countries, it would be interesting to include comparable data points related to countries’ legal systems as well as the co-existence of other possibly supportive governance systems.<sup>7</sup> Second, for possible updates of the data set behind this study, it would be relevant to measure developments in national RIA systems over longer time horizons; to explore additional, verifiable criteria for RIA development (than just publication of RIAs two years after the reform’s launch); and to explore correlations with a broader range of outputs, such as the number of regulations proposed and promulgated, quality of stakeholder consultation, as well relevant macro-economic variables.<sup>8</sup>

> 7 This could entail combining insights from Renda (2015) and Damonte, Dunlop and Radaelli (2014), who have looked at this issue from a conceptual and developed-country perspective.

> 8 As seen in part in Jacobzone, S., C. Choi and C. Miguet (2007).

32
# 5. Bibliography

- Adelle, C., Macrae, D., Marusic, A. and F. Naru (2015) “New development: Regulatory impact assessment in developing countries—tales from the road to good governance.’ _Public Money & Management_ , 35(3): 233238.

- ADB (2006) _Technical Assistance Report. Kingdom of Cambodia: Capacity Building for SME Development – Phase II._

- ADB (2008) _Report and Recommendation of the President to the Board of Directors. Proposed Program Cluster, Loan for Subprogram 1 and Grant. Kingdom of Cambodia: Promoting Economic Diversification Program._

- ADB (2010) _Project Administration Manual. Kingdom of Cambodia: CREST Project – Regulatory Impact Assessment Subproject._

- ADB (2014) _Cambodia: National Rollout of the Regulatory Impact Assessment in Government._

Adelle, C. Weiland, S., Dick, J., González Olivo, D., Marquardt, J., Rots, G., Wübbeke, J. and I. Zasada, (2016) “Regulatory impact assessment: a survey of selected developing and emerging economies.” Public Money & Management, 36(2), 89-96 (2014 report version: - <u>http://liaise kit.eu/sites/default/files/LIAISE%20D0%208%20RIA%20in%20Developing%20and%20Emerging%20E conomies_1.pdf).</u>

- Andrews, M. 2013 (2013) _The Limits of Institutional Reform in Development: Changing Rules for Realistic Solutions._ Cambridge

- Baily, P. (2008) “Cambodian Small and Medium Sized: Enterprises: Constraints, Policies and Proposals for Their Development.” In Lim, H. (ed.), _SME in Asia and Globalization_ , ERIA Research Project Report 2007-5, pp.1-36.

- Banya, P. and R. Waddington (2007) “Getting to Better Regulation: Lessons from East Africa.” Paper submitted to: Donor Committee for Enterprise Development, _Africa Regional Consultative Conference_ : 5-7 November, 2007, Accra, Ghana.

- Beach, D. and R.B. Pedersen (2013) _Process-Tracing Methods. Foundations and Guidelines_ . University of Michigan Press.

- Botswana Ministry of Finance and Development Planning (2008) _National Development Plan 10_ .

Botswana National Vision Council (1996), _A Framework for a Long Term Vision for Botswana_ .

- Caroll, P. (2014): “Ex Ante Evaluation in Australia and New Zealand: the case of Regulatory Impact Assessment”. Paper presented at ECPR Standing Group on Regulatory Governance 5th Bienial Conference 2014

- Council for Law Reporting of Kenya (2013) _Statutory Instrument Act No. 23 of 2013._

- DAI Europe Ltd (2007) _Support to Improve the Legal and Regulatory Environment for Business in Uganda. Phase 2, Final Report._

- Damonte, A., Dunlop, C.A. and C.M. Radaelli (2014) “Controlling bureaucracies with fire alarms: policy instruments and cross-country patterns.” _Journal of European Public Policy_ , 21(9): 1330-1349.

- De Francesco, F. (2012) “Diffusion of regulatory impact analysis among OECD and EU member states.” _Comparative Political Studies_ , 20(10): 1-29.

- Dolowitz, D.P. and D. Marsh (1996) “Who learns what from whom: a review of the policy transfer literature.” _Political studies_ , 44 (2): 343-357.

- Dolowitz, D.P. and D. Marsh, (2000) “Learning from abroad: the role of policy transfer in contemporary policymaking.” _Governance_ , 13: 5-24.

Dunlop, C. and C.M. Radaelli (eds.) (2016) _Handbook of Regulatory Impact Assessment_ . Edward Elgar Publishing.

- Gill, D. and P.J. Intal (2016a) _The Development of Regulatory Management Systems in East Asia. Deconstruction, Insights, and Fostering ASEAN's Quiet Revolution._ Economic Research Institute for ASEAN and East Asia.

33
- Gill, D. and P.J. Intal (2016b) _The Development of Regulatory Management Systems in East Asia - Country Studies_ . Economic Research Institute for ASEAN and East Asia.

Gill, D. (2016) “Defining Regulatory Management Systems.” _ERIA Discussion Paper Series_ .

Government of Kenya (2012) _National Handbook for Public Policy Making_ .

- Government of Uganda (2000) _Medium-term Competitive Strategy 2000-2005._

- Hatzis, A. N. and N, S. (2007) “From Nothing to Too Much: Regulatory Reform in Greece. _NBER Working Paper_ No. 13/2007 _._

- Hertin, J. et al. (2009) “Rationalising the policy mess? Ex ante assessment and the utilisation of knowledge in the policy process.” _Environment and Planning A_ , 41: 1185–1200.

- Independent Evaluation Group (2013) _IEG Evaluative Note, Kenya Regulatory Reform._

- Independent Evaluation Group (2015) _Investment Climate Reforms: An Independent Evaluation of World Bank Group Support to Reforms of Business Regulations._

- International Monetary Fund (2013) _World Economic Outlook Database_ .

- Jacobs, S. (2016) “Towards a Simpler and Practical Approach” in Dunlop, C. & Radaelli, C. _Handbook of Regulatory Impact Assessment_

- Jacobs, S., Ladegaard, P.F. and B. Musau (2007) “Kenya’s Radical Licensing Reforms, 2005-2007: Design, Results, and Lessons Learned.” _Africa Regional Consultative Conference_ : 5-7 November, 2007, Accra, Ghana.

- Jacobzone, S., C. Choi and C. Miguet (2007) “Indicators of Regulatory Management Systems.” _OECD Working Papers on Public Governance,_ 2007/4, OECD Publishing.

- Jalilian, H., Kirkpatrick, C. and D. Parker (2007) “The impact of regulation on economic growth in developing countries: A cross-country analysis.” _World Development_ , 35(1): 87–103.

- Jordan, A.J. and J. Turnpenny, (eds.) (2015) _The Tools of Policy Formulation: Actors, Capacities, Venues and Effects._ Edward Elgar Publishing.

- Kirkpatrick, C., Parker, D. and Y.F. Zhang (2004) “Regulatory Impact Assessment in Developing and Transition Economies: A Survey of Current Practice.” _Public Money & Management_ , 24(5): 291-296.

- Kirkpatrick, C. and D. Parker (2004) “Regulatory impact assessment and regulatory governance in developing countries.” _Public Administration and Development_ , 24: 333–344.

- Kirkpatrick, C. and D. Parker (2012) “Measuring regulatory performance. The economic impact of regulatory policy: A literature review of quantitative evidence.” _OECD Expert Paper_ No. 3, August 2012.

- Kirkpatrick, C. (2016) “Developing Countries.” In Dunlop, C. and C.M. Radaelli (eds.) _Handbook of Regulatory Impact Assessment_ . Edward Elgar Publishing.

- Ladegaard, P. (2005) “Improving Business Environments through Regulatory Impact Analysis - Opportunities and challenges for developing countries.” Paper prepared for the _International Conference on Reforming the Business Environment_ , Cairo, Egypt, 29 November - 1 December 2005.

- Ladegaard, P. and Scott Jacobs (2010): _Regulatory Governance in Developing Countries_ .

Marsh, D. and J.C. Sharman (2009) “Policy diffusion and policy transfer.” _Policy Studies_ , 30:3, 269-288.

Ministry of Finance, Planning and Economic Development of Kenya (2004) _A Guide to Good Regulation._

- Nyaga, R. (2007) “Curbing the Flow – Improving New Policy Proposals through RIA.” Presentation given at the: _Kenya-Tanzania Workshop on Regulatory Reforms,_ 29 August 2007, Nairobi, Kenya.

- OCED (1995) _Recommendation of the Council on Improving the Quality of Government Regulation_ . OECD Publishing, Paris.

OECD (1997) _Regulatory Impact Analysis: Best Practices in OECD Countries_ . OECD Publishing, Paris.

- OECD (1998) _Indicators of Regulatory Capacities in OECD Countries: Analysis and Interpretation_ . OECD Publishing, Paris.

- OECD/APEC (2005) _Integrated Checklist on Regulatory Reform_ . OECD Publishing, Paris.

OECD (2006) _Cutting Red Tape: National Strategies for Administrative Simplification_ .

34
- OECD (2008) _Building an Institutional Framework for Regulatory Impact Analysis (RIA): Guidance for Policy Makers._ OECD Publishing, Paris.

OECD (2009a) _Regulatory Impact Analysis: A Tool for Policy Coherence_ . OECD Publishing, Paris.

OECD (2009b) _Indicators of Regulatory Management Systems_ . OECD Publishing, Paris.

OECD (2010) _Aid-For-Trade Case Story_ . Kenya Business Regulatory Reform Unit (BRUU).

OECD (2012) _Recommendation of the Council on Regulatory Policy and Governance_ . OECD Publishing, Paris.

OECD (2015) _OECD Regulatory Policy Outlook 2015_ . OECD Publishing, Paris.

- Opio-Lukone, V. (2005) “Curbing the Flow - Improving New Policy Proposals through Regulatory Impact Assessment.” Presentation to the conference on _“Cutting red tape in Eastern Africa”._

- PEAP - Government of Uganda (2000) _Uganda Poverty Eradication Action Plan_ .

Peci, A. (2016) “Latin America” in Dunlop, C. & Radaelli, C. _Handbook of Regulatory Impact Assessment_

- Penev, S. and A. Marušic (2011) _Progress in Transition and Reform Implementation in Serbia - Comparing to Other Western Balkan Countries (with Main Focus on Regulatory Reform)._ World Bank Group.

- Radaelli, CA, Dunlop CA, Maggetti M, Russel DJ, The Many Uses of Regulatory Impact Assessment: A MetaAnalysis of EU and UK Cases, _Regulation and Governance_ , vol. 6, no. 1, 2012, 23-45

- Renda, A. (2015) “Regulatory Impact Assessment and regulatory policy.” In OECD (ed.) _Regulatory Policy in Perspective. A Reader's Companion to the OECD Regulatory Policy Outlook 2015_ . OECD Publishing, Paris.

- Sanderson, I. (2002) “Evaluation, Policy Learning and Evidence-Based Policy Making.” _Public Administration_ , 80: 1–22.

- Staronova, K. (2010) “Regulatory impact assessment: Formal institutionalization and practice.” _Journal of Public Policy_ , 30(1): 117-136.

- Turnpenny, J., Radaelli, C.M., Jordan, A., and K. Jacob (2009) “The policy and politics of policy appraisal: Emerging trends and new directions.” _Journal of European Public Policy,_ 16: 640-653.

WBG (2008) _Uganda’s RIA Reforms._ WBG (2009) _Making it Work: ‘RIA Light’ for Developing Countries._ WBG (2010a) _Governance Frameworks and Tools for Effective Regulatory Reform_ .

WBG (2010b) _Regulatory Capacity Review of Kenya._ WBG/IFC (2012) _Advisory Services Completion. Kenya Regulatory Reform_ .

- WBG (2013) _Worldwide Governance Indicators_ .

WBG (2015) _Doing Business 2015_ . Doing Business Report Series.

World Economic Forum (2015) _The Global Competitiveness Report 2014-2015_ .

Welch, D. (2007) “Regulatory impact assessment in East Africa.”

- Zhang, Y.F. and Thomas, M. (2009) “Regulatory reform and governance: a survey of selected developing and transition economies.” _Public Administration and Development_ , 29: 330–339.

- Zhang, Y.F. (2010) “Towards Better Regulatory Governance?” _Public Management Review_ , 12:6, 873-891.

35
# Annex 1: Data points for mapping of RIA reforms and methodology for scoring

_The following data points were used to map the RIA reforms. The first set of questions focus on broad generic data, used to show that: (i) the country is within scope of the study; to determine whether reforms are within the first two years of implementation (and thus “too early to call”); and whether there are overlapping reforms. Additionally, it lists secondary data sources from where the data needed for this study can be derived:_

- Country name:

- General info

- Income level

- Period in which the reform took place

- Presence of donor/IO support

- Presence of overlapping reforms

- Source of data

_The next step of the analysis looked at specific design factors and scored their presence in the RIA reform (methodology is further explained in the report):_

## **1) Formal political commitment**

- **0 p** : No strategic policy document for RIA was presented during the reforms.

- **0.5 p** :  At least one strategic policy document was drafted by the executive during the reform, which explicitly endorses and commits the government to RIA reform (although it lacked an operational long-term plan for RIA adoption/RIA mainstreaming within a broader, wholeof-government regulatory reform and screening mechanisms on the stock of existing legislation).

- **1 p:** At least one strategic policy document was drafted by the executive explicitly endorsing and committing the government to RIA reform. This document provides also an operational long-term plan for RIA adoption/RIA mainstreaming within a broader, whole-ofgovernment regulatory reform and/or includes screening mechanisms on the stock of existing legislation.

## **2) Integration in rule-making process**

- **0 p** : The reform did not produce legal basis for RIA.

- **0.5 p** :  The reform has produced a legal requirement mandating RIA, such as a law, decree or legislative drafting memorandum (but it did not clarify and instruct on who is going to draft the RIA, when within the policy-making cycle, or provide a clear provision for RIA coverage).

- **1 p** : A legal requirement mandating RIA was introduced, such as a law, decree or legislative drafting memorandum which also clarifies and instructs who is going to draft the RIA, when within the policy-making cycle, provides a clear provision for RIA coverage (such as: applied only to bills? Only to business-related regulation? Only above a certain estimated impact?) and puts forth the basic content of the RIA.

## **3) Oversight body**

- **0 p** : No RIA oversight body established.

- **0.5 p** :  A body was established tasked with coordination and planning of RIA activity with a clear mandate specifying RIA oversight prerogatives (quality control with possibility to “return” poorly drafter RIAs), but no or limited major tasks in addition to this.

36
- **1 p** : A body tasked with coordination and planning of RIA activity was created, which has a clear mandate specifying RIA oversight prerogatives (quality control with possibility to “return” poorly drafted RIAs), which is also tasked with other tasks such as issuing RIA guidelines and carrying out RIA advocacy and training.

# **4) RIA coverage, guidelines, methodologies**

- **0 p** : The reform produced no guidelines, manuals, checklists and templates for drafting RIA documents.

- **0.5 p** :  The reform produced guidelines, manuals, checklists and templates for drafting RIA documents. They provide clear indications on analytical methodologies as well (see WBG, 2009: 31) and elaborate (at a minimum) on the principle of proportionality. However, they are not tailor-made clearly matching long-term goals of the executive and available administrative capacities.

- **1 p** : The reform produced guidelines, manuals, checklists and templates for drafting RIA documents. They provide clear indications on analytical methodologies as well (WBG, 2009: 31) and elaborate (at a minimum) on the principle of proportionality. These are tailor-made clearly matching long-term goals of the executive and available administrative capacities and they set out strategy for data collection.

# **5) Consultation**

- **0p** : The reform did not produce procedures for integrating public consultation with RIA.

- **0.5 p** :  RIA templates were produced, setting minimum standards for the consultation procedure (topology of stakeholders, method of consultation and duration) early in the RIA process; moreover, and there is a clear indication for integrating the opinions of the consulted parties and their feedback in the RIA document (but consultation does not take place sequentially on different drafts of the RIA document (i.e. if consultation is cross-cutting along the RIA process)).

- **1 p** : RIA templates were produced, whereby minimum standards for the consultation procedure (topology of stakeholders, method of consultation and duration) are established and engrained early in the RIA process; moreover, there is a clear indication for integrating the opinions of the consulted parties and their feedback in the RIA document and consultation takes place sequentially on different drafts of the RIA document (i.e. if consultation is cross-cutting along the RIA process).

# **6) Capacity building**

- **0 p** : There was no training on RIA within the reform.

- **0.5 p** :  The reform package included technical/methodological training sessions for civil servants (but not a systematic training system).

- **1 p** : Training included the piloting of one or more RIAs on existing legislation or relevant and topical legislative proposals and/or training included not only regulators, but also members of coordination/oversight bodies. The construction of local RIA transfer capacity (“training of trainers”) and the organization of study tours indicate a sound capacitybuilding activity.

_The final step looked at whether the RIA system appeared “alive” at the time of writing. The simple criterion was whether RIA documents were being published._

- 7) Functional / non-functional two years after reform commencement

37
# Annex 2: Semi-structured survey questionnaire

_This questionnaire includes questions asked during semi-structured interviews in the in-country case studies. Interviewees included government representatives, NGOs, various organizations, and donors. Questions were selected and adjusted in the context of the interviewee’s background._

## Tracing the RIA reform process

_Information below for all different initiatives, such as:_

- _Comprehensive RIA reforms with multiple components, such as training, oversight, guidance materials, pilots, etc._

- _Individual ad hoc training sessions_

- _Ad hoc support to draft guidance materials and templates_

- _Ad hoc support in draft legislation for the introduction of RIA_

## Origins of RIA

1. When did the reforms take place?

2. How was the idea of RIA reform first conceived (e.g. government learned about it through international networks, suggested by donors)?

3. What was the problem it was supposed to address/What was the key motivation for its introduction? What were the expected benefits? How did the government learn about the benefits as a solution to its problem?

4. Was there an internationally funded program that foresaw the introduction of RIA? Was it part of a larger donor funded program and what type of program was that (investment climate, trade, governance, etc.)? Was RIA required through conditionality in donor programs, or through requirements in trade agreements, or else?

5. How was the cooperation between the government and the involved donor or other international organization framed? How was the agreement between the parts designed?

6. Which governmental body acted as main counterpart to the donor/international organization? Where was it located within the government’s structure? Why was this organization selected (own initiative, previous relations with donors, delegation within government)? Do you consider that the chosen body was fit for the purpose in terms of capacity and resources?

7. How long did the diagnostic phase last? What actors participated in the diagnostic? What was studied in the diagnostic and who carried it out?

8. Were there other major public policy reforms (as part of other programs) overlapping/running in parallel with RIA? Were these administered by the same organization in government?

9. Were there actors (political, bureaucratic, private, etc.) that openly opposed RIA?

10. How were reform champions identified (by donors? by government?) and what role did they play in the reforms? Can we find out something about the overall support across government, and the political clout of the champion?

## Introduction and governance of the (planned) RIA system

11. What were supposed to be the key components of the RIA system in your country (governance template)? What was supposed to be the institutional framework/governance arrangement for RIA (e.g. central coordinating or oversight body/RIA unit; officials/RIA units in departments, existing body is responsible for RIAs [Cabinet, Parliament/Congress, National Planning Body, etc.])?

12. Can you briefly elaborate on the roles and activities of different key actors in reform design and early stage of implementation? How was cross-government coordination and buy-in ensured?

13. Do you think that the different actors – including donors, government, external RIA experts - involved in reform implementation shared a common understanding of and aims for RIA reform?

14. How many workshops/seminars/raising awareness events were run? Who conducted such events? And who participated in such events? How were the participants selected?

38
15. Did a specific model of “RIA system” (governance and practice) emerge from these events?

16. Was the specific model presented as one of several models as alternatives for the government?

17. How was the specific RIA system adjusted to country-specific environment and problems?

18. Was there a detailed action plan for the introduction of the chosen model of RIA? Did the government conduct study tours to other countries and what was the impression? How much time was envisaged for the reforms?

19. How was RIA supposed to be introduced and scaled up (e.g. “big bang”, pilot ministries, pilot areas, from simple to advanced)?

20. How was RIA established at first in practice? Bill, governmental/ministerial directive, decree, strategy document without legal obligations, etc. At what stage of the reform process was this introduced?

21. Was RIA supposed to apply to all regulations? Were there exclusion criteria? Was RIA supposed to apply to upcoming, existing regulations/bills or both?

22. Did the chosen model foresee the establishment of an RIA oversight unit? What was supposed to be the role, composition, sources of funding and operational tasks of the RIA oversight body? If RIA systems were introduced by donors, how were the capacity and resources required discussed with government prior to reform?

23. Overall, do you consider that RIA, as planned in the implementation stage, was well integrated into the policy making and governance process (was it streamlined into practices of public servants, were consultation procedures and training related to other similar practices in government, etc.)?

24. What were the donor contributions (money for operation of institutions and systems, or simply TA)?

25. What were the main hurdles in the establishment of a legal obligation for RIA?

26. What were the main hurdles in the establishment of an RIA oversight unit?

# Practice of RIA

27. What training was conducted? Who was training aimed at? Who conducted it? How many officers were trained? Were there pilot RIAs conducted and published?

28. How was RIA training received by the trainees? Were the training sessions closer to a cooperative endeavor or to top-down teaching? Did civil servants generally _buy_ the new RIA mentality?

29. Did the reform introduce a training program? Was it internalized in broader civil servant training?

30. Were guidelines/handbooks for drafting RIAs produced (as a result of the training)? Have they been published? Did a template for RIA clearly emerge?

31. Drawing on these guidelines/handbooks, what were supposed to be the essential steps (the template) in the RIA process? Were certain analytical tools (e.g. Cost Benefit Analysis, Standard Cost Model, Multi Criteria Analysis, etc.) encouraged or required?

32. How was stakeholder consultation handled in the chosen RIA template? Were other government departments or agencies supposed to be consulted? What provision was there for quality control or evaluation of the RIA system (i.e. annual reports on RIA implementation)?

33. Do you consider the chosen template appropriate in terms of existing administrative capacity and resources?

34. How did oversight work? What were the sources for the oversight unit’s power? How did it work in practice – did the oversight unit succeed in acting as a watchdog for quality in RIA? How did the role change over time?

35. Do you have some final considerations about the introduction of RIA in your country, what worked and what did not?

## Depending on the answers to the previous questions – Control variables

36. Were there any other contributing factors for the introduction of RIA (i.e. increasing competition with neighboring states, a deliberate move towards a market economy)?

37. Were there any events that triggered the need and therefore introduction of RIA (i.e. a crisis, election, a particular case in the media, requirements of regional or international obligations or agreements)?

39