# 2. Analytical framework and research design

The analytical framework of the study is designed to address a number of key questions about the design, implementation, and dissemination of RIA systems in developing countries. This entails a descriptive snapshot of RIA reforms implemented in developing countries over the last 15 years (2001-2016), an evaluation of the outcomes in terms of functionality of these RIA systems, and an attempt to identify causes for the implementation challenges. A summary of the five key research questions and the associated research design is summarized in Table 1 and elaborated in the subsequent sections.

**Table 1.** Research questions, approach and data sources

|Researchquestion|Approach& Operationalization|Data sources|
|---|---|---|
|**How many developing**<br>**countries have launched RIA**<br>**reforms?**|Identification and template mapping of all RIA reforms<br>launched in developing countries between 2001-2016.|Desk research; local World<br>Bank staff; external experts;<br>http://rulemaking.worldbank.<br>org/ria-documents|
|**What are the main design**<br>**features of RIA reforms in**<br>**developing countries?**|Mapping of all identified RIA reforms according to full or<br>partial observation of six good practices: i) Formal<br>political commitment; ii) Legal integration of RIA in the<br>policy process; iii) Presence of a regulatory oversight<br>body; iv) Presences of RIA guidelines and methodologies;<br>v) Presence of consultation mechanisms; and vi) Presence<br>ofcapacity building andRIApiloting activities.|Desk research; local World<br>Bank staff; external experts.|
|**Were the RIA reforms**<br>**successful or not?**|“Successful RIA reforms” are defined as RIA systems which<br>are functional i) two years after reform launch ii) and<br>which produce and publish RIA statements on a continuous<br>basis.|Desk research; local World<br>Bank staff; external experts.|
|**Can particular design features**<br>**be associated with successful**<br>**and unsuccessful RIA**<br>**reforms?**|Creation of a “RIA good practice index” based on full or<br>partial observation of the above good practices. RIA<br>reform scores from 0-6. Reforms with scores of 3.5 or<br>above categorized as “goodpractice reforms”.|Authors of the study.|
|**Can we identify key factors**<br>**causing certain RIA reforms to**<br>**be unsuccessful?**|Testing of six hypotheses to understand how they can alter<br>the success RIA reforms.|Desk research; four case<br>studies; stakeholder interviews.|

## How many developing countries have launched RIA reforms?

The first step of the study was to identify the number of RIA reforms implemented in developing countries in the last 15 years. The data collection involved the mapping of RIA reforms in developing countries across a range of data points, including period implemented, the role of donors and international organizations, and other factors in the reform environment such as reforms that have overlapped with the RIA reforms. The empirical research for this study does not rely on surveys and questionnaires administered to public officials. It primarily relies on existing larger surveys and databases (especially the World Bank Group’s Global RIA Database and Global Indicators of Regulatory Governance<sup>1</sup> ), contributions by in-country based World Bank staff, as well as expert validation by Word Bank Group staff. The information is triangulated among several secondary sources including: reports by international organizations, academic publications, case studies, governmental reports, and consultant reports, and to some extent also newspaper articles and blog posts. This approach was chosen in part due to the limited response rate as well as validation challenges related to email and postal surveys administered to public officials (see for example Kirkpatrick et al. 2004). The template for information collection can be found in Annex 1.

> 1 This can be accessed at _rulemaking.worldbank.org/ria-documents_ and _rulemaking.worlbank.org._

7