metadata: authors: 0: author: Narayan,Ambar
1: author: Narita,Renata
2: author: Skoufias,Emmanuel
count: Dominican Republic
docna: 0: docna: Does access to information empower the poor
            ? evidence from the Dominican Republic
docty: Policy Research Working Paper
subtopic: National Governance,Regional Governance,Government Policies,Public Sector Economics,Educational Sciences,Social Accountability,Youth and Governance,Public Financial Management,Public Finance Decentralization and
            Poverty Reduction,Inequality,Local Government
abstracts: cdata!: This paper summarizes the results of the
            impact evaluation of the Access to Information pilot project
            on empowerment of citizens in poor municipalities in the
            Dominican Republic. Among the dimensions of empowerment
            investigated are civic knowledge, awareness and use of the
            right to information, perceptions of and trust in public
            services and institutions, civic participation, and measures
            of local governance.  Data were collected in two rounds: a
            baseline round at the end of 2010 and a follow-up round in
            mid-2012. No impact is found on awareness and the use of
            information under the specific Access to Information rules.
            However, it is observed that individuals address more
            general complaints to governments as a result of the Access
            to Information program regardless of whether these are
            classified under the ATI law or not. Some positive and
            statistically significant impacts are found on local
            government responsiveness, prioritization and decisions
            about the municipal budget, and trust in and satisfaction
            with some local government services.
geo_reg_key: 119228,517206,517191
repnme: repnme: Does access to information empower the poor
            ? evidence from the Dominican Republic
display_title: Does access to information empower
            the poor evidence from the Dominican Republic
listing_relative_url: /research/2014/05/19591427/access-information-empower-poor-evidence-dominican-republic
topicv3: Local Governance m1326016 737,Poverty m1326687 695,Budget Preparation and Management
            Techniques m1326766 2060,Education m1327048 470,Right to Information m1327454 2031
publishtoextweb_dt: 2014-05-01T00:00:00Z
no_of_pages: 49
author_keys: 0: author_key: 167324
1: author_key: 905759
2: author_key: 177660
model_extractions: WPS6895 Policy Research Working Paper 6895 Does Access to Information Empower the Poor? Evidence from the Dominican Republic _Emmanuel Skoufias Renata Narita Ambar Narayan_ The World Bank Latin America and the Caribbean Region Poverty Reduction and Economic Management Department May 2014
Policy Research Working Paper 6895

# **Abstract**

This paper summarizes the results of the impact evaluation of the Access to Information pilot project on empowerment of citizens in poor municipalities in the Dominican Republic. Among the dimensions of empowerment investigated are civic knowledge, awareness and use of the right to information, perceptions of and trust in public services and institutions, civic participation, and measures of local governance. Data were collected in two rounds: a baseline round at the end of 2010 and a follow-up round in mid-2012. No impact is found on awareness and the use of information under the specific Access to Information rules. However, it is observed that individuals address more general complaints to governments as a result of the Access to Information program regardless of whether these are classified under the ATI law or not. Some positive and statistically significant impacts are found on local government responsiveness, prioritization and decisions about the municipal budget, and trust in and satisfaction with some local government services.

This paper is a product of the Poverty Reduction and Economic Management Department, Latin America and the Caribbean Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http:// econ.worldbank.org. The authors may be contacted at eskoufias@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **DOES ACCESS TO INFORMATION EMPOWER THE POOR?**

**EVIDENCE FROM THE DOMINICAN REPUBLIC**

**Emmanuel Skoufias (World Bank) Renata Narita (Universidade de São Paulo) Ambar Narayan (World Bank)**

## **JEL classification:** H0, C14

**Keywords:** Access to Information law, empowerment, program evaluation

_Corresponding author:_ Emmanuel Skoufias, The World Bank, 1818 H Street NW, (MSN: I 4- 405) Washington DC 20433-USA. tel: (202)458-7539. fax: (202) 522-3134. e-mail: eskoufias@worldbank.org _Acknowledgements:_ The authors are grateful to Sarah Keener, Helen Grandvoinette, and Anupama Dokeniya for valuable suggestions.
# **1. Introduction**

The monitoring of public institutions by citizens is widely believed to be a necessary condition for the successful implementation of institutional reforms aiming towards more effective and accountable governments. In the last decade, governance and institutional reforms have been promoted in variety of middle income and developing economies. The implementation of these reforms recognizes that the quality of institutions in these countries plays an important role for economic development and poverty reduction. [Keefer and Knack (1995, 1997), Mauro (1995), Hall and Jones (1999), La Porta et al (1999), Acemoglu, Johnson and Robinson (2001)].

One example of such reforms is the right-to-information (RTI) laws – laws that establish the rights of citizens to have access to public information or exercise such a right already written in the constitution.<sup>1</sup> Since the 1980s, RTI laws have been implemented in more than 90 countries, starting with developed democracies. However, most of the new adopters are countries in Eastern Europe, Asia, Africa, Middle East and Latin America facing difficult governance environments and pressures from international aid agencies and intergovernmental bodies to adopt reforms that improve transparency and create more established democracies (Dokeniya, 2013) Having a law of access to information about government decision-making and expenditures can help citizens hold governments accountable to their citizens. But simply passing a law does not necessarily or immediately translate into improved governance (Banisar 2006; Calland and Bentley 2011). Rather, a number of conditions have to be in place for freedom of information laws to increase government transparency. In addition to a functioning government system, there must be some degree of political will on the part of government, a civil society actively engaged in pushing for and monitoring use of the law, and an involved and active media. Thus programs towards improving supply and demand for information are thought to be the way to promote a more effective use of the RTI laws.

> 1 The acronyms RTI (right to information), ATI (access to information) and FOI (freedom of information) will be used interchangeably for laws of access to information.

2
One example of such programs is the Access to Information (ATI) pilot project in the Dominican Republic (DR). Following the adoption of the Freedom of Information laws in 2004 and 2007, this project has aimed to educate citizens in poor municipalities about their legal right to request and receive information from government institutions, with the long-term objective to increase transparency and accountability in public service provision. Project implementation begun in the third quarter of 2009 completed in December 2011. The insights gained from this pilot are expected to influence the next phase of the program as well as Bank support to initiatives aimed at increasing accountability and transparency in government.

To the best of our knowledge, little is known about the effects of such interventions in the Dominican Republic or in other developing countries. The little evidence that exists provides useful qualitative insights and before-and-after comparisons. In Mexico, for example, findings suggest that while the average person rarely exercised the right to obtain information; the most uneducated groups almost never used it. In response, the Federal Freedom of Information Institute developed the _Comunidades_ project to raise awareness of the law among the country’s most disadvantaged individuals. The project involved twenty civil society organizations that worked with people in poor communities of nine Mexican states to teach them about the potential uses of the law and procedures for requesting public information. An evaluation of this project associates the use of the law in these communities with an increase in trust, training, and monitoring of the process (GuerreroAmparan and Toledo, 2009).

Similarly, Roberts (2010) studies the Right-to-information Act in India and finds that a substantial barrier to accessing information is lack of awareness about rights under the Act, particularly in rural areas, among females and socially disadvantaged groups. In addition, the author points other correlates of low use of the law such as lack of administrative capacity, low enforcement and hostility faced by individuals when placing requests.

A notable shortcoming of the studies mentioned above is the absence of a credible counterfactual that allows “causal” inferences about the effects of ATI interventions on 3
different dimensions of empowerment among citizens.<sup>2</sup> This paper is an effort to fill this lacuna through the blending of qualitative methods with rigorous quasi-experimental quantitative methods employed to construct a credible counterfactual group.<sup>3</sup> Specifically, we employ the difference-in-differences method on individual survey data collected before and after the completion of the ATI pilot project using observations from treatment municipalities and a comparison group of municipalities selected by the method of “nearest neighbor” matching.

The ATI pilot relates to empowerment in many ways. First, information itself is power. In particular civic education allows individuals to be better equipped to take advantage of opportunities, access services, exercise their rights, and demand better institutions and services. In addition knowledge about the freedom of information law’s roles on institution development could turn individuals into legal activists empowered to exercise the law.<sup>4</sup> Second, participation and inclusion of the poor together are critical at ensuring that limited public resources represent local priorities and do not exacerbate inequalities that could be reinforced by the exclusive participation of the local elites. The World Development Report 2004 is devoted entirely to the role of citizens’ voice and participation in holding politicians accountable. Empowerment of poor citizens can increase their influence in policymaking or align their interests with those of politicians, thus making basic services reach the poor. The report also defends that “non-governmental or civil society organizations can also help amplify the voice voices of the poor, coordinate

> 2 Our research also contributes to the literature on information and participatory programs. Previous impact evaluations of such programs in developing countries have mostly focused on public health and education service delivery. Studies typically find positive impacts on knowledge about health issues (Duflo, Dupas, Kremer and Sinei, 2006), on awareness of parents about the school roles or education (Pandey, Goyal and Sundararaman, 2007); Keefer and Khemani, 2011), on participation in community meetings or in the monitoring of service providers (Pandey et al, 2007; Pandey, Goyal and Sundararaman, 2007; Bjorkman and Svensson, 2009), in volunteering for small group community work (Banerji, Duflo, Glennerster and Khemani, 2010), on investment in schools (Reinikka and Svensson, 2011; Bjorkman, 2008; Andrabi, Das and Khwaja, 2009), and on parental investment in education (e.g., Keefer and Khemani, 2011).

> 3 Goncalves (2009) is perhaps the closest study to ours. She analyzes the gradual implementation of Participatory Budgeting (PB) across Brazilian municipalities. This program is similar to the ATI pilot, but more specifically, it consists of a participatory mechanism in which citizens negotiate with government officials over budget allocation and investment priorities. Using the difference-in differences method, she finds that the adoption of PB increased the spending that matches popular preferences as expressed in the participatory fora. In particular, it almost doubled spending in health and sanitation in the PB municipalities compared to the municipalities where PB was not implemented.

> 4 For related discussion on legalist studies, see for example Gauri (2009), World Bank (2005) and references therein.

4
coalitions to overcome their collective action problems, mediate on their behalf through redress mechanisms, and demand greater service accountability.” Finally, trust in state institutions and civil society contributes to greater participation and builds pressure for improved local governance, whether in setting priorities for expenditures, or providing access and quality of basic public services.

We explore several dimensions of empowerment by analyzing the short-run effects of the intervention on intermediate outcomes measured by indicators of individual civic knowledge, awareness and use of the right to information, perceptions of and trust in public services and institutions, civic participation, as well as measures of local governance.

The next section provides some background on the Access to Information project in the Dominican Republic. Section 3 presents the evaluation design and the combination of quantitative and qualitative methods used to evaluate the impacts of the ATI program. Section 4, discusses the findings while section 5 concludes with a discussion of the implications of the evaluation findings for the expansion of the pilot to the national level.

# **2. Background on the ATI Program**

In July 2004, the Dominican Congress approved the Law of Free Access to Information (FOI), [no. 200 of 07/2004], which makes it compulsory for government agencies to make information accessible to the public by allowing every citizen to monitor government actions. This kind of information includes budget allocation, status of public works projects, program implementation status, procurement and the government’s payroll, among others. The law also specifies the creation of an Office of Access to Information (OAI) in every government agency. These offices must guarantee permanent free access to up-to-date public information to all citizens. All agencies must establish information systems that are capable of providing this information on demand, and are liable to penalties if they refuse to provide information to the public.

With the support of the Japanese Social Development fund and the World Bank, the Dominican government launched an intervention pilot project with intervention components aimed at enhancing both the supply of and the demand for public information 5
in seven poor municipalities in the DR. (see Box 1) Grupo Gestion Moderna (GGM), a consultancy firm, and Centro Juan Montalvo (CJM), a local NGO, were jointly selected to implement the project.

The project included a number of activities aiming to increase the supply and demand for information. On the supply side GGM worked with local government agencies to raise their awareness about the importance of complying with the law. GGM also assisted local government agencies with organizing their management information systems and with establishing their OAI. On the demand side, CJM worked with local community organizations to raise their awareness about the importance of exercising their right to access public information. The project trained community organizations on a range of issues, including what types of information can be requested, how to request it, what to do in the case of refusal, and how the information once received can be used to hold municipal governments accountable. The knowledge was transmitted through workshops organized by CJM in different community organizations in each treatment municipality.

# **3. Evaluation Design and Data**

The intervention was carried out at the municipality level. A total of seven municipalities were selected by the program implementers, Centro Juan Montalvo (CJM) and Grupo Gestion Moderna (GGM). These municipalities were selected from five provinces covering different regions in the country, urban and rural areas, based on their relatively high poverty rates (above 40 percent, province level), willingness of the local mayor and council to cooperate in developing information systems and opening offices of information, and the existence of a functioning participatory budgeting process.

Given that the treatment was not randomly selected and the lack of other sources of data to evaluate the outcomes of interest, matching was a natural choice. Each intervention municipality was matched with a group of five municipalities using nearest neighbor matching to minimize the statistical distance between the units based on population size (total, young and elderly), poverty rate, quality of life index, illiteracy, unemployment and remittances rate; the proportion of children per household, mean adult education, fraction 6
of male household heads, proportion of households lacking basic services in energy, garbage collection, subject to contamination by stagnant water and trash accumulation. As a final step, only one municipality, the first or the second closest, was selected based on consultation with the program implementers as a “control” for each municipality in the treatment group. Thus, a total of seven treatment and seven control municipalities were selected into the study. It is important to bear in mind that the participatory budgeting process is also present in the control municipalities.<sup>5</sup> It was not possible to ascertain the willingness of the local mayors in the control municipalities in developing information systems and opening offices of information as was done in the process of selecting the seven treatment municipalities. Table A.1, in the Appendix, lists the municipalities which entered the quantitative impact evaluation and indicates those selected for the qualitative baseline study.

Quantitative data were collected in two rounds for a broad range of indicators from the seven ATI municipalities, and from seven comparable municipalities that did not receive project support.<sup>6</sup> The baseline survey round took place between December 2010 and January 2011, prior to the implementation of the ATI pilot program. The follow-up round took place in June 2012, six months after the completion of the intervention activities (in December 2011). The study was designed to detect a 10% relative increase in the primary and related outcomes indicating knowledge of the specific rights under the ATI law, satisfaction with requested information from public offices, civic participation, local governance, trust in public and non-public institutions, and perception about main public services. This resulted in a sample size of 1,932 households (966 households for the treatment and 966 for the control group), representing seven municipalities per treatment group, 10 clusters ( _estratos_ ) per municipality, and around 13 to 14 households per cluster.

> 5 Additional details on the potential role of participatory budgeting in the impact evaluation of the ATI pilot are provided below.

> 6 Gallup Dominican Republic was contracted to conduct the field work for the baseline and end-line surveys. All instruments were translated into Spanish, and pre-tested prior to use in the field. Data collection instruments were administered to respondents in Spanish by native speakers.

7
Both the treatment and control groups comprise a representative sample of the population of households in the 14 municipalities of the study in each round.<sup>7</sup> The baseline survey above was complemented by the collection of qualitative data that included twelve focus group discussions (separate groups were organized with men and women, with a total of 95 participants) that took place in two treatment and two control communities with representatives of civil society organizations (CSO), individual interviews with representatives from community organizations, including neighborhood associations, women’s organizations, and a range of other social, cultural and development associations; radio and print journalists, and three mayors who had been in office about a month before the survey took place (Dudwick et al, 2012). Meetings were also held with two to three municipal council members in each of the four communities. An operational or process evaluation was also carried out to verify consistency between project design and actual implementation. The process evaluation report shed light on the challenges faced during the course of the intervention and helped interpret the findings of this impact evaluation.

The combination of quantitative and qualitative data provided a very good basis for understanding the dynamics of change in the pilot communities. The group and individual discussions revealed the complexity of the context and the many activities already taking place in municipalities, including donor projects and NGO activities, directed at increasing people’s awareness of public affairs. Another issue that surfaced was the pervasive fear that requesting information would lead to public officials to see them as confrontational. Overall, the main issues identified during the baseline and implementation of the ATI project may be summarized as follows.

Six years after the FOI law was enacted, only about twenty-six percent of respondents knew about transparency legislation. This generally reflects the low levels of education in the poor communities in the study, where just under 10 percent of households had a member with a university degree. Members of community organizations however

> 7 Attrition is not an issue since the two rounds of survey represent cross-sections of households in the 7 treatment and 7 control municipalities (i.e. not a panel of households). The follow-up survey contained 1,918 interviewed households. 8
were more likely to be aware of the law, even when characteristics such as education, occupation, gender, and age were taken into account.

Community organizations are the principal conduits for obtaining information. This is linked to (i) existing transparency organizations such as _Participacion Ciudadana_ who have worked with local NGOs to disseminate information about Participatory Budgeting and transparency legislation, (ii) improved capacity and empowerment of many CSOs in the past ten years, especially those representing urban and relatively better educated segments of the population, (iii) concerns that officials would consider requests for information about financial matters confrontational, and respond by firing a close family member or friend working in the public sector. Other individuals also reported fear of physical reprisals, related to a long history of authoritarian regime.

Local politicians and civil servants were considered as the most serious bottlenecks when it comes to accessing information. Local mayors are very powerful as acknowledged by community representatives and municipal officials. On the other hand, local officials were aware of rising public expectations, and understood they were under greater scrutiny from CSOs and media. Yet, they stressed the financial challenges faced in setting up mechanisms for providing information to public, they also expressed their intention to disseminate information through the local mass media, government bulletins and newsletters but were vaguer about setting up procedures for responding to information requests. This is probably a reflection of their perception about what access to information should be or simply from the fact that relatively few officials knew the procedures for requesting information under the FOI law, and some were unsure about whether individual citizens could pursue requests.

A recent process evaluation on the status of ATI project implementation suggested that this project fulfilled its objective to establish Social Audit Committees (SAC) in each municipality of intervention. Further, all SACs participated in a broader national network ( _Foro Ciudadano_ ) through the local support from Centro Juan Montalvo. This connection would guarantee that requests for information would be placed formally and through an established national organization, thus ensuring a successful and speedy delivery of information.

9
Despite this, the SACs formed lacked basic governance rules. A presidential decree establishes that five members of any SAC should be selected during an open meeting with community organizations. Any resident in the community can become a member, except elected civil servants or those working directly in the activities executed by the public institution hosting the SAC. In addition, the creation and composition of the SACs should be notified to the Department for Prevention of Administrative Corruption (DPCA), thus preventing the establishment of long-lasting commissions which could lead to conflict of interest and capture. The implementation evaluation report reveals that the creation of SACs in some municipalities did not follow these rules. Many of the SAC members in the municipalities of the intervention are civil servants or CBO members with limited skills to monitor infrastructure projects or to analyze municipal budgets, hence showing low independence and little capacity within the SACs.

While there were oversight institutions, the GGM and the CJM, supervising the actions of SACs and working together with the CBOs _probably_ played a major role in ensuring that local governments provided information. As soon as the ATI project finished in November 2011, SACs in at least three out of seven municipalities faced severe problems to finance their operations, since participants could not afford paying for transportation. As a consequence, meetings and requests for information stopped after the completion of the ATI pilot project.

These challenges encountered during the pre-program and implementation phases of the program provide useful explanations for the observed impacts (or absence of impacts) of the ATI pilot program.

# _a. Indicators of impact_

Quantitative data were collected to investigate whether the ATI project had impacts on knowledge, access to information, civic participation, local governance, trust in public and non-public institutions and perception about main public services. These are believed to be important intermediate mechanisms for the information legislation to ultimately have positive results impacts on public transparency and accountability. Specifically, we investigate the ATI impacts on knowledge about the FOI law, existence of offices of access 10
to information, knowledge on existing related processes/projects such as the Participatory Budgeting and other civic knowledge for example local and central government responsibilities in terms of main services provision, local and central government activities, projects and knowledge on who is the mayor or the president.

In terms of use of information, we look into whether individuals have asked for public information to a government official or department. As for civic participation, we investigate individual participation into collective action activities: participation in CBOs, in Participatory Budgeting meetings, in protests or strikes. In addition, we investigate whether the individual placed a complaint to the government, voted in municipal elections, or supported campaigns for city council candidates.

While knowledge, access to information and civic participation are intended to measure the first order impact on the demand side, local governance is intended to measure the direct impact on the supply side of information. Proxies for local governance indicate whether local government consults the citizens’ opinions for project prioritization, for making decisions and whether the government is spending in key areas, always as perceived by households. This said, it is possible that an improvement in local governance is driven by an increase in awareness of individuals pushing for greater transparency rather than the supply side directly improving government information provision. While are not able to disentangle supply from demand effects, we can infer which one is likely to be more predominant given the intervention design and how the program was implemented in practice.

Regarding individual perception of public institutions, public services and CSOs that could improve the collective participation of individuals and demand information, we investigate whether individuals trust government institutions (central, local, justice courts and police) as well as whether they trust non-government institutions such as CBOs, NGOs, political parties, church, besides neighbors in their community. In addition, we look at the satisfaction of individuals with each of the key public services in their community.

11
# _b. Balancing tests_

The baseline quantitative paper presents several balance tests and detailed descriptive analysis for all the variables contained in the questionnaire. In this paper we present the balancing tests only for the main variables. Tables A.1 and A.2 show comparison of means between treatment and control groups at the baseline. Given that we used matching to obtain the control group, the p-values refer to a test in which we compare means in the treatment group with ‘weighted’ means in the control group.<sup>8</sup> In terms of the main household characteristics, Table A.1 shows that the null hypothesis of equality of means between treatment and control groups is _not_ rejected for most key variables. The tests where the null is rejected show that at the baseline the treatment may be slightly worse than the control group in terms of basic services of sanitation, sewer, electricity, water, garbage collection and house property. All other variables show great balance of household characteristics between treatment and control groups.

Table A.2 reports the main outcome variables. All except one are found very similar across treatment and control groups. Under a 90% confidence level, trust in CBOs is lower in the ATI municipalities than in the control ones. Other than this, all other tests indicate that at the baseline, there were no major differences between outcomes in the treatment and control groups.

# _c. Difference-in-Differences Matching (DDM) estimator_

Our estimation relies on quasi-experimental methods. The program implementers chose the treatment municipalities and we paired the treated municipalities with comparison or control municipalities using a matching procedure. Baseline analysis showed that most key exogenous and program variables are well balanced, which reinforces our attempt to obtain robust impact estimates. Our strategy then consists of comparing means of outcomes between municipalities assigned to treatment to mean outcomes in the control municipalities before and after the initiation of the ATI pilot. The parameter of impact that we estimate is the intent-to-treat effect (ITE), which is an estimate of the impact of making

> 8 These weights are proportional to the number of times a control municipality (from the universe, excluding the treatment group) is matched with another municipality in the treatment group.

12
the ATI program available in the treatment municipalities regardless of whether households actually participate (by choice or not) in the training sessions offered by the ATI program. The ITE is of particular interest to policy makers since it captures the operational efficiency of the program, such as inefficiencies in the delivery of the program, as well as any potential general equilibrium or spillover effects of the program. Below we detail our estimation method.

The typical matching estimator pairs each municipality in the treatment group to an observationally similar municipality (or several) in the control group and compares the differences in their outcomes as the effect of the program (Rosenbaum and Rubin, 1983). In practice, using outcomes post- intervention, this would be estimated from: where Y1i represents the average of outcome Y within the treatment municipality _i_ and Yoj the average of outcome Y within the control municipality _j_ which was paired to _i_ . W(i, j) are weights which depend on the metric used for the distance between treatment and matched to treatment municipalities.<sup>9</sup> One of the maintained assumptions for the validity of this ITE estimate of the impact of ATI based on the matching estimator is the conditional independence assumption (CIA) assumption which assumes the absence of the role of any unobserved variables such as political affiliation of mayors etc. into the selection into the treatment group (i.e. selection based on observables). In fact the choice of the seven treatment municipalities was decided by GGM and CJM in charge of the program implementation and was based on the willingness of the municipality government to engage on the supply-side of the project and the existence of a functioning participatory budgeting process.

> 9 Only one control municipality satisfying the common support region was chosen to pair with each treated municipality. Weights are proportional to the number of times a control municipality is used to form a match in the treatment group (see Abadie and Imbens, 2002).

13
The availability of the baseline survey allows us to estimate the impact of ATI based on the double-difference approach whereby the matching estimate of the ITE ex-post is netted out of any preexisting differences between the treatment and control municipalities. Since observed characteristics and outcomes were balanced across the treatment groups at baseline, inclusion of the baseline does not change the results, but leads to more precise estimates.

Given that we have repeated cross-sectional data, as opposed to longitudinal data, the estimator following Smith and Todd (2005) can be implemented as: where t and t’ indicate periods after and before the program start date, respectively. This procedure differs from the one above in that matching is performed now twice, before and after the program starts.<sup>10</sup> The second term in the ITE-DDM estimator differences out from the ITT-M estimator any pre-program differences in the outcomes between treatment and control municipalities, hence producing more robust estimates of program effects. We consider time invariant covariates, the ones we use to construct our control group. Given this and that the control municipality remained control after the program started, it is plausible to assume W(i, j, t)= W(i, j, t′), i.e. the matching procedure is performed only once, before the intervention.

The standard error for a DDM estimator does not have an analytical form. Therefore, we construct standard errors using bootstrapping.

# _d. Potential threats to the evaluation design_

> 10 The difference-in-differences matching estimator is defined in Heckman, Ichimura and Todd (1997) and Heckman, Ichimura, Smith and Todd (1998). Blundell and Costa-Dias (2008) also present a variant of the estimator by Smith and Todd (2005) applied to a similar context, with repeated cross section data.

14
The evaluation design above does not distinguish explicitly the potential effect of the Access to Information pilot from the effect of participatory budgeting. As pointed out above, one of the selection criteria for the municipalities in the pilot ATI, was the existence of a functioning participatory budgeting process. However, the implementation of participatory budgeting meetings was not uniform across the seven treatment municipalities. For example, the Monte Plata municipality had functioning participatory budgeting, whereas San José de los Llanos and Hato Mayor municipalities did not.<sup>11</sup> It is important to point out, however, that the participatory budgeting process is also present in the control municipalities. According to the baseline survey about 25% of the households in both treatment and control municipalities knew what a participatory budget process was, and most of them heard about it through friends and family (43% and 38% in control and treatment).<sup>12</sup> Approximately 53% and 60% of respondents in the control and treatment municipalities, respectively, who reported knowing what a participatory budget process was, reported that participatory budget meetings were being undertaken in their municipality (this difference is statistically significant at the 10% level but not at the 5% level), and approximately half of them had participated in at least one meeting, with no significant differences between the two groups. Among those people that had participated in the meetings, 76% in the control municipalities and 60% in the treatment municipalities had expressed their preferences during these meetings (statistically significantly different at the 5% level).

To the extent that the participatory budgeting processes were already in place in both treatment and control municipalities and their effects on people's perceptions have already taken place prior to the implementation of the ATI, then one could reasonably argue that the application of the DiD estimator is likely to yield an estimate of the impact of the ATI program that is "net" of the effects of participatory budgeting on key outcome indicators. However, this statement needs to be qualified further, since participatory budgeting was "work in progress" with possibly measurable impacts on people's perceptions and attitudes during the life of the ATI project.

> 11  See Lima (2011). 12 See Table 1 Appendix B of Lima (2011).

15
# **4. Results**

In this section, we review the results on awareness and use of information, participation in collection action activities, local governance, trust in public institutions and CSOs, and satisfaction with public services. The reported estimator is the intent-to-treat effect obtained using difference-in-differences with matching (DDM) as we explained in the previous section.<sup>13</sup>

# _Awareness of and Access to Information_

Table 1 shows the main results of the ATI project on awareness and use of the FOI law. The ATI pilot’s impact is positive, since it appears to raise knowledge about rights under the FOI law by 3.6 percentage points (on average). However, the impact is not statistically significant.

While the average impact is not statistically significant, the project could have had positive impacts on awareness of FOI law among certain sub-groups, which is not possible to determine due to the limited sample sizes. Since initial awareness of transparency legislation associates positively with covariates such as the level of education or wealth, the ATI project could have been more effective among more educated and wealthier groups. In addition, it could have been more effective among individuals who participate in community organizations since these have been actively disseminating information in many communities. Findings from the qualitative evaluation highlight that many of the community leaders and officials interviewed during the study credited their awareness to workshops and seminars that _Participacion Ciudadana_ had held throughout the country on transparency legislation.

# **Table 1 here**

Although provision of information by public institutions is a mandate, by 2011, 60 percent of public institutions still lacked information offices (World Bank 2011). While

> 13 In Appendix C, we also estimation results  using the simple ex-post difference with matching to compare with the DDM estimates. Also, in Appendix D, we report results of a bias correction method, which uses matching with regression.

16
there was potential for the ATI intervention to improve government capacity and thus knowledge on existence of OAIs, our evaluation findings show no relevant impact of the intervention on knowledge about existence of an OAI. It appears that this knowledge has increased similarly in both treatment and control groups which would be consistent with the presence of spillover effects across treated municipalities. It also appears that knowledge is greater in the chosen ATI municipalities both in the baseline and endline surveys, thus implying no differential effect from the intervention.

In terms of general civic knowledge, when households were asked to identify whether a certain public service is provided by the central or local government, the proportion of correct answers is higher in treatment than in control municipalities for key services. Specifically, the proportion of correct response on education – that it is a central government responsibility – was 4.2 percentage points higher in treatment areas than in control areas. Surprisingly, despite similarity between the ATI program and other transparency projects such as the Participatory Budgeting, knowledge on the latter wasn’t influenced by the ATI program. Neither of these findings was found to be statistically significant.

Regarding the use of information, the surveys show a large increase in requests for information to public offices or officials in the ATI municipalities and a small decrease in the control municipalities. The differential effect is positive, 1.5 percentage point, but not statistically significant. That this parameter is small also reflects findings raised by the qualitative baseline study that identified several barriers to accessing information, especially in the poorer communities in the DR. First, participants in the focus groups blamed pervasive illiteracy for making people unaware of their rights as citizens. Because people do not understand their rights, they do not make demands – as a result, professionals or lawyers are the ones most able to pursue these rights on their behalf. Second, although people wanted to know more about what the public authorities were doing, they feared that demanding information about financial matters could appear confrontational. Participants of the study attributed fear of reprisal to the legacy of dictatorship, under which they lived for 32 years. Thus they seemed to believe that the 17
government would only respond to well-educated, well-spoken or well-connected people (for example, CBO members), but ignore those of lower status.

Nonetheless, our quantitative evaluation shows that CBO members were just a bit more likely to access information than non-CBO members across ATI and non-ATI municipalities, and that this difference is not significant. In principle, this shows that barriers to access information seem to extend to everyone including organized individuals.

Another dimension of access to information that was studied was whether the individual had complained to the local government in the year prior to the survey date. The project led to an increase of 4.5 percentage points in the proportion of households that submitted complaints to the government. Unlike placing requests under the FOI law, submitting complaints is much more general and covers a much broader range of issues than the FOI law does.

In this regard, the qualitative paper for this evaluation shows that all interviewed individuals (in selected treatment and control municipalities) acknowledged the right of citizens to access information from public institutions. However, this notion was present among those who knew as well as those who did not know about the existence of the law. In addition, the rights to access information are stated in a very general and vague way. There is no clear understanding of the ways this information should be accessed or the type of information people would like to obtain. In many cases it is even confused with social demands like holding public officials accountable to their campaign promises, or getting the council to pay for medicine and funeral arrangements for members of the community (Vega, 2011)

# _Participation in collective action activities_

Membership in community-based organizations (CBO) is considered high in the DR. Nearly fifty per cent of the households have someone who belongs to a religious, farmer, neighbor or a non-governmental organization. Table 2 shows that pre-program difference in CBO membership between treatment and control regions is not significant, and that membership has increased for both groups. Surprisingly, this is higher in the non-ATI than in the ATI municipalities. While there is no statistical significance, this is rather puzzling, 18
since the ATI project worked through CBOs to raise awareness about the law. One would expect that this would have created stronger incentives for group membership in treated more than in control municipalities. One possible reason why it did not could be a substitution effect. Where individual CBO members become more empowered to request information, there is likely to be lower demand for collective action.

However, the fact that group membership has grown over the two rounds of the survey in both treatment and control municipalities is interesting in itself and probably a reflection of the Municipal Law 176-07 that complements the FOI law 200-04. A qualitative baseline assessment reports that the Municipal Law 176-07 promoted the development of CSOs, through greater access to public information, use of public media and financial assistance, to propel their participation in the municipal administration.<sup>14</sup> Engagement in other transparency projects such as the Participatory Budgeting has similarly increased in both ATI and non-ATI municipalities, which is to say, independently of the ATI project intervention.

# **Table 2 here**

# _Local governance_

As Table 3 reports, the impacts of the ATI on local governance appear to be the strongest so far. They indicate a higher rate of consultation of citizens in government decision making across samples and periods. The general decline from the baseline to the follow up survey is somewhat puzzling. It may have occurred due to the local elections that took place just before the collection of the baseline survey, thus reflecting candidates’ campaign or commitment to greater accountability than previous administrations.

Nonetheless, looking at the differential effect of the ATI, we observe a positive and large effect, i.e. the reported consultation rate of citizens by local governments in the treatment municipalities is higher by 7.8 percentage points compared to consultation rate

> 14 The financial incentives to the creation and development of non-profit organizations consist of a competitive grant in which the organizations with the project which could benefit mostly the community would win, as established by art. 227 of the Law No. 176-07.

19
in the control municipalities. Proportion of citizens who believe that local governments are spending in key areas for their community is also higher (by 8.1 percentage points) in the treatment than in the control municipalities.

# **Table 3 here**

The importance and statistical significance of these results are intricately linked to how the program operates at the local level. However, we cannot disentangle which factor, supply of or demand for information, is primarily responsible for this result. The pre-program and implementation evaluation seem to point to failures of the ATI intervention in creating institutions and capacity that would improve the quantity and quality of information. On the other hand, the increasing strength and training of local CBOs are believed to be the strongest component of the program, pushing governments to address complaints and to consult citizens for the execution of public investments in key areas for the community.

Qualitative findings from interviews with mayors also reveal their perception about transparency in the ATI municipalities of Bayaguana and Hato Mayor del Rey. According to these, a transparent management is perceived to increase citizens’ trust in mayors, which would favor the re-election of mayors who promote such trust. Hence, trust of households in the government seems to be an important channel through which transparency laws can translate into greater transparency of public institutions. In this context, the impacts of ATI intervention on trust in public private institutions, discussed below, acquire special meaning.

# _Trust_

Table 4 reports the effects of the ATI program on trust in public and non-public institutions. The measure in use is the proportion of households who responded as having complete trust or some trust in these institutions.

Consistent with the positive results we found for most local governance outcomes, we also observe a positive effect of the ATI program on the level of trust of households in the local (municipal) government. One of the components of the ATI intervention involved working with municipalities to increase capacity to respond to demand for information.

20
That reported trust in local governments increased by 9.5 percentage points in the municipalities with the ATI program – while trust in other institutions of the public sector did not show a significant increase – is consistent with a direct effect of the program on the supply (and maybe even quality) of public information at the local level, at least in the way individuals perceive it.

# _Trust in individuals and CSOs_

Though positive, the effects of the ATI program on trust in CSOs (such as church, political parties, NGOs and CBOs) are not statistically significant. However, it is worth noting that the trust in CBOs is large and higher by 8.6 percentage points due to the program (though statistically insignificant). The second component of the ATI program consisted of working intensively with the CBOs to increase awareness in the communities. The increase in trust in CBOs, present only in the treatment group, is indicative of the effectiveness of this channel.

Findings from the qualitative study also indicate that one reason that people were more likely to trust civil society organizations is their history of resistance to the authoritarian regime. Social movements or activism were historically supported by peasant movements, Christian communities, cultural and sports clubs, and NGOs.

# **Table 4 here**

# _Satisfaction with public services_

As Table 5 shows, there is little or no evidence of project impacts on satisfaction with public services, with the exception of maintenance of public parks. Services for which there is no significant impact on satisfaction include those provided by the central government (such as health, education, electricity, sewer and water) or the municipality (such as garbage collection, roads, and transport). For most of these services, there seems to be a common time trend of increased satisfaction in both treatment and control municipalities. The resulting net treatment impact is null in the case of health, education, transportation and roads. The only significant and positive impact observed is on satisfaction with maintenance of public parks (greater by 9.3 percentage points in the ATI municipalities). That any positive impact on satisfaction in ATI municipalities was limited to a service 21
under the purview of the local government makes sense – the limited size of the pilot meant that any increase in demand for accountability among citizens would have affected the incentives of only the municipal government.

# **Table 5 here**

What remains less clear, however, is why the positive impact on satisfaction occurred for one municipal government service (maintenance of public parks) only and not the others. The qualitative study appears to provide some clues. First, most people care about municipal budgets and information on payrolls and expenditures. Second, spending on maintenance or construction of parks (and to a lesser extent) is among the most visible to citizens. During focus group meetings and interviews, people seemed concerned with taxes and budgets, and particularly with what is being done with their taxes, focusing mostly on visible signs of improvements (or lack thereof) in their community. As seen in Chapter 1, a respondent mentioned a children’s park that was to be paid for from the local budget – “from our pockets” - but has remained unfinished for years. Others talked about using the right to information to get the council to give money to their organizations or to find ways to have the local government build things for the community, such as parks and bridges. For most people, having more information was a way to oversee the expenditure allocation of local, and sometimes central governments (Vega, 2011). Knowing about these concerns, and responding to increased demand from citizens for accountability in the ATI treatment areas, local governments in ATI municipalities might have focused on spending to improve the quality of visible services like public parks.

# _Heterogeneous impacts_

For younger households (40 years or less), effects are twice as much effects for the average, for all outcomes except access to rights. The p-values reduce in all cases, suggesting that effects are much higher so detectable with the sample we have. In the limit, this also suggests a positive and higher effect of the ATI pilot on awareness of younger households, if we had a larger sample.

# **Table 6 here**

22
# **5. Conclusions and Considerations for Project Expansion**

This paper summarizes the results of the impact evaluation of the ATI pilot project on empowerment of citizens in poor municipalities in the Dominican Republic. Among the dimensions of empowerment investigated are civic knowledge, awareness and use of the right to information; perceptions of and trust in public services and institutions; civic participation; and measures of local governance.

The quantitative data collected in two rounds; baseline at the end of 2010 and follow up in mid-2012, is based on the premise that the project could have some measurable impacts one and a half years after its initiation. This relatively short time horizon may be sufficient for some indicators to change but certainly not all. Indeed, as it is apparent in the paper, many indicators do not change relative to the comparison group, or even if they do the changes (or the impacts) are too small or statistically insignificant. Nevertheless it is important to point out that some positive (and statistically significant) impacts are observed on local government responsiveness, on prioritization and decisions about municipal budget, and on trust in and satisfaction with some local government services.

Bearing these caveats in mind, the results of this paper shed some light on the type of outcomes that can be influenced by interventions enhancing the supply of and demand for public information in poor areas, and potential areas of improvement for such interventions. Specifically, we observe no impact on awareness and access of information under the specific ATI rules. However, we do observe that individuals manage to address more general complaints to governments as a result of the ATI program regardless of whether these are classified under the FOI law or not. We also do not find any impact on participation in collective action activities. But the ATI pilot appears to have been more effective in other dimensions that are detailed below: The ATI intervention increased the consultation of citizens by government about where to invest at the local level, as perceived by citizens. It also resulted in an increase in expenditures that match people’s preferences in the communities. These results seem to be linked to how the program operates. There seems to be a greater role played by the demand side of the intervention as pointed out during the pre-program and interim stages 23
of the evaluation. The increasing strength and training of local CBOs are believed to be the strongest component of the program, pushing governments to address complaints and to consult citizens for the execution of public investments in key areas for the community.

The ATI intervention also improved trust in local governments. This is in line with the previous result indicating significant improvement in local government transparency, as perceived by citizens. Greater pressure for improved local governance and more response from mayors probably contributed toward the program’s success in improving trust in municipal governments.

The ATI intervention increased satisfaction with maintenance of public parks, but had no significant impact on satisfaction with other services provided by the municipality or the central government. Citizens seem to associate freedom of access to public information with the right to know about municipal budgets – how the taxes they are contributing to the municipal governments are being spent by the government. It seems intuitive that people tend to be more focused on investments that are more visible to them, such as public parks, which could explain why municipal government might have focused on improving the quality of this type of spending.

Even though the ATI intervention demonstrated some success in providing incentives for better governance at the municipality level, there are a few potential areas of improvement in the design of these types of interventions that deserve serious consideration if they are to lead to greater empowerment and improved governance.

Most of the interventions supported by the ATI pilot – raising awareness and educating citizens about how to demand information – occurred through community organizations. As mentioned earlier, CBOs are also generally seen as the principal conduits for request for information, and the ATI pilot leverages (and reinforces) this perception. In the past ten years, CBOs have been supported as part of initiatives to promote democratization and government transparency. However, this has occurred primarily among urban and relatively better educated segments of the population. In poor municipalities such as the ones included in this pilot, CBOs are likely to lack the resources and the training required to demand information effectively. While the ATI pilot attempted to correct for the lack of capacity through initiatives to train CBO members, these may have 24
been inadequate and short-lived. Future programs of a similar type can become more effective by focusing even more on – and improving the content and length of – such training.

Intervening through community organizations is probably the most effective way to increase demand for information, as members and leaders of such organizations are relatively better off, more educated, more engaged in entrepreneurial activities, and thus more knowledgeable and successful in requesting information about government actions. On the other hand, channeling information requests through community leaders may run the risk of capture by local elites, which could in turn imply that the concerns of the poorest or most disadvantaged citizens are inadequately represented. While the risk of elite capture is something this study has not analyzed, it is a risk that merits consideration in the design of future ATI-type interventions. Whether elite capture of CBOs has occurred in these areas will also be a subject of upcoming research using the evaluation survey data.

Since CBOs are likely to have limitations in terms of outreach among citizens, should future programs use media sources, such as television, more intensively? Recent literature shows the potential for ICT to enhance government outcomes (Zhen-Wei Qiang et al, 2006). The process evaluation report reveals that information campaigns through TV and radio were carried out only for one quarter of the calendar year, towards the end of the program implementation period. Despite this, the quantitative baseline survey reveals that when individuals are asked to respond spontaneously about how they heard about the FOI law, 60 percent answered TV, 27 percent radio, 18 percent family and friends, 11 percent newspapers, 5 percent community meetings, 5 percent CBOs, and less than 1 percent from other activities such as workshops. These numbers were similar in the follow up survey, but with the difference that the percentage of those who heard about the law from community meetings or CBOs increased to 9 and 7 percent respectively. Given this, future projects of this type should consider using media channels more intensively to disseminate information, to complement efforts through CBOs.

The results from the surveys also suggest that efforts to raise awareness about right to information laws are likely to be more successful if the type of information that is disseminated is customized or tailored to fit more closely with the immediate concerns and 25
needs of citizens. A program to attract the interest of people would be one that is _localized_ , and has clear relevance to issues close to the immediate concerns of citizens. In poor municipalities where the ATI pilot was implemented, people seem to be most interested to demand information on services that create jobs or generate income, or visibly affect the quality of their day-to-day lives (such as poor maintenance of public parks) – and related to that, what their municipality is doing with the taxes they pay. Given this, the focus of the ATI pilot on information about services provided by (and expenditures by) the municipal government seems to be appropriate. That said, there may be scope to further strengthen the “localization” component of future programs with similar objectives.

The process evaluation suggests that the governance of social audit committees (SAC) was a major challenge. Most SAC members were civil servants or CBO members with limited skills to monitor infrastructure projects or to analyze municipal budget, resulting in low independence and capacity within these committees. We cannot disentangle the effects attributable to supply or demand side interventions. However, based on what we know about the implementation of the project, it appears that the impacts we find were mainly a result of demand side interventions, namely ones that seek to increase demand for information and accountability (of municipal government) among citizens.

The ATI pilot seems to have an important implication for future programs: demandside interventions by themselves, even with uneven implementation, can have some impact on citizen empowerment and local governance. Beyond this, what the results imply for future projects – on the efficacy of supply versus demand-side interventions – is unclear. Given that the impacts appear to have resulted mainly from demand-side interventions, one could argue that it would be more cost-effective for future programs to focus even more on the demand side at the expense of the supply-side. On the other hand, one could also argue that the impacts of the program could have been higher with a more effective implementation of the supply side of the intervention, by improving the quantity and quality of information provided to citizens.

Finally, it is interesting to note that the impacts seem to have occurred in the _absence_ of any significant impact on citizen awareness or knowledge of the ATI law. This seems to suggest that improving knowledge about the law among the average citizen is not 26
a _necessary_ condition for citizen empowerment to occur through demand-side interventions. In the case of the ATI intervention, empowerment seems to have occurred mainly through increased engagement of CBOs in the process, even though these organizations are low on capacity and have limited outreach among citizens. Whether this is the most cost-effective way to use ATI law to improve citizen empowerment is however a question that this project cannot shed light on. To address this question, it would be useful to design interventions that combine different potential ways to raise awareness on ATI law, and evaluate these against each other to find the most cost-effective ways of increasing awareness and citizen empowerment. This could include, for example, outreach through CBOs, media channels and combinations thereof – and each method could be evaluated against the others for impact and cost-effectiveness. We hope that the findings of this study will be a catalyst for such experimentation and research, since the use of ATI law is a powerful potential tool for citizen empowerment, which can raise the quality of governance and services that affects the lives of poor people everywhere.

27