metadata: authors: 0: author: Ling,Cristina Marosan
1: author: Roberts,Dawn
count: World
docna: 0: docna: Evidence of development impact from
            institutional change : a review of the evidence on open budgeting
docty: Policy Research Working Paper
subtopic: Macro-Fiscal Policy,Economic Adjustment and Lending,National Governance,Government Policies,Public Sector Economics,Hydrology,Social Accountability,Youth and Governance,Public Financial Management,Public Finance Decentralization and
            Poverty Reduction,Inequality
teratopic: Governance,Macroeconomics and Economic Growth,Poverty Reduction,Public Sector Development
abstracts: cdata!: Despite the growing body of literature
            examining the effectiveness of transparency and
            accountability initiatives, there remains limited
            substantiation for whether and how open budgeting
            contributes to reductions in poverty and improvements in the
            lives of the poor. This paper reviews available evidence and
            conclude that institutional changes can contribute to
            higher-level outcomes in certain contexts. The approach
            first draws from existing studies of transparency and
            accountability initiatives and then follows their references
            to broaden the evidence base. The findings highlight the
            importance of measuring budget transparency, accountability,
            and participation and tracing their outcomes along an
            incremental, nonlinear results chain. Logical links or
            ongoing loops in this sequence include the interplay or
            interdependency among these three dimensions; the subsequent
            achievement of key, often mutually reinforcing, intermediate
            development outcomes; and ultimately, improved program or
            service delivery as the key lever for influencing
            development impact. Rather than establishing standard
            indicators, the process begins to identify which aspects of
            the institutional change are valid for measurement and what
            contextual factors to consider. Overall, this review serves
            as a starting point and underscores the need for further
            investigation to establish effective measurement practices
            of institutional change and build an evidence base for
            understanding the relative robustness of institutional
            change paths and the context in which they are likely to matter.
repnme: repnme: Evidence of development impact from
            institutional change : a review of the evidence on open budgeting
display_title: Evidence of development impact from
            institutional change : a review of the evidence on open budgeting
listing_relative_url: /research/2014/07/19874670/evidence-development-impact-institutional-change-review-evidence-open-budgeting
topicv3: Fiscal Transparency m1326224 722,Budget Transparency m1326239 734,Water m1326522 137,Poverty m1326687 695,Budget Preparation and Management
            Techniques m1326766 2060
publishtoextweb_dt: 2014-07-01T00:00:00Z
historic_topic: Governance,Macroeconomics and Economic Growth,Poverty Reduction,Public Sector Development
no_of_pages: 32
author_keys: 0: author_key: 657368
1: author_key: 763786
model_extractions: WPS6968 Policy Research Working Paper 6968 Evidence of Development Impact from Institutional Change A Review of the Evidence on Open Budgeting _Cristina Ling Dawn Roberts_ World Bank Institute Capacity Development and Results July 2014
Policy Research Working Paper 6968

# **Abstract**

Despite the growing body of literature examining the effectiveness of transparency and accountability initiatives, there remains limited substantiation for whether and how open budgeting contributes to reductions in poverty and improvements in the lives of the poor. This paper reviews available evidence and conclude that institutional changes can contribute to higher-level outcomes in certain contexts. The approach first draws from existing studies of transparency and accountability initiatives and then follows their references to broaden the evidence base. The findings highlight the importance of measuring budget transparency, accountability, and participation and tracing their outcomes along an incremental, nonlinear results chain. Logical links or ongoing loops in this sequence include the interplay or interdependency among these three dimensions; the subsequent achievement of key, often mutually reinforcing, intermediate development outcomes; and ultimately, improved program or service delivery as the key lever for influencing development impact. Rather than establishing standard indicators, the process begins to identify which aspects of the institutional change are valid for measurement and what contextual factors to consider. Overall, this review serves as a starting point and underscores the need for further investigation to establish effective measurement practices of institutional change and build an evidence base for understanding the relative robustness of institutional change paths and the context in which they are likely to matter.

This paper is a product of the Capacity Development and Results, World Bank Institute . It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at cling@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Evidence of Development Impact from Institutional Change: A Review of the Evidence on Open Budgeting**

Cristina Ling Dawn Roberts

## **JEL Classification** : H50, H61, H72, H75

**Keywords** : accountability, citizen engagement, civil society, fiscal transparency, human development outcomes, participatory budgeting, public expenditure

**Acknowledgements** :  This paper has benefitted greatly from the guidance of Samuel Otoo, and from feedback provided by Nicola Smithers, Keith McLean, Tiago Peixoto, Neil Cole, Sanjay Agarwal, and participants at the October 2013 PRMPS-WBI meeting on fiscal transparency.
# **1. Introduction**

This report explores the critical question of “What is the poverty impact of the Bank’s development efforts?” faced by all Bank programs in this era focused on achieving key poverty targets. Previously, in order to demonstrate results, it was enough to measure project outcomes and show how they met project development objectives. President Kim has raised the bar not only to demonstrate program outcomes, but to show how they contribute to overall poverty reduction. While this is challenging enough for standard infrastructure projects for which tangible outcomes exist and are easily measured, this task is even more difficult for institution building and capacity development initiatives for which the outcomes are intangible. Hence, the question we seek to answer in particular for the World Bank Institute (WBI) is "What is the development impact of the institutional changes targeted by WBI?" Answering these questions is important both for accountability purposes and for better understanding our practices and the conditions under which they are most effective. First, we have to be able to answer to the Board, our donor partners and clients that what WBI is doing or seeking to achieve is important for improving the lives of the poor. In other words, it is important to determine whether or not we are doing the right things i.e., are the institutional changes we are targeting the right ones that actually influence development impact? Second, under what conditions are these institutional changes most effective in achieving human development outcomes?

WBI has focused its efforts to innovate and experiment with various instruments and techniques to empower, influence and change institutions yet the impact of these institutions to actually help the marginalized and the poorest of the poor is difficult to prove with hard evidence. Further, it has yet to establish when and how these instruments are most effective. Thus, a review of the existing evidence is in order to determine whether and when institutions contribute to improved development outcomes and have an impact on poverty reduction.

A cursory review of the literature and systematic reviews exploring poverty impact revealed not only the breadth of this topic, but also that these investigations’ conclusions tend to be at a general level, reporting whether interventions were successful in meeting stated objectives but not in terms of detailing what worked and what did not work during implementation (Santibanez 2013, Fennell, van Gevelt, and Safdar 2013).

The methods used in standard program evaluations and rigorous impact evaluations with experimental designs and randomized control trials (RCTs) control for a number of key factors that are actually critical to understanding how future interventions could be better implemented, replicated or scaled up. Thus, the necessary information for discovering the science of delivery is missing. This paper, therefore, seeks to unpack the question of when and how there is institutional impact on poverty reduction by focusing on a WBI practice area that is also highly relevant for the Bank – i.e., open budgeting in the field of public financial management.

2
# **_1.1 Objectives_**

In this paper, we explore the key question of whether there is evidence linking open budgeting to high level development outcomes that reflect improvements in the lives of the poor. For the purposes of this investigation, we follow the conventional way poverty reduction is typically operationalized in the area of open budgeting and public financial management that is, as improved access to better quality public services for the poor. We explicitly do not focus on macroeconomic benefits nor include any empirical literature about the relationship between fiscal transparency and growth in this review.

We identify and piece together evidence demonstrating the development impact of Open Budgeting by conducting a review of the available literature. We examine evidence supporting and refuting various small incremental steps in a results chain to establish linkages between open budgeting and improved human development outcomes and improvements in public services. This approach first draws from existing studies of transparency and accountability initiatives and then follows their references to broaden the evidence base. The review is not exhaustive but is instead intended to highlight findings for further investigation and discussion.

The second key question we explore is whether or not we can identify “leading indicators of institutional change impact.” These are concrete observable milestones marking points of progress in open budgeting interventions that eventually contribute to development outcomes for the poor.

In this paper, we start with the overall goal of poverty reduction and unpack it as improvements for the poor in the delivery of basic public services such as health, water, sanitation, roads and transport. We then look to see whether studies identified indicators of institutional changes that affect the success of various open budgeting interventions and if so how they were defined. Unpacking how changes in institutional conditions influenced development outcomes through observable leading indicators would provide insights on which institutions to target and how.

# **_1.2 Institutional changes: Transparency, accountability and participation_**

We explore this by considering three key institutional changes and how they affect poverty: fiscal transparency, accountability, and participation. Fiscal transparency refers to the public disclosure of fiscal information related to the budget process (OECD 2002). This includes publication of information in a timely and systematic manner. Accountability refers to “answerability,” in terms of the need for government leaders to provide information and justify their actions and “enforceability,” reflecting potential negative consequences when the answerability falls short (McGee and Gaventa 2011).

Accountability can include procedures for public hearings and investigations, public audits, and transparency institutions and independent judiciaries. Participation is about citizens participating in the budget decision making monitoring and audit process and how and when public funds are spent. Participatory budgeting (PB) can happen through direct citizen involvement in voting for 3
budget allocations, for instance, or through delegation to a council or elected PB leader who represents the citizenry in the budget decision-making process.

The three are often interrelated and mutually reinforcing. In other words, some argue that transparency is necessary for accountability because without information, citizens could not hold their governments accountable (de Renzio, Guardian newspaper). In addition, participatory budgeting increases fiscal transparency that thereby improves accountability of local officials to deliver better public services (Baiocchi, Heller, Chaudhuri and Silva 2006).

However, it has also been argued that holding governments accountable is what makes governments comply with transparency, and that participation encourages accountability (Wampler 2004, Ackerman 2003). Given the complex nature of the relationships between these institutional changes, we examine each of them in the following as both unique concepts and other times as part of endogenous interactions.

Overall, the normative expectation is that these institutional changes will lead to better development results. That is, budgets will better meet the needs of the citizenry and the poor will be better represented and served by including the citizenry. This paper will present evidence supporting and refuting this as well as some more textured findings about the conditions under which these institutional changes are likely to be more or less effective in varying contexts. In addition, within each of the following sections on transparency, accountability and participation, we present the indicators used to measure development outcomes and institutional level changes when available in the studies.

# 2. **Fiscal Transparency**

**The process of identifying any causal chains to which transparency contributes should start with a definition of what such transparency includes** . The OECD’s Best Practices for Budget Transparency (2002) defines this term quite simply as “the full disclosure of all relevant fiscal information in a timely and systematic manner.” Robinson (2006) highlighted two dimensions of transparency that are particularly relevant to the budget process. The first relates to the mandate for the executive to “divulge the sources of data and information used to frame decisions” and second focuses on the transparency of the budget process in the public domain including the basis on which priorities are formulated and the distribution of roles and responsibilities (p.12).

**Evidence is emerging that transparency can contribute to key intermediate development outcomes under the right conditions.** Identifying these outcomes and their accompanying factors is critical for understanding whether and how budget transparency matters for any efforts to reduce poverty and improve human development indicators over the longer term. As McGee and Gaventa noted in their broad literature review of transparency and accountability initiatives, “many initiatives are not underpinned by a clear articulation of exactly what outcome or impact is sought, or of how the actions and inputs contemplated are expected to generate that outcome or impact” (2011 p.15). The exploration of evidence and assumptions for every stage of the results chain, including the more immediate incremental changes, is needed to define the causal sequence and identify high priority contextual variables.

4
**_2.1 The effects of transparency related to good governance and institutional quality_**

**One potential benefit of transparency is the improved accountability of government** . The general public, civil society, media, and the legislature are better positioned to hold the executive to account when they have sufficient information on the allocation and use of public resources. Robinson (2006) reviewed the findings from six detailed case studies of independent budget work and noted “transparency can help to reduce the scope for corruption through the misallocation of expenditures or the diversion of public resources for private ends.” For example, in Mexico, Fundar’s efforts to increase the transparency of the budget policies first led to new federal legislation to ensure public access to information and then to the establishment of a new review and enforcement agency to “compel government departments to comply with the provisions of the legislation” (p.26).

**However, transparency alone does not guarantee greater accountability.** Joshi (2011, p.7) explored existing evidence of causal relationships between transparency and accountability and their impact and effectiveness in the service delivery arena and found that “many initiatives are focused at increasing transparency and amplifying voice, without examining their link with accountability and ultimately responsiveness.” Her review of several studies led to the conclusion that a key contextual factor is the “willingness from the public sector to support attempts to improve accountability” (p.19). The tenuousness of the connection between information accessibility and accountability is echoed in numerous other studies, underscoring the need for further investigation and analysis (McGee and Gaventa 2011; Bellver and Kaufmann 2005; Carlitz 2012).

**Overall, the evidence indicates that increased transparency can lead to better quality governance and reduced corruption under the right conditions.** Islam (2003) explored the links between information flows and institutional quality and concluded that countries with better information flows govern better. Her analysis focused in particular on the existence of freedom of information laws and a constructed “transparency” index that measured the frequency with which economic data are published in countries around the world. Other studies further confirm an association between greater budget transparency, reduced corruption, and improved governance (Kaufmann and Bellver 2005; Hameed 2005).

# **_Indicators_**

The measurements of transparency that were used to trace the effects related to **_good governance_** and **_institutional quality_** in the studies discussed above were generally based on a review of legislative records and/or the construction of a transparency index using available economic data:

- Robinson (2006) identified the _implementation of federal legislation_ to ensure public access to information (Y/N) and the creation of a _review and enforcement agency_ to compel governments to comply with the provisions of legislation to ensure public access to information (Y/N).

- Islam (2003) constructed a _transparency index_ using 11 representative variables from four sectors (real, fiscal, financial and external) for a total of 169 countries.

- Islam (2003) also used the passage of a _freedom of information act_ to confirm a country’s progress towards becoming more transparent.

5
- **_2.2 The effects of transparency for enhanced budgets and more efficient spending_**

**Increasing access to budget information can also enhance the resources available and improve their allocation for more efficient spending** . The experience of the Republic of Korea has been that enhancing budget transparency has “facilitated the use of information by various stakeholders” so that civil society organizations can influence budgetary priorities and National Assembly members could upgrade policy debates among other benefits (You and Lee 2011, pp.30-33). Robinson emphasized the importance of transparency from the perspective of equity and social justice, where secrecy “can result in priorities that are biased towards elite interests and not tailored to the needs and priorities of the poor.” The case studies he reviewed signaled a connection between budget transparency and increased allocations for social welfare expenditure priorities, especially for reproductive health in Mexico, child support grants in South Africa, and tribal development expenditures in Gujarat (pp.12-20).

**Improving the tracking of aid allocation and expenditures can contribute to better resource management in aid delivery** . By analyzing data from 70 countries collected via a country-level survey on mutual accountability and transparency, Martin (2010) concluded that strengthening transparency resulting from the better tracking of aid information contributed to the better prioritization of spending needs for a pro-poor orientation and overall improvements in budgeting and spending related to aid delivery. Key factors reflecting improvements in transparency included the breadth of information available (i.e., to include projected future disbursements, compliance with national and international aid effectiveness targets, etc.) and the timeliness and consistency of information available from a range of sources.

# **Indicators**

Reviews and studies to date have focused largely on qualitative data from case studies where the results logic is understood within a unique context. However, the analyses do demonstrate the types of indicators that can be used to measure needed _changes in transparency_ that _contribute to enhanced budgets and more efficient spending,_ such as in the following examples.

- Use of a national mutual accountability and aid transparency survey (Martin 2006) to collect multi-stakeholder input from each country to assess the presence and quality of mechanisms for ensuring transparency, such as:

- The existence of a national aid policy document defining the government’s priorities on aid with formal provider targets

- The use of forums in which providers and recipients discuss development results and aid issues

- The inclusion of budget analyses from independent monitoring groups, parliaments, and civil society organizations

- Use of Public Expenditure Tracking Surveys to identify budget leakages and publicize them for enhancing budget allocations to pro-poor priorities (Joshi 2010)

- Level of inadequacy of existing allocations and discrepancies between commitments and actual disbursements based on independent budget analysis (Robinson 2006)

6
# **_2.3 The effects of transparency on citizen participation in the budget process_**

**Fiscal transparency can enhance participation by improving the quality of public debate and the contributions of citizens and CSOs to the policy-making, budgeting, monitoring and implementation process** . Robinson (2006) concluded, based on the review of six case studies, that improving the amount and quality of budget information available could diversify the range of actors engaged in the deliberation of budget priorities. One notable example of the potential for this causal chain has been the experience of the Extractive Industries Transparency Initiative (EITI), which works to ensure that revenues accruing to governments from extractive sectors are well documented and publicly disclosed. Disclosing this budget information allows for greater scrutiny of how revenues are being used, provides a platform for public engagement, and fosters public debate (Eads and Krakenes 2010, Reite T. et al 2011).

# **Transparency is a necessary but not always sufficient condition for fostering citizen**

**engagement** . In their review assessing the impact of transparency and accountability initiatives, McGee and Gaventa (2011) highlighted examples of how access to budget information is a key ingredient for participatory budgeting. Their findings included that freedom of information can contribute to improved public understanding and public participation (citing Hazell and Worthy 2009) and that the Right to Information campaign in India contributed to new legislation and mobilized stakeholders to use information to achieve development objectives (citing Jenkins 2007). However, Zhang (2012) reported on a pilot field experiment in Kenya exploring how an information campaign about politician spending could impact citizen engagement. In this case, local participation in the monitoring of public goods increased only when citizens received instructions about how to take action in addition to the information about the missing funds.

# **_2.4 Links between transparency and development impact_**

**There is some evidence that budget transparency leads to stronger human development**

**outcomes.** Limited studies have been conducted to date that directly connect budget transparency to higher level development outcomes; however, the research by Fukuda-Parr, Guyer, and Lawson-Remer (2011) does demonstrate how indicators measuring transparency can be operationalized to analyze development effects. Recognizing the limited empirical evidence available on the impact of budget transparency, Fukuda-Parr et al, used a variety of techniques to analyze the relationship between budget transparency as measured by the Open Budget Index (OBI) and a range of indicators of development outcomes. Their findings were that the overall OBI is a significant predictor of child and infant health outcomes and access to improved drinking water, even when controlling for the effects of income and region. This analysis confirmed and elaborated on previous studies by Bellver and Kaufmann (2005) and Islam (2006) in which the researchers constructed their own indicators of transparency and identified positive relationships between transparency and development outcomes and governance characteristics (Fukuda-Parr et. al. 2011).

# **_Indicators_**

The OBI is a compilation of rankings of budget transparency made by objective (nongovernment) country experts. Because the sub-indices in some cases correlated more strongly 7
with some development outcomes than the overall OBI did, the researchers explored the impact of some components of budget transparency separately. Specifically, a series of regression analyses explored whether associations between transparency and _human development outcomes_ are significant when the impacts of different income levels and geography are taken into effect. The _regime type_ and _institutional quality indicators_ included:

- Freedom House<sup>1</sup> Civil Rights Index score

- Freedom House Political Rights Index score

- Open Budget Index score

- Legislative strength index, based on subset of OBI questions

- Supreme Audit institution strength index, based on subset of OBI questions

- Citizen participation index, based on subset of OBI questions

- Polity IV Regime Type<sup>2</sup>

# **_2.5 The factors or conditions needed for transparency to have the desired effect_**

**In each case, assessing the quality of transparency or the presence of certain characteristics is required in order to identify associated outcomes.** Numerous studies have suggested that transparency in budgeting does not always matter and does not guarantee progress toward development outcomes. For example, a broad systematic review of econometric and quantitative studies exploring the impact of changes in the transparency of infrastructure procurement and delivery found that increasing transparency did not contribute to the desired outcomes in more than half (53%) of the 862 observations (Annamalai et al 2012). McGee and Gaventa (2010) concluded that it is not yet possible to generalize the effects of factors for articulating a reliable results chain, but their review summarized _two categories of factors that are likely to contribute to the effectiveness of transparency_ in contributing to intermediate and higher-level outcomes:

- _State responsiveness_ (supply) factors— that is, the level of democratization, political will, broader enabling legal frameworks, political incentives, and sanctions.

- _Citizen voice_ (demand) factors—such as the opportunities for citizens and CSOs to use budget information and participate in budgeting processes, and the degree to which transparency and accountability initiatives interact with other collective action strategies.

**Transparency leads to accountability only under certain conditions.** Fox (2007) highlighted an important distinction between “clear” versus “opaque” transparency, where “clear transparency sheds light on institutional behavior” and “permits interested parties (such as policy makers, opinion makers, and programme participants) to pursue strategies of constructive change” (p.667). Transparency is more likely to produce the intended effects when it fulfills both the condition of “publicity”—having relevant disclosed information actually reach the intended audience—and the condition of “political agency”—having mechanisms where citizens can take action in response to the disclosed information (Peixoto 2013).

> 1 Freedom House is an independent watchdog organization dedicated to the expansion of freedom around the world and collects data on various related issues (among other activities).

> 2 The Polity IV Project continues the Polity research tradition of coding the authority characteristics of states in the world system for purposes of comparative quantitative analysis.

8
**The effectiveness of independent budget groups in using information varies according to circumstances.** In his review of case studies, Robinson (2006) concluded that independent budget groups could play a positive role to facilitate development outcomes when certain explanatory factors are in place. Any analysis must also therefore consider the _strength and flexibility of broader alliances in civil society_ , the _quality of relationships established with government and the legislature to share information_ , the _quality of legislative engagement in budget deliberation and review_ and the _flexibility of the budget process._ The independent budget groups themselves must acquire legitimacy for their work by producing timely, accurate and accessible data.

# 3. **Accountability**

**Accountability, often interrelated with transparency, is an important end in itself.** While there are varying definitions of accountability, there is broad agreement that the term ideally refers both to “answerability” in terms of the need to provide information and justify one’s actions and “enforceability” reflecting possible penalties or consequences when the answerability falls short (McGee and Gaventa 2011). Common definitions refer to “horizontal” accountability as that relating to various mechanisms or institutions providing checks and balances among the branches of the state and “vertical” accountability referring to that between citizens and the state (Carlitz 2010; Robinson 2006). There has also been growing focus on “societal accountability” mechanisms of a hybrid form, termed “diagonal accountability,” which blur the separation between the state and society (Ackerman 2004).

## **_3.1 Accountability can enhance budgets and contribute to better service delivery_**

**Holding duty bearers accountable for spending commitments can lead to an enhanced budget for supporting the needed infrastructure development and pro-poor services.** Malajovich and Robinson (2006) examined the case of Developing Initiatives for Social and Human Interaction (DISHA) in Gujarat, India in which DISHA informs elected local councilors of specific allocations under the state budget for infrastructure development and checks whether implementation is under way. If the local official was found to be unaware of the allocation, DISHA places pressure on the local administration to ensure that funds are used for their designated purpose. Similarly, the Uganda Debt Network (UDN) relied on community monitors to report the misuse of budget resources to local authorities so that resources allocations could be reassigned in line with the budget provisions (Robinson 2006, Renzio et al. 2006). This misuse was identified by noting such problems as the lack of conformity with technical specifications in the building of classrooms, the absence of teachers, and the lack of essential drugs in health facilities.

**Community monitoring has been shown to improve the quality of services.** Notable cases have included communities engaging with local health services in Uganda to monitor providers and improve the responsiveness of service delivery and teachers increasing their attendance rates in India in response to improved incentives and accountability mechanisms (Carlitz 2010, Duflo et al 2012, Bjorkman and Svensson 2009). The literature has indicated promising effects of accountability initiatives but also underscored the need for caution in interpreting these findings, particularly given the weaknesses of public satisfaction indicators for assessing service quality and the added value of using accountability mechanisms together with incentives and user engagement strategies (Deichmann and Lall 2007; Duflo et al 2012).

9
# **_Indicators_**

The exploration of accountability and its effects on service delivery have relied on qualitative methods in many cases to articulate results chains that have been achieved in those specific contexts. Examples of indicators and approaches for assessing accountability have included the following:

- **_Accountability_** achieved (and strengthened) was assessed by DISHA through a review of government budget data errors and the number of timely errata statements issues (for accuracy of the budget) and by tracing the process of expenditure tracking wherein DISHA requests information on the progress of specific allocations and contacts Panchayats to find out if the funds were disbursed for the relevant purposes. This laborintensive expenditure tracking allowed DISHA to tally and report on “under-spends” committed by the government over time (Malajovich and Robinson 2006).

- **_Quality of services_** was measured according to citizen assessments. Bjorkman and Svensson (2009) conducted a randomized field experiment on community-based monitoring of public primary health care providers in Uganda. The experiment covered 50 communities in nine districts. NGOs facilitated village and staff meetings in which members of the communities discussed baseline information on the status of health service delivery relative to other providers and the government standard. Community members were also encouraged to develop a plan identifying key problems and steps the providers should take to improve health service provision. The primary objective of the intervention was to initiate a process of community-based monitoring that was then up to the community to sustain and lead. Treatment practices, including immunization of children, waiting time, examination procedures, and absenteeism, improved significantly in the treatment communities.

# **_3.2 The effects of accountability for good governance and institutional quality_**

**Accountability can help to curb corruption** . The UDN and DISHA examples help to demonstrate how community monitoring mechanisms make it more difficult for officials to divert budget resources for undesignated purposes. This is a common finding in the literature in terms of the value of holding stakeholders engaged in the budget process accountable. A range of mechanisms such as community scorecards, social audits, and various complaint mechanisms have been used effectively to ensure better that budgets are allocated and implemented as planned (Deininger and Mpuga 2005, Sing and Vutukuru 2020, Caseley 2003).

**Social accountability can promote good governance.** The interplay between transparency, accountability, and participation surfaces when citizens gain access to information that increases their disillusionment with the quality of governance and “moves them beyond electoral participation” (Gaventa and McGee 2013). Malena et al (2004) argue that social accountability mechanisms (e.g. mass mobilization, use of the courts, etc.) “Allow ordinary citizens to access information, voice their needs, and demand accountability between elections.” This “more informed, organized and systematic manner” of engaging bureaucrats increases the “chances of effective positive change” (p.5).

10
# **The specific role and value of participatory approaches for achieving accountability**

**remains unclear.** Olken (2007) used randomized controlled trial methodology to examine two different approaches to accountability and found that participatory mechanisms such as anonymous feedback forms or accountability meetings had negligible impact whereas the increase in government audits led to an 8 percent decrease in missing expenditures (cited in McGee and Gaventa, 2010). In contrast, other studies argue for the value of citizen monitoring and advocacy but use less rigorous approaches (Deininger and Mpuga 2005, Sing and Vutukuru 2020, Caseley 2003).

# **Indicators**

**_Accountability_** was measured using two approaches based on RCT (Olken 2007). The first focused on participatory monitoring methods, including counting the _frequency of accountability meetings and the submission of anonymous feedback forms_ . The second counted the _frequency of government audits_ in terms of the probability that a particular entity would be audited. The resulting effect on corruption in this case was assessed in terms of the amount of missing expenditures.

# **_3.3 The link between accountability and human development outcomes_**

**Emerging evidence suggests there is a link between accountability and higher-level human development outcomes.** The analysis conducted by Fukuda-Parr et al (2011) on the relationship between the OBI and various development outcomes also included tests for various sub-indices constructed from specific subsets of questions in recognition that “some aspects of budget transparency are more important for enabling positive development outcomes than others” (p.18). For example, sub-indices were created for the strength of the legislative branch, supreme audit institution and citizen participation, and in some cases, these sub-indices were stronger predictors than the overall OBI score. The results showed that accountability, even when controlling for GDP per capita and region, was more strongly associated with health development outcomes than the OBI.

# **_Indicators_**

- **_Accountability_** was measured by the Supreme Audit Institution Strength Index (FukudaPar et al. 2011). This index reflects the straight average of country scores for a subset of questions in the Open Budget Survey.

- **_Participation_** was measured by Fukuda-Par et al. (2011) through the Citizen Participation Index that reflects the straight average of country scores for a subset of questions in the Open Budget Survey about opportunities for citizens to participate in the budget process.

**Studies examining the outcomes of community monitoring for improved service delivery show some effects on improved human development outcomes over time.** For example, the randomized field experiment conducted by Bjorkman and Svensson (2009) in Uganda for 50 communities showed positive effects related to the weight of infants receiving services by health providers and a marked decrease (33%) in under-5 mortality just one year after the first round of baseline community meetings.

11
# **4. Participation**

## **Participatory budgeting refers to the devolution of decision-making authority and**

**monitoring to local actors** . Public participation in the budget process can take place at different stages of the cycle, including budget preparation, implementation and oversight. The purpose is to increase transparency, tap into local sources of information, improve accountability of elected officials and public service deliverers and encourage innovation. Participatory budgeting creates incentives for public involvement and strengthens civil society through an institutional mechanism that can contribute to building an empowered citizenry including the poor (Baiocchi, Heller, Chaudhuri and Silva 2006).

**Participation by the community ranges from direct democracy to indirect delegate models.** _Direct democracy_ is where citizens are entitled to participate directly in neighborhood meetings to prioritize publically funded projects where each participant can vote and in some cases also be elected as a delegate or councilor. Conversely, in _indirect democracy_ or community-based representative democracy, citizens’ perspectives are carried out through delegate councils or leaders through social movements or neighborhood associations. Participation can take place through approving the budget or in some cases it has been argued, through simple consultations with citizens where the executive and legislative branches make the decisions. The decisionmaking bodies in participatory budgeting (PB) are often built on preexisting social and political frameworks such as neighborhood associations. Alternatively, they can be formed expressly for the purpose of PB through creating a PB council that establishes the decision making system, resource allocation criteria, when and how often plenary meetings will take place along with their agendas, and the finalized budget to present to the municipal councils.

According to Cabannes’ (2004) analysis of participatory budgeting based on 25 municipalities in Latin America and Europe, PB varies by population size, municipal resources, styles of participation (popular participation vs. local government), degree of consolidation of the experiences and amount of funds allocated to PB. Baiocchi, Heller, Chaudhuri and Silva define direct participation as citizens participating in open decision-making forums such as neighborhood assemblies, and delegative participation as new forms of representation through delegate councils.

## **_4.1 Influence of participation on citizen behavior_**

**The logic behind the practice of participatory budgeting influencing citizen behavior is that participation in decision making makes people more likely to comply because they have a say in the process** . Dal Bo, Foster and Putterman’s (2010) experimental findings suggest that individuals are more cooperative and compliant when they are allowed to participate. The results showed that democratically selected policies had a greater impact than policies that were imposed in a non-democratic way. Participatory processes led to greater cooperation both in civic responsibilities (e.g., paying taxes) and in complying with policies related to people’s work and livelihood (e.g., farmer cooperation with irrigation policies).

**Participatory budgeting increased knowledge about taxes and public services and subsequently led to increased tax revenues** . Innovations in budgeting that incorporated direct

12
participation by citizens and new budget processes and patterns of decision-making led to wider public involvement especially among poor citizens in Porto Alegre, Brazil. Lower income citizens understood that increases in taxes would mean increases in services and benefits for them as a result of participating in regional meetings (Schneider and Baquero 2006).

Schneider and Baquero (2006) showed that participatory budgeting mobilized support for tax reforms among all citizens regardless of income level. The common determinant influencing satisfaction with tax contributions was confidence in participatory budgeting. Citizens who trusted in participatory budgeting were more satisfied with their tax contributions. The mediating factor distinguishing between the poor and middle class was perceived benefit. A perceived increase in non-material benefits (i.e., improved efficiency, reduced corruption) increased middle class satisfaction. On the other hand, an increase in material benefits (e.g., investment in social sectors) increased satisfaction by the lower class. Perceptions about what are the benefits of participatory budgeting also differed by income level. The middle class was more likely to link PB to democratic values and more transparent government whereas the lower class was more likely to associate PB with investments for poor and increased social assistance. However, all citizens perceived both material and non-material benefits from participatory budgeting. In short, citizens’ positive perceptions of PB were associated with enhanced satisfaction with their tax contributions and thereby increased tax revenues.

These patterns were evident in a broader comparative analysis of 25 countries in Latin America and Europe that showed that PB led to increases in tax revenues and a decrease in delinquencies (Cabannes 2004). The immediate visibility of the work and services that resulted from PB changed citizens’ views on taxes and their related habits. For example, the PB advisors reported, “The community, on learning what the municipality’s budgetary and financial situation is, becomes aware of its budgetary restrictions. Then, when there are not enough resources for the implementation of its projects, the community decides to collaborate with personnel, financial resources or materials, aiming not only at increasing the resources available for them, but at enlarging the infrastructure initially approved” (p. 36).

# **_Indicators_**

- **_Trust in participatory budgeting_** and perceived benefits were measured by survey responses (Cabannes 2004, Schneider and Baquero 2006).

- **_Compliance_** and increases in revenue were measured by tax receipts (Cabannes 2004, Schneider and Baquero 2006)

# **Citizens are more willing to pay taxes when they perceive that their preferences are taken**

**into consideration by public institutions** . Frey, Benz and Stutzer’s (2004) review of the literature argues that procedural utility – individuals’ preferences about how instrumental outcomes are generated – are a value to citizens’ satisfaction along with the actual outcomes themselves (e.g., in taxation, wealth redistribution, and public good allocation). These arguments support research suggesting that if participatory budgeting makes citizens feel like they have a say in the priority-setting and that their voices are heard, they will be more satisfied with the outcomes even if they are not optimal.

13
Torgler and Schneider (2009) present evidence from around the world (59 countries) showing that if citizens perceive that their interests are properly represented in political institutions (institutional quality is high), the likelihood that they would avoid paying taxes and participate in the underground economy decreases. The other significant determinant is tax morale indicating citizens’ beliefs in their moral obligation to pay taxes as a contribution to society. Likewise, Frey and Frey (2002) demonstrate that individuals experience higher utility when they perceive the taxation process as more respectful and are therefore more willing to pay their taxes.

# **_Indicators_**

- **_Institutional quality_** or perceptions that political institutions represent citizens’ interest in the Torgler and Schneider 2009 study are measured using the “Quality of Governance” index which measures perceive quality of governance (Kaufman, Kraay, and Mastruzzi 2003). The key dimensions are: voice and accountability in terms of the political process, civil liberties and political rights; political stability and absence of violence such as perceptions about the stability of government; capacity of the government to effectively formulate and implement policies including government effectiveness to produce policies and deliver goods, and regulatory quality indicating policy friendliness of environment to businesses; respect for citizens and the state for institutions that govern economic and social interactions including rule of law or the degree of agents’ confidence in and compliance with rules of society and control of corruption or perceptions about the degree of corruption.

- **_Shadow economy_** was measured in the Torgler and Schneider 2009 study as the deliberate concealment of market-based legal production of goods and services from public authorities including avoiding payment of: income, value added or other taxes, social security contribution, minimum wages and complying with safety standards and administrative procedures.

- **_Tax morale_** is measured in the Torgler and Schneider 2009 study as an individual’s belief in the moral obligation to pay taxes as a contribution to society

**Citizen participation in decision making and establishing laws and policies led to smoother project implementation** . For instance, Bardhan’s (2000) experiment shows that citizens’ participation led to fewer hurdles in implementation of an agricultural irrigation project. He found that when average farmers believe they had a say in water rules, rather than the elite or government, they are more likely to be positive about the water allocation system and about rule compliance by other farmers. In fact, the farmers who were typically most guilty of violating the rules (elite) were more likely to follow the rules when they helped to establish them.

# **_4.2 The influence of participatory budgeting on government_**

**Participatory budgeting can increase accountability by municipal governments that then improve openness and transparency** . Participatory budgeting engages citizens and encourages their engagement beyond just sharing their opinions on the budget allocations and identifying projects most important for their communities. Through institutional mechanism of PB, citizens can hold government accountable for budget outcomes in a public forum that contributes to greater transparency.

14
Various qualitative case studies in Brazil, Mexico, the United States and India showed that PB improved decision-making and accountability (Wampler 2004, Ackerman 2003). Wampler’s case studies showed that PB enhanced accountability in Porto Alegre and Reclife and also slightly transparency; however, outcomes from citizen participation on the policymaking process were limited because most of the decision-making power was in the mayor’s office.

PB was primarily effective through providing a mechanism for citizens to express their opinions and to confront government officials directly in public meetings, which translated into increased openness and transparency (Wampler 2004).<sup>3</sup> Although the evidence to date confirming a relationship between the legal institutionalization of participative mechanisms and development outcomes remains limited, Ackerman (2003) also concluded that it is necessary to formally institutionalize participatory procedures in order for PB to work properly.

# **_Indicators_**

The case studies included only vague qualitative descriptions of what can be interpreted as indicators.

- For instance, indicators of _development outcomes_ included the implementation of public works.

- Likewise, indicators of improvements in _vertical accountability_ were simply described as citizens playing a larger role in decision making through engaging in meaningful deliberation and negotiation thereby reducing the role of municipal governments.

- Similarly, _increased transparency_ was defined as governments making more information available.

**Citizen participation in the budget process can enhance budgets for service delivery and infrastructure for the poor and implementation capacity** . Various studies suggest that participatory budgeting leads to better accountability; lower levels of corruption; improvement in public service delivery; reversion of priorities to pro-poor policies; more social justice redistribution in allocation of resources; efficiency gains and more rational administration (Souza 2001, Navarro 2001, Ackerman 2003, Cabannes 2004, World Bank 2004, You and Lee 2013).

PB has been associated with improving the legitimacy of decision-making processes; monitoring of citizen feedback on public policies and services; communication between citizens and government officials; and better understanding of public demand (Martin and Sanderson 1999, Lerner and Wagner 2006, Donnelly-Roark et al. 2001). In addition, it has been shown to improve the capacity of government to allocate funds across divisions to implement planned projects. For example, in the case of Brazil, the PB process produced a budget that included more accurate estimates of receipts and the state spent an amount that was closer to planned expenses (Schneider and Goldfrank 2002).

# **_Indicators_** :

- **_Efficiency_** was defined as whether the PB process allocated sufficient funds across organizations to implement planned projects (Schneider and Goldfrank 2002).

> 3 On the other hand, PB did not have a direct effect on accountability in Sao Paulo due to the administration’s lack of support.

15
- **_PB_** was measured as the percentage of the electorate that participated in the PB process the year before designing the budget (Schneider and Goldfrank 2002).

**Participatory budgeting can improve public management and public service delivery.** For instance, a comparison of participatory budgeting at the county level in Brazil showed that on average, counties where participatory budgeting policies have been implemented are better managed, have lower levels of corruption, and fewer irregularities than socio-economically similar counties without participatory budgeting (Zamboni 2007). This comparison was based on ten counties with PB and ten counties without PB. The counties were matched on variables that influence governance such as voter turnout, the presence of the seat of a judiciary district as well as the number of civil servants and their wages and socio-economic factors like literacy rate, education level, and per capita income.

**Municipalities governed through a system of local participatory democracy were found to increase access to electricity, sewerage, and education faster than those communities ruled by political parties.** In a study of public good provision in poor communities in Oaxaca, Mexico, Diaz-Cayeros et al. (2013) estimated the effects of _usos y costumbres_ —a form of participatory democracy prevalent in indigenous communities—on the provision of local public goods. The research design benefited from an important institutional innovation in the state that allowed indigenous communities to choose their forms of governance. To assess the effects of local governance institutions on the provision of public goods, the authors used propensity score matching and a first differences approach to generate inferences from observational data.

# **_Indicators_**

- The **_effectiveness of participatory budgeting_** was operationalized using proxies based on financial indicators of local public administration including: level of own revenue collection and spending on public capital (Zamboni 2007). The rationale is that effective PB leads to increases in taxes collected, improved local government financial management, and increased capital spending.

- Diaz-Cayeros et al. (2013) also assessed the **_effectiveness of participatory budgeting_** in terms of the allocation of funds for service provision. In this case, where communities could self-select to use either a form of participatory democracy or to be ruled by a political party, a matching procedure was designed to create an appropriate counterfactual through which equally poor, isolated, indigenous communities could be compared in their forms of governance. The dependent variable was the rate of change of public good coverage, in terms of the percentage of the municipal households who have water, sewerage, electricity, and that are illiterate.

- Another type of indicator used by Baiocchi et al. (2004) was based on perceptions of the **_effectiveness of participatory governance_** . This was measured by key actors in local government and civil society as well as a compilation of socio-economic data. Better governance was measured as a variation in the number of and nature of offences found by auditors in the course of the random audits program with respect to the level of compliance with public sector management guidelines and legal requirements at the local level.

16
**Participatory budgeting can lead to increased public services, more transparent delivery, and a greater effort to target poor communities.** Twelve country case studies in Brazil, India, Kenya, Uganda, Sri Lanka and St Vincent illustrated various successes from the introduction of participatory budgeting. For example, in different parts of Kenya, extensive citizen participation led to improved revenue collection and new projects targeted toward the poor. They were successfully implemented because local residents oversaw the projects (Blore, Devas and Staler 2004). Likewise, the “Kerala experiment” in India increased projects for the poor through PB. The “people’s campaign” for democratic decentralization mobilized hundreds of thousands of activists and volunteers to plan and help implement projects for not only the poor but also the “untouchable scheduled caste and scheduled tribe populations” who rarely benefited from project funds (Isaac and Franke 2002). It also brought about special projects for women as well as encouraged female elected officials and activists to participate in public life. There were major breakthroughs in the delivery of public services such as drinking water, sanitation, roads, education and public health services. Outside observers and local residents reported that public services were delivered for the people who needed it most.

# **_Indicators_**

Success in delivering **_services to the poor_** was measured through _project outcomes,_ by Isaac and Franke (2002), including:

- number of houses built for the poor (i.e. members of scheduled or former untouchable caste and tribal caste);

- number of sanitary latrines and percentage of households with access to latrines, within 200 meters;

- number of persons with job training and percentage reduction in unemployment rate; number of cooperatives; and

- number of women in new job areas outside the home.

**PB led to more pro-poor projects through changes in the decision making process in which poor citizens had privileged access and therefore higher levels of participation (Cabannes 2004).** For instance, the over-representation of poor among participants gave them additional power in the participatory budgeting deliberations. Further, the formula used to distribute funds to make other institutional decisions such as boundaries of different regions, gave preference to the poorest neighborhoods that had the weakest infrastructure. For instance, there was an increase in municipal spending on primary education, sanitation, housing and paving, all of which were primarily areas of deficiency in poor neighborhoods.

# **_Indicators_**

- Public **_services for the poor_** were measured by _access to services and infrastructure_ . For example, indicators included: proportion of household with access and connections to potable water sources and municipal sewerage systems; percentage increase in number of paved roads (in kilometers); garbage service and street lighting.

- **_Investments_** were also measured by _average minimum salaries by head of households_ across neighborhoods.

- **_4.3 The effects of participatory budgeting on poverty outcomes_**

17
**Participatory budgeting is linked to positive impacts on poverty reduction through its ability to redirect budget allocations to pro-poor investments and better implement them** . In the previous section, we presented evidence that PB can increase budget allocations to propoor projects and that it can also lead to smoother implementation of related projects. The studies below demonstrate the final link to poverty impact and how the pro-poor projects implemented due to PB alleviated poverty to a degree under certain conditions.

**PB was significantly related to reducing extreme poverty.** Baiocchi, Heller, Chaudhuri and Silva (2006) show that implementing participatory budgeting in 5,507 municipalities in Brazil translated into direct, tangible developmental benefits for the poor. Municipalities with PB were compared to those without PB to determine whether municipalities’ performance improved due to PB over time. The statistical analysis showed that on average, PB increased progress in reducing extreme poverty by more than 40%, controlling for all other factors (initial condition and selection bias). There were overall improvements in the economy as well as for the poor. For example, there was a raised index of progress in reducing the GINI coefficient of income inequality by .05%, and a reduced share of income earned by the richest ten percent by over five percentage points.

**PB contributed to improved development outcomes.** The study by Baiocchi et al. (2006) found that participatory budgeting led to significant improvements in services for the poor including an increase in the percentage of municipal expenditures to health and sanitation; raised index of progress towards universal access to the municipal water network; and a raised index of progress in reducing overall poverty and also specifically among children. Goncalves (2013) similarly confirmed that the adoption of participatory budgeting at the municipal level is associated with increased expenditures on basic sanitation and health services and that this reallocation led to a significant reduction in infant mortality. The analysis was based on a panel data set comprising municipal level information on the adoption of participatory budgeting, public expenditures and health outcomes in Brazil from 1990 to 2004

# **_Indicators_**

**_Participatory budgeting outcomes_** were operationalized by Baiocchi et al. (2006) along four dimensions: municipal finances, public service delivery, human development, and growth and inequality. The indicators were measured as follows.

- _Municipal finance_ : Share of capital expenditures; Share of expenditures on education and crime; health and sanitation; housing and urban development; social assistance and pensions; and share of the budget deficit

- _Public service delivery:_ Percentage of population with access to basic public services including: municipal trash collection services sanitation systems, water systems and electricity connection

- _Human development_ : Human Development Index, infant mortality rate (per 1,000), under-five mortality rate (per 1,000), percentage of children ages who are not in school by age group (7-14 or 15-17)

- _Growth and inequality_ : per capita annual income; percentage of population who are poor, percentage of population who are indigent, percentage of children ages 14 and below who are poor, and/or who are indigent; Income share (%) of poorest 20% of population;

18
income share (%) of richest 10 percent of the population; gini coefficient of income inequality

# **_4.4 The conditions under which PB is more and less effective_**

**Some approaches to participatory budgeting work better under certain conditions.** McGee and Gaventa (2013) noted that “the impact of participatory budgeting in Porto Alegre, with the city’s long history of citizen engagement and (at the time of innovation) a political leadership highlight committed to its success, sets it apart from other contexts which lack these contextual conditions” (p.S19). In this case, participation by poor citizens in budgetary decision-making led to sharp increases in the reach and coverage of essential services, particularly among the poor when there was _a partnership between the executive branch and an activist civil society_ (Baiocchi et al. 2004, Sousa Santos 2005, World Bank 2008).

The World Bank’s impact evaluation of PB in Porte Alegre showed that PB led to improved access to piped water and sewerage. Moreover, PB served as a mechanism for improving propoor capital investments that contributed to the amelioration of living conditions for the poor, that is, _when PB was adopted for at least ten years_ .

In contrast to the direct participatory approach, Ireland’s tripartite approach to participation brought together government, trade unions and producer organizations under the National Economic and Social Council (NESC) through which they engaged in long-term yearly consultations on the economic strategy including tax policy and expenditures. The results indicated that the _PB delegate approach was successful in the long term_ in contributing broadly to growth and macroeconomic stability, rather than solely focused on the poor (Brautigam 2004).

An analysis of ten Latin American countries showed that Chile and Costa Rica’s outcomes related to pro-poor spending in various sectors were most effective compared to other countries yet citizens did not participate in the budgeting process. The findings suggested that _rather than participation by the poor directly, it was participation by pro-poor political parties and coalitions that were key to pro-poor budgeting_ (see also Wampler 2004). This conclusion was also supported by the case of Mauritius where the poor were represented through a progressive left of center political party, which encouraged participation, by the citizenry in budget consultations.

# **_Indicators_**

_Pro-poor spending outcome indicators in various sectors_ included:

- adult literacy rate

- proportion of population with improved sanitation

- proportion of population with access to water sources

- under-5 mortality rate

- life expectancy at birth

- Human Development Index value

19
Data were based on rankings by panels of country experts on different aspects of transparency and citizen participation collected in a survey by the Center on Budget and Policy Priorities (Brautigam 2004).

# **5. Conclusions and Implications**

We found emerging support in the literature for the first key question we set out to answer, which was whether or not we could find evidence of institutional impact on poverty outcomes. The findings in the literature show linkages between the institutional changes achieved by participatory budget initiatives and improvements in the access of the poor to better quality services. Under certain conditions, open budgeting practices can contribute to development impact.

Transparency, particularly as assessed by the Open Budget Index, is a predictor of child and infant health outcomes and access to improved drinking water. Accountability mechanisms have been found to contribute to similar improvements in human development indicators. Participatory budgeting, in terms of the devolution of decision-making authority to local actors, has been associated with improvements in public management and service delivery and has been significantly related to the reduction of extreme poverty. In short, the available evidence underscores the need to better understand how participation, transparency, and accountability in budget processes and policies matter and under what conditions.

An incremental non-linear results chain best articulates the process through which institutional changes in open budgeting can contribute to higher-level human development outcomes. Together, the analyses included as part of this review suggested that the basic steps in this logical sequence and their ongoing interaction for mutual reinforcement are positioned as shown in the figure below.

20
# **Figure 1. Open Budgeting Higher Level Results Chain**

<!-- Start of picture text --> Development impact:<br>Increased access to better<br>quality services for the poor<br><!-- End of picture text --> <!-- Start of picture text --> Improved program<br>delivery<br><!-- End of picture text --> <!-- Start of picture text --> Public<br>participation<br><!-- End of picture text --> <!-- Start of picture text --> budget<br><!-- End of picture text --> <!-- Start of picture text --> Transparenc<br>y<br><!-- End of picture text --> <!-- Start of picture text --> Accountability<br><!-- End of picture text --> An examination of the incremental changes leading from open budgeting to human development outcomes highlights some key dynamics through which the needed changes are sustained and/or strengthened to support intermediate outcomes related to improvements in program delivery. Logical links or ongoing loops in this sequence therefore include:

- _Interdependency among transparency, participation and accountability._ Local actors play important roles for holding government accountable during the budget cycle and fostering the transparency of budget information. Transparency, in turn, is critical for enabling participation, providing the information that local actors need for priority setting. Public sector entities that are held accountable are more likely over time to keep their budget policies and processes transparent and to support a participatory process in setting priorities.

- _The subsequent achievement of key, often mutually reinforcing, intermediate development outcomes._ Institutional changes do not lead directly to poverty impact. Instead, the majority of studies focus on smaller incremental steps related to improving the allocation and execution of budgets for more efficient spending, enhancing budgets available to address pro-poor priorities, and reducing the diversion of funds for unintended expenditures (e.g. corruption).

21
- _Improved program or service delivery as the key lever for influencing development impact_ . Enhancing budgets and using them more efficiently are important outcomes because they allow for more or better service delivery, which potentially broadens the access of the poor to better quality services and ultimately effects human development outcomes. In some cases, the institutional changes occurring through open budgeting can affect the quality of services directly, particularly through the use of accountability mechanisms for quality assurance and improvement purposes.

The review of available evidence therefore confirms that targeted institutional changes related to open budgeting might matter for higher-level development outcomes. There is a recurring set of measurable attributes related to transparency, accountability and participation that can help to increase the access of the poor to higher quality services. However, the literature also presents a formidable caveat for development planning, which is that the specific potential benefits of these institutional changes will always be determined in part by the local context and the presence of certain accompanying conditions.

One of the key questions we sought to answer with this research was whether or not we could identify a set of core leading indicators that could serve as a tool to help ensure that open budgeting practitioners are on the right track. The findings show this proves to be a challenge due to the need to define a nuanced and incremental results chain within each unique context. Further inquiry is necessary based on action research where institutional changes are identified and codified during program design and revised based on real data and experiences during project implementation.

Outcomes of institutional impact will then be easier to track with existing baselines and monitoring data which will make it possible to assess long-term poverty impact. Currently, the research does not document key institutional changes and link them to outcomes. However, the literature has highlighted key characteristics or areas for measurement related to transparency, accountability, and participation that help to indicate that the needed changes are happening if certain accompanying conditions and factors are also in place.

Practitioners have proven methods for measuring common development outcomes (e.g. infant mortality, access to potable water, etc.), but effective approaches for measuring the less tangible institutional changes related to open budgeting in order to trace their relationship to development outcomes have been more elusive. This review of the evidence provides initial guidance on how transparency, accountability and participation can be measured and what conditions should accompany these institutional changes to facilitate their contribution to higherlevel development impact.

Rather than the establishment of standard indicators, this review highlights which aspects of the institutional change are valid for measurement and what accompanying factors to consider. A list of these areas and conditions is presented in the table below. The specific methods and indicators appropriate for assessing transparency, accountability, and participation will depend on the local context.

22
# **Table 1. Tracking Institutional Changes on the Path to Development Impact**

|**Indi**<br>**Are**|**cators /**<br>**as for Measurement**|**Data Sources /**<br>**Methods**|**Supporting Conditions**|**Reference**|
|---|---|---|---|---|
||Establishment of<br>federal legislation to<br>ensure public access to<br>budget information|Legislative records|Presence of an<br>enforcement agency or<br>mechanismwithin the<br>public sector|Robinson (2006)|
||Movement according<br>to a “transparency<br>index” (defined by<br>Islam 2003)|Constructed using 11<br>representative<br>variables from 4<br>sectors (real, fiscal,<br>financial, and<br>external) for 169<br>|Passage of a freedom of<br>information act|Islam (2003)|
|||countries|||
|**C Y**|Level of discrepancies<br>between commitments<br>and actual<br>disbursements|Independent budget<br>analysis<br>or<br>Public Expenditure<br>TrackingSurvey|Presence of a proven<br>accountability mechanism<br>to address discrepancies|Joshi (2010)<br>Robinson (2006)|
|**T R A N S P A R E N**|Breadth of information<br>available about<br>international aid, as<br>reflected by:<br>*degree to which<br>projected future<br>disbursements and<br>targets are published<br>*timeliness and<br>consistency of<br>information from<br>various sources<br>*compliance with<br>national and<br>international aid<br>effectivenesstargets|Expert review|Presence of demand-side<br>(citizen voice) factors to<br>ensure that information<br>can be used to make<br>changes as needed—such<br>as opportunities for<br>citizens and/or CSOs to<br>participate in setting<br>budget priorities|McGee and<br>Gaventa (2010)|
||Country-level<br>comparisons using<br>independent,<br>comparative, and<br>regular measure of<br>budget transparency<br>and accountability<br>around the world.|Overall Open Budget<br>Index—rankings on<br>budget transparency<br>questions by non-<br>government country<br>experts|Index shows relationship<br>with development<br>outcomes even when<br>controlling for other<br>factors. Note: demand-<br>side and supply-side<br>factors play a role in<br>helping to change OBI<br>rankings over time.|Fukuda-Parr,<br>Guyer, and<br>Lawson-Remer<br>(2011)|

23
|**Y**|Institution of a regular<br>and ongoing<br>community monitoring<br>plan|Minutes from village<br>and staff meetings<br>Ratings on community<br>scorecards|Systematic user<br>engagement in the<br>development and<br>implementation of the<br>plan (demand-driven) and<br>incentives for providers to<br>respond to feedback from<br>monitoring|Bjorkman and<br>Svensson (2009)|
|---|---|---|---|---|
|**B I L I T**|# of “under-spends” by<br>government during<br>defined time period|Independent budget<br>analysis and review of<br>errata statements|Direct communication<br>with agency and<br>population not receiving<br>allocatedfunding|Malajovich and<br>Robinson (2006)|
|**O U N T A**|Probability of a public<br>sector entity receiving<br>a government audit|Computed based on<br>the frequency and<br>targeting of auditing<br>by the relevant<br>authority|Presence of a supply-side<br>accountability mechanism<br>supporting government<br>auditing function|Olken (2007)|
|**A C C**|Supreme Audit<br>Institution Strength<br>Index|Rankings by non-<br>government country<br>experts on a subset of<br>budget transparency<br>questions in the Open<br>Budget Index Survey<br>related to the presence<br>and functioning of the<br>Supreme Audit<br>Institution|Index shows relationship<br>with development<br>outcomes even when<br>controlling for other<br>factors.|Fukuda-Parr et al.<br>(2011)|
||Trust in participatory<br>budgeting and<br>perceived benefits|Survey ratings|Tax receipts or other<br>evidence available to<br>confirm revenue and<br>compliance related to<br>taxes|Cabannes (2004)<br>Schneider and<br>Baquero (2006)|
|**A T I O N**|Perceptions that<br>political institutions<br>represent citizens’<br>interests|Quality of Governance<br>Index (see Kaufman,<br>Kraay, and Mastruzzi<br>2003)|Existence of tax morale<br>reflecting sense of<br>individual’s obligation to<br>pay taxes as a<br>contribution to society|Schneider and<br>Baquero (2006)|
|**A R T I C I P**|Efficiency of<br>participatory budgeting<br>process|Independent review to<br>assess whether PB<br>process allocated<br>sufficient funds across<br>organizations to<br>implement planned<br>projects|Presence of participatory<br>budget—as reflected by<br>percentage of electorate<br>that participated in budget<br>process to set priorities|Schneider and<br>Goldfrank (2002)|
|**P**|Effectiveness of<br>participatory budgeting|Proxies based on<br>financial indicators of<br>local public<br>administration such as<br>level of own revenue<br>collection and|Need for improved local<br>government financial<br>management and<br>increased capital spending|Zamboni (2007)|

24
||spending on public<br>capital.|||
|---|---|---|---|
|Engagement of local<br>actors in overseeing<br>budget implementation<br>(e.g., citizen oversight<br>of local development<br>projects)|Qualitative case<br>studies (e.g., could<br>include review of local<br>development plans,<br>meeting minutes,<br>surveys of<br>stakeholders,<br>observations,etc.)|Participatory<br>accountability<br>mechanisms are in place<br>to respond to citizen<br>concerns related to project<br>implementation|Blore, Devas and<br>Staler (2004)|

The findings signaling how institutional changes can contribute to higher-level outcomes in certain contexts highlight the importance of measuring budget transparency, accountability, and participation and tracing their incremental outcomes along a results chain. Documenting these change processes over time would help practitioners to plan for and assess the effectiveness of knowledge initiatives. Overall, this review serves as a starting point and underscores the need for further investigation to establish effective measurement practices of institutional change and build an evidence base for understanding the relative robustness of institutional change paths and the context in which they are likely to matter. We recommend that WBI begin to collect such data with its Open Budgeting program to advance our understanding of institutional impact on poverty in this area.

25