metadata: authors: 0: author: Ivanyna,Maksym
1: author: Shah,Anwar M.
count: World
docna: 0: docna: Citizen-centric governance indicators :
            measuring and monitoring governance by listening to the
            people and not the interest groups
docty: Policy Research Working Paper
subtopic: Human Rights,Judicial System Reform,National Governance,Government Policies,Health Care Services Industry,Educational Sciences,Youth and Governance,Governance Indicators
teratopic: Public Sector Development,Macroeconomics and Economic Growth,Governance,Finance and Financial Sector Development
abstracts: cdata!: Governance indicators are now widely
            used as tools for conducting development dialogue,
            allocating external assistance, and influencing foreign
            direct investment. This paper argues that available
            governance indicators are not suitable for these purposes as
            they do not conceptualize governance and fail to capture how
            citizens perceive the governance environment and outcomes in
            their countries. The paper attempts to fill this void by
            conceptualizing governance and implementing a uniform and
            consistent framework for measuring governance quality across
            countries and over time based on citizens' evaluations.
            Using data from the World Values Survey (and other sources)
            we implement this framework into practice and build
            citizen-centric governance indicators for 120 countries over
            the period 1994 to 2005.
repnme: repnme: Citizen-centric governance indicators :
            measuring and monitoring governance by listening to the
            people and not the interest groups
display_title: Citizen-centric governance
            indicators : measuring and monitoring governance by
            listening to the people and not the interest groups
listing_relative_url: /research/2010/01/11653099/citizen-centric-governance-indicators-measuring-monitoring-governance-listening-people-not-interest-groups
topicv3: Justice m1326106 840,Education m1327048 470,Governance Indicators m1327488 834,Human Rights m1327778 2203,Health m1504661 3480
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Public Sector Development,Macroeconomics and Economic Growth,Governance,Finance and Financial Sector Development
no_of_pages: 41
author_keys: 0: author_key: 997007
1: author_key: 93058
model_extractions: WPS5181 Policy Research Working Paper 5181 Citizen-centric Governance Indicators Measuring and Monitoring Governance by Listening to the People and Not the Interest Groups _Maksym Ivanyna Anwar Shah_ The World Bank World Bank Institute Governance Division January 2010
Policy Research Working Paper 5181

# **Abstract**

Governance indicators are now widely used as tools for conducting development dialogue, allocating external assistance, and influencing foreign direct investment. This paper argues that available governance indicators are not suitable for these purposes as they do not conceptualize governance and fail to capture how citizens perceive the governance environment and outcomes in their countries. The paper attempts to fill this void by conceptualizing governance and implementing a uniform and consistent framework for measuring governance quality across countries and over time based on citizens' evaluations. Using data from the World Values Survey (and other sources) we implement this framework into practice and build citizen-centric governance indicators for 120 countries over the period 1994 to 2005.

This paper—a product of the Governance Division, World Bank Institute—is part of a larger effort in the department to develop analytical methodologies for governance assessments to guide public sector reform efforts in developing countries. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at shah.anwar@gmail.com.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Citizen-centric indicators: governance Measuring and monitoring governance by listening to the people and not the interest groups

Maksym Ivanyna Bavarian Graduate Program in Economics (Regensburg, Germany) and World Bank Anwar Shah, World Bank Keywords: citizen-centric governance indicators, rule of law, fair governance, service delivery, quality of life, trust in government, government oversight, public opinion JEL-classification: H10, H11, H83, I31, O10
# Citizen-centric indicators: governance Measuring and monitoring governance by listening to the people and not the interest groups

### Maksym Ivanyna

Bavarian Graduate Program in Economics (Regensburg, Germany) and World Bank Anwar Shah<sup>∗†</sup>

## 1 Introduction

Since the publication of pioneering work on measuring governance quality by Huther and Shah (1998), there has been a proliferation of composite worldwide governance indicators purporting to measure various aspects of governance quality (see Arndt, 2008, for the history and politics of governance ratings). The growth of these indicators has been spurred by generous support by the development assistance community, especially multilateral development finance agencies, and the infinite appetite of media and the academic community for governance assessments and country rankings. Governance indicators are now being used as tools for conducting development dialogue, allocating external assistance and influencing foreign direct investment. Each new indicator series is now released with great fanfare from major industrial country capitals and the popular press uses these indicators to name and shame individual countries for any adverse change in rank order over time or across countries. The development assistance community is increasingly using these indicators in making critical judgments on development assistance. The World Bank’s International Development Association (IDA) allocation - a window of subsidized lending to the developing world and the United States Agency for International Development’s Millennium Challenge Account use various governance indicators as criteria for allocating external assistance. At the same time, some of the recent findings of these indicators have also led to much controversy and acrimony and thereby con-

> ∗The views expressed in this paper are those of the authors alone and should not be attributed to World Bank and its Executive Directors. Authors are grateful to the participants of Advanced Academic Update ”Governance Indicators and Assessments - Impact and Future Trends” (Maastricht Graduate School of Governance, Netherlands, January 21-22, 2009), Bavarian Graduate Program in Economics Research Workshop (Bayreuth, Germany, January 22-23, 2009) for comments on earlier versions of this paper.

> †Comments are welcome and should be addressed to: ashah@worldbank.org.

1
tributing to complicating the dialogue on development effectiveness.<sup>1</sup> In view of the influential nature of these indicators and potential to do harm if judgments embodied in these indicators are biased and erroneous, it is imperative that they capture critical dimensions of the quality of governance and all countries are evaluated using uniform and reasonably objective assessment criteria.

Do the existing indicators meet this test? While the literature on this subject is woefully inadequate and thin, four widely used indicators - namely the World Bank’s Worldwide Governance Indicators (WGIs), Overseas Development Institute’s World Governance Assessments (WGAs), Mo Ibrahim Foundation’s Indexes of African Governnace (IIAGs) and the United Nations Economic Commission for Africa’s African Governance Report Indicators (AGRIs) - all lack a conceptual framework on governance, lack a citizen-based evaluations and have time and country assessment inconsistencies, making their rankings suspect. A number of recent papers have been especially critical of WGIs for lacking ”concept” (implying lack of clarity in conceptualization) and ”construct” (implying lack of clarity in measurement) validity, sample bias (mostly interest group views), lack of transparency and time inconsistency of definitions and measurements (see Arndt, 2008, Arndt and Oman, 2006, Kurtz and Schrank, 2007, Iqbal and Shah, 2006, 2008, Langbein and Knack, 2008, Schrank and Kurtz, 2008, Thomas, 2006, Thompson and Shah, 2003). One of the most important limitations common to all available composite indexes of governance is that they fail to capture how citizens perceive the governance environment and outcomes in their own countries.

For governance assessments to be useful for policy purposes, they must conceptualize governance and provide uniform and consistent criteria for measuring governance across countries and over time. Foremost concerns for such measurement should be citizens’ evaluation of governance environment and outcomes in their own countries supplemented of course by objective indicators of the same. For development assistance purposes, these indicators could be supplemented by expert-based evaluations. There is some work available on objective indicators as done by the Doing Business indicators of the World Bank and on expert-based evaluations as done for the Global Integrity Index. The most important void in our knowledge is how citizens view governance environment and outcomes in their countries. This paper takes a first step to fill that void.

The rest of the paper is organized as follows. Section 2 discusses conceptual issues in measuring governance, specifies a citizen-centric conceptual framework on measuring governance quality. Section 3 presents an empirical framework, data sources and aggregation techniques. Section 4 presents preliminary results. In Section 5 we discuss the robustness of our results, as well as the contributions and limitations of the empirical approach. A concluding section outlines an agenda for future research.

> 1See Iqbal and Shah (2008) for examples of indefensible country ranking and questionable crosscountry and time series comparisons by one of the more widely used indicators

2
# 2 Conceptualizing and measuring governance quality in a comparative context

Governance is a fuzzy yet fashionable buzzword and its use in the literature has exploded in recent years. Dixit (2008) notes that there were only 4 citations in EconLit in the period 1970-1979 compared to 15455 in the most recent period of 2000-2007 and currently Google lists more than 152000 pages of this literature. According to American Heritage, Random House and Merriam Webster dictionaries, governance is equated with government and is defined as the ”exercise of authority and control” or ”a method or system of government and management” or ”the act, process or power of governing”. Huther and Shah (1998) defined governance as ”a multi-faceted concept encompassing all aspects of the exercise of authority through formal and informal institutions in the management of the resource endowment of a state. The quality of governance is thus determined by the impact of this exercise of power on the quality of life enjoyed by its citizens” (p.2). The World Bank Governance and Anti-corruption (GAC) Strategy (World Bank, 2007) defines it as ”the manner in which public officials and institutions acquire and exercise the authority to shape public policy and provide goods and services” (p.3).

For our current purpose, none of the above definitions with the sole exception by Huther and Shah, is helpful in serving as an operational guide to carry out a comparative review of quality of governance across countries or even of one country over time. This is because of their singular focus on the processes/institutions which do not lend themselves to easy or fair comparability across countries and sometimes not even within one country without conducting deeper analytical studies. There can be little disagreement that same processes and institutions can lead to divergent governance outcomes just as dissimilar processes could yield similar outcomes in two different countries. For example, anti-corruption agencies in countries with fair governance helps curtail corruption but in countries with poor governance prove either to be ineffective or worse a tool for corrupt practices and victimization (Shah, 2007). As another example, budget secrecy prior to its presentation to the parliament is just as important under parliamentary form of government as in Canada, UK, India, New Zealand, as open and participatory budget determination process is to presidential form of government as in the USA. There can be little disagreement that both types of processes have the potential to advance public interest but may succeed or fail in different country circumstances. During the past two decades, we have also seen that single party dominant political systems in China, Malaysia and Singapore have shown dramatic results in improving governance outcomes whereas pluralistic party systems have also shown positive results in other countries such as Brazil and India. Similarly monarchy has shown positive results in UK but unwelcome results in Nepal. Even similar electoral processes do not always lead to representative democracy and may instead yield aristocracy (elite capture) in some countries and corrupt oligarchies in others. In fact, Aristotle’s main argument for elections was based upon the premise that these would produce aristocracy, a form of government he considered superior to median voter rule (see Azfar, 2008). Andrews (2008) argues that such ”good governance picture of effective government ... constitutes a threat, promoting isomorphism, institutional dualism and ”flailing 3
states” and imposing an inappropriate model of government that ”kicks away the ladder” today’s effective government climbed to reach their current state.”(p.2) In any case, such comparisons of processes and institutions out of their context are almost always ideologically driven and value laden and could not be acceptable as unbiased professional (scientific) judgments. This also explains that while citizens of Bangladesh, China, India and Malaysia over the last decade have experienced remarkable improvement in governance outcomes, available primary indicators fail to capture these accomplishments due to their focus on processes at the neglect of outcomes. Even for the world as a whole, the information revolution by letting the sun shine on government operations, has brought about dramatic improvements in government accountability, but the WGIs with their on one-size-fit all vision of the world, have consistently failed to notice or recognize such a mega change. These indicators rank China in the lowest percentile on voice and accountability but according to the former Auditor General of Canada, China has the most effective public accounts committee anywhere which has a track record of holding government to account for malfeasance (Dye, 2007). Furthermore local governments in China have relatively much larger role in public service provision than most countries. Local governments below the provincial level account for about 54% of consolidated public expenditures in China compared to about 4% in India and about 27% in OECD countries (see Shah and Shah, 2006). Thus having the decision making closer to people, directly elected local governments, and party oversight of local government performance - all work to create a system of voice and accountability that is quite unique to China and not easily comparable to other countries (see Qiao and Shah, 2006). China has also demonstrated superior government effectiveness through its unique and unparalleled success in alleviating poverty and improving the quality of life of its citizens over the past two decades. About two decades ago, China had about 35% of its population below poverty level compared to less than 2% in 2006 (see Shah and Shen, 2007). In conclusions, comparisons of governance institutions requires deeper analytical work through in-depth comparative studies rather than aggregate indicators. Such indicators are more usefully used to compare governance outcomes and complementary analytical studies of institutions and process can be used to explain varying outcomes. Of course, governance outcomes also assume commonly shared values but it is relatively less problematic than one-size fit-all prescriptions on processes.

To have meaningful governance comparisons across countries and over time, one needs to have concepts which are somewhat invariant to time and place and are focused on citizens’ evaluations rather than interest groups’ views. To this end, we define governance as an exercise of authority and control to preserve and protect public interest and enhance the quality of life enjoyed by citizens. Note that this definition encompasses both the governance environment (quality of institutions and processes) as well as governance outcomes.

4
# 2.1 Towards a simple framework for assessing country governance quality

Considering a neo-institutional perspective, various orders of government (agents) are created to serve, preserve, protect and promote public interest based upon the values and expectations of the citizens of a state (principals). Underlying assumption is that there is a widely shared notion of the public interest. In return, governments are given coercive powers to carry out their mandates. A stylized view of this public interest can be characterized by four dimensions of governance outcomes.

- Responsive Governance. The fundamental task of governing is to promote and pursue collective interest while respecting formal (rule of law) and informal norms. This is done by government creating an enabling environment to do the right things - that is it promotes and delivers services consistent with citizen preferences. Further, the government carries out only the tasks that it is authorized to do that is it follows the compact authorized by citizens at large.

- Fair (equitable) Governance. For peace, order and good government, the government mediates conflicting interests, is focused on consensus building and inclusiveness and ensures a sense of participation by all and protection of the poor, minorities and disadvantaged members of the society.

- Responsible Governance. The government does it right i.e. governmental authority is carried out following due process with integrity (absence of corruption), with fiscal prudence, with concern for providing the best value for money and with a view to earning trust of the people.

- • Accountable Governance. Citizens can hold the government to account for all its actions. This requires that the government lets sunshine in on its operations and works to strengthen voice and exit options for principals. It also means that government truly respects the role of countervailing formal and informal institutions of accountability in governance.

Given the focus on governance outcomes, Table 1 presents some preliminary ideas for discussion on how to operationalize these concepts in individual country assessments.

The above simple framework captures most aspects of governance outcomes especially those relevant for development policy dialogue and can serve as a useful starting point for a consensus framework to be developed. In any event, there can be little disagreement that one cannot embark on measuring governance quality without first defining and defending an appropriate framework that measures governance - a point also emphasized by Thomas (2006) and the European Commission (see Nardo et al., 2005). Once a consensus framework is developed then one needs to focus on only a few key indicators that represent citizens’ evaluations and could be measurable with some degree of confidence in most countries of the world and could be defended for their transparency and reasonable degree of comparability 5
# <u>Table 1: Governance outcomes and relevant considerations</u>

|Governance outcome|Relevant considerations|
|---|---|
|Responsive<br>governance|• public services consistent with citizen preferences;<br>• direct possibly interactive democracy;<br>• safety of life, liberty and property;<br>• peace, order, rule of law;<br>• freedom of choice and expression;<br>• improvements in economic and social outcomes;<br>• improvements in quantity, quality and access of<br>public services;<br>• improvements in quality of life;|
|Fair governance|• fulfllment of citizens’ values and expectations in<br>relation to participation, social justice, and due<br>process;<br>• access of the poor, minorities and disadvantaged<br>groups to basic public services;<br>• non-discriminatory laws and enforcement;<br>• egalitarian income distribution;<br>• equal opportunity for all;|
||• open, transparent and prudent economic, fscal<br>and fnancial management;<br>• working better and costing less;|
|Responsible<br>governance|• ensuring integrity of its operations;<br>• earning trust;<br>• managing risks;<br>• competitive service delivery;<br>• focus on results;|
||• justice-able rights and due process;<br>• access to justice, information;|
|Accountable<br>governance|• judicial integrity and independence;<br>• efective legislature and civil society oversight;<br>• recall of ofcials and rollbacks of program possible;<br>• efective limits to government intervention;<br>• efective restraints to special interest capture.|

Source: Shah (2008) 6
and objectivity.<sup>2</sup> Having an enormous number of indicators, which could not be scrutinized, is nothing but a distinct disadvantage for a measure that aims for wider acceptance and confidence.

Implementation of the above framework requires a worldwide survey with uniform questionnaire honing on the four dimensions of governance identified above across countries. Given that such a survey is not available and costly to commission, in the following section, we take a pragmatic approach based upon available survey data to develop rough indexes of governance quality.

# 3 Citizen-centric governance: Empirical framework

Following Table 1, public interest is characterized by four dimensions of governance outcomes - responsive governance, fair governance, responsible governance, and accountable governance. Each of these categories is split further on sub-categories in order to characterize a concrete governance outcome (such as improvements in quality of life, safety, peace, etc.). Public opinion survey, with the questions assigned to each subcategory, should be used for the assessment of governance.

The procedure of the assessment consists of the two main steps. First, data source - the raw data from inter-country public opinion survey - is chosen. The responses on questions in the survey, which characterize governance outcomes, are recorded. Second, the responses are aggregated in order to achieve governance index for each country from the sample.

In what follows, we consider both steps in detail.

## 3.1 Data

Reliable, comprehensive and consistent through time and space source of data is essential for qualitative estimation of citizen-centric governance indicators (CGIs). With an additional requirement of being publicly accessible and, preferably, free of charge, such data source hardly exists at present. There is a database of governancerelated questions included into different surveys across the world (Governance Surveys Database published by the World Bank). In principle, each of these questions could be included into our estimation (questions taken separately from different polls) if the data is available. However, as the experiments in the construction of surveys suggest (see Bertrand and Mullainathan, 2001, for examples), even the small difference in the formulation of a question (assigned to the same sub-criterion) or the sequence of questions in a survey may bring significant discrepancies in the responses for the same country and same sub-criterion. Therefore, we decided to use only one data source, which covers sufficient amount of countries. Effectively it means, that almost the same questionnaire is used in all participating countries.

> 2see Andrews and Shah (2005) for details and relevant indicators of an approach that emphasizes citizen-centric governance and Shah and Shah (2006) for citizen-centered local governance and relevant indicators

7
The principal data source for our further analysis is the World Values Survey (WVS) project, conducted by WVS Association (see WVS, 2008). Table A2 shows its characteristics in comparison with other potential data sources. WVS provides an acceptable compromise of consistency and coverage for showing an initial picture of citizen-centric governance indicators. On the one hand, WVS publishes quite outdated information (with the time lag of 2-3 years after actual survey was taken), and only a few questions from this survey are relevant for our purposes (since the survey is about cultural values, not governance). On the other hand, WVS provides quite comprehensive geographical coverage (97 countries with all major economies included) combined with acceptable time coverage and questionnaire.

The coding (which is used further in text and in the dataset) and questions assigned to each sub-criterion of governance are presented in the Table A1 of Appendix. As one can see, for a few sub-criteria, specified in the Table 1 of the paper, no survey questions are available. This is a drawback of WVS, as this survey was not constructed to evaluate governance. However, each governance outcome has a sufficient representation by questions in order to get reasonable estimates.

Based on the data from WVS (questions from the Table A1 of Appendix), as well as from the other freely available data sources (AFR, ASB, TI GCB - see Table A2 for notation), a unique dataset was constructed, which can be used for the evaluation of citizen-centric governance indicators by any researcher. 421994 people’s responses (256152 of them by WVS) on 74 different questions (20 from WVS) are recorded in this dataset. 125 countries are covered, 97 of them by WVS. The records in the dataset can be sorted by the gender, income, education of a respondent, as well as by the sub national administrative unit of his/her residency.

For the reasons explained above our main estimation procedure is based on 3 waves of the World Values Surveys depending on the year when the surveys were taken. Wave 1 includes countries surveyed from 1994 to 1998, wave 2 - from 1999 to 2004, and wave 3 - from 2004 to 2008. In addition to questions from WVS, in the wave 3 we also use one question about corruption from Transparency International Global Corruption Barometer (see TI, 2005).

As an alternative to the WVS, we apply additional data sources in our estimation of citizen-centric governance indicators. In particular, in this paper we report the results when using Gallup World Poll data points, which are available freely from the Worldwide Governance Indicators (WGI) project (see WBI, 2008).<sup>3</sup> 4 questions from GWP are used in WGI. While this coverage is quite limited, yet it allows us to estimate 3 governance outcomes for a wide range of countries.

# 3.2 Aggregation

The underlying assumption of our empirical investigation is that the quality of governance in a given country directly affects governance outcome, which is being analyzed in a certain survey question. Thus, the answers of survey respondents - citizens of this country - are better for each question the higher is the quality of governance in the country. At the same time, answers of the respondents are random

> 3Gallup World Poll, described in the Table A2, is itself very expensive (28 thousands US Dollars per year), and therefore cannot be used as a base for a rigorous, replicable research

8
variables, which are subject to personal errors: where i = 1, .., M is the index of a country, j = 1, .., Ni is the index of a respondent (total number of respondents, obviously, changes from country to country), and k = 1, .., K is the index of a question in a survey (thus of a particular governance outcome). sijk is the answer on question k of the respondent j in the country i. Each response was normalized by us on a scale from 0 to 1, with 0 being the worst answer, and 1 being the best answer. gi is the quality of governance in the country i. It does not depend neither on concrete respondent, nor on specific question. Coefficient βk reflects a degree, to which governance affects the answer of a respondent. Note that it does not depend on country or respondent. Finally, ǫijk ∼ N (0, σik<sup>2is the personal</sup> random error of the respondent j in the country i, which may also depend on a specific question. Each error is independently normally distributed with zero mean and the variance σ<sup>2whichmaydependoncountryandspecificquestion.</sup> ik<sup>,</sup> The expression for gi can be rewritten: <u>1</u> where wk = βk<sup>-arethequestion-specificweightsassignedtoeachquestion.The</sup> weights are normalized to add up to one -<sup>�K</sup> k=1<sup>= 1-sothatgiisbetween0and1</sup> for each country. For our main estimation, and for further comparative analysis, the weights are exogenously chosen and are reported in the Table A1 of the Appendix. They reflect the relative importance of every question in assessment of governance (i.e. ”satisfaction with life in general” is clearly more comprehensive than ”satisfaction with health” or ”satisfaction with environment”), as well as alleviate certain data deficiencies (i.e. European countries were not asked some questions in the second wave of WVS, so these questions received lower weight). At the same time, the weights can be easily changed to tailor one’s specific research agenda or check the robustness of the results.

Given our assumptions, the most efficient, unbiased, and consistent estimator for the governance in country i is just the sample mean of weighted averages of citizens’ responses, the estimator for the governance’s variance is adjusted sample variation: We gave up more sophisticated data mining approaches (e.g. principal component analysis, canonical analysis or random projections) for the sake of transparency and simplicity. The choice of weights or aggregate procedure does not significantly change the appearing governance picture (see Section 5). Our procedure is maximally open and simple in order to allow for a further research and analysis. Besides, in addition to the governance scores we report and analyze the aggregate responses on each question, which makes our indicators ”actionable”, and allows drawing the conclusions, which are completely independent of weights and aggregation proce9
dure.

# 4 Citizen-centric governance: Preliminary rankings

Based on the estimation procedure described above we report our results in this section. First, we analyze citizen-centric indicators (CGIs) as well as responses on separate questions in all countries in 3 waves of World Values Surveys and Gallup World Poll. Then we compare the indexes by groups of countries, through time (across 3 waves), and with other governance indicators (in particular, Worldwide Governance Indicators). In the last subsection, we give examples of sub-national CGIs in several countries.

## 4.1 Country rankings: Waves 1 to 3

The countries’ citizen-centric governance indicators (CGIs) are presented in Figure 1 and Figure 2. On the first figure we show the estimations based on the data from World Values Survey, for the second figure we use the data from Gallup World Poll (see Section 3.1 for details about data sources). All 3 waves of WVS surveys are shown in Figure 1: (a) Wave 1 - for surveys taken between 1994 and 1998 (53 countries), (b) Wave 2 - for surveys taken between 1999 and 2004 (71 countries), (c) Wave 3 - for surveys taken between 2005 and 2008 (51 countries).

The maps of citizen-centric governance evaluations are, in our opinion, more convenient tool for analysis than the tables with more than 100 records, though those are also available from authors at the request. In Figure 1 we split our sample of countries into 3 broad categories (6 categories in Figure 2): from dark-green highgovernance-quality countries to light-green low-governance-quality countries. While developed countries (especially Scandinavian countries and Switzerland) show stable and high grades, it is rather unexpected that East Asian countries (especially, Vietnam, China) are relatively high rated. In some countries of the Middle East (Jordan, Saudi Arabia) the popular support of the government is also ”unexpectedly” high. At the same time, countries of Central and Eastern Europe are always in the lowest percentiles of the samples.

In Figure 3 we compare citizen-centric governance indicators with corresponding Worldwide Governance Indicators (WBI, 2008), which are considered to be the ”gold standard” of governance assessment by the media. The scale changes from dark-green for countries, which were severely underestimated by WGIs, to darkred for countries, which were greatly overestimated. 27 out of 82 countries in our sample were over- or underestimated at a significance level less than 25% (9 at a level less than 5%) by WGIs in comparison to our assessments. The pattern described in the paragraph above is supported: Middle East and East Asian countries are mostly underestimated (with China, Vietnam, Iran and Saudi Arabia being the leading outliers), while Central and Eastern European countries are too praised by WGI (Latvia, Lithuania, Moldova and Hungary being the leading outliers). Apparently, our indicators reflect last decade’s obvious successes of East Asian and Middle 10
<!-- Start of picture text --> HM >0.55<br>HN 0.45-0.55<br><0.45 Zé ti<br>(a) Wave 1: ~~ gy<br>1994-1998 %<br>ss<br>MM >0.< pe a NW%<br>MN 0.5-0.6 es<br><0.5 } a<br>(b) Wave 2: j ?<br>1999-2004<br>i ts : ><br>, a pe _TMp< iy‘}<br>|<br>MM >o.s . ty? ‘ve<br>MN 05-06. w» ee:oy;x4<br>(c) Wave 3: 4 is<br>2005-2008 7<br><!-- End of picture text -->
<!-- Start of picture text --> Ma >0.67 * > pe<br>WM 058-067 2 TM . és Nene.<br>WNmm 0.50-0.58 Ps eeNe<br>Py) 00 . 3443 -0. 4350 [f . | a<br><0.34 i ~~ gy<br>no data -<br><!-- End of picture text -->

<!-- Start of picture text --> SSS = — ~—<br>u.<2.5% Ory A : ’ are<br>Wily 510% ie “eS<br>U.10-25% 9 i 2<br>WN0. 10-25% s p , .<br>MMH0.5-10% = | -<br>WMH 0.255%<br>WM0. <2.5%<br><!-- End of picture text -->
<!-- Start of picture text --> 06 aw ry 7 f} \~} \\ so<br>t=]<br>a v S| EAT ><br>0.40.3 \/i REe., -| E®\4Poi >.<br>v<br>0.2 T t — 7<br>seS &A AS© s& &SY &&S£ &&€ Ro© 7Oy© sw© ro*& SsEK =SS <© < *5 c<br>* x» > v Ss g & s Ss £ & Ss S& <&<br>eo & s we € Ss Ss ¢ Fe<br>—e World —e—EU-15 ©—CEE w= East Asia<br><!-- End of picture text -->
fear to disclose their true opinion about government. Alternatively, when mass media in a country are controlled by the government, people in this country may be indoctrinated to believe and trust those on the top. In the Section 5.2 we analyze these possible effects and their magnitude for the countries from our sample.

# 4.2 Intertemporal comparison

The consistent through time questionnaires of the WVS and repeated surveys during three waves allow us to assess the progress of the governance in certain countries. In particular, citizens of 41 countries were surveyed both during the first wave of WVS (1994-98) and during the second wave (1999-2004). Surveys both from the second wave and the third wave (2005-2008) are available for 33 countries.

In Table 2 we report the countries which achieved the biggest progress in each governance outcome (both from Wave 1 to Wave 2, and from Wave 2 to Wave 3). Not surprisingly, the list is dominated by developing countries and the countries in transition - of 110 positions (10 governance outcomes plus CGIs themselves) only 14 are taken by developed countries (Spain and Germany between waves 1 and 2, and Japan between waves 2 and 3). These numbers clearly reflect increase in the standard of living and stable economic growth in certain parts of the world. Especially it concerns the speedy economic recovery of CEE countries after the horrible post-communist ”hangover” of the 1990s. The most commonly mentioned countries are Nigeria, Venezuela, Latvia, Bangladesh, Moldova between waves 1 and 2, and Turkey, Russian Federation, Jordan, India and South Africa between waves 2 and 3.

The governance in the world (over the sample of countries surveyed by WVS) statistically significantly (at the level of less than 1%) increased from wave 1 to wave 2 (see Figure 5) - in contrast to the WGI’s world of unchanging governance quality - but practically did not change from wave 2 to wave 3. As it can be seen from the figure the main driver of the growth in world’s quality of governance was increasing (in practically all regions) satisfaction of the citizens with their financial situation. This trend was kept from wave 2 to wave 3 as well, but the overall progress was apparently mitigated by the fall of confidence in governments, courts, army, etc. in developing and countries in transition (though CEE countries still ended up progressing from wave 2 to wave 3).

# 4.3 Subnational CGIs

Our estimation procedure as well as dataset collected allows us to extend citizencentric governance indicators from countries to their subnational units. The idea is to aggregate the citizens’ responses not over the whole country, but over its jurisdictions. For the Wave 3 of WVS there are 1121 of them in the sample - usually the second tier of a country’s administrative structure (in some countries - groups of second tier jurisdictions).

The examples of some countries are given in Figure 6. On the left we depict Germany, and on the right - Italy. Both countries were surveyed in 2006. In Germany 14
Table 2: CGI <u>(WVS):</u> top <u>performers</u> by the <u>progress</u> in time

|Governance outcome|Top-performers:<br>Wave|Top-performers:<br>Wave|
|---|---|---|
||1 to Wave 2|2 to Wave 3|
|Total CGI|Nigeria,<br>Germany,<br>Venezuela,<br>Latvia,<br>Finland|Turkey,<br>Russian Fed-<br>eration, Jordan, South<br>Africa, India|
|Responsive<br>gover-|||
|nance|||
|safety of life, order, rule|Macedonia,|India, Morocco, Japan,|
|of law|Bangladesh,<br>Nige-<br>ria, Venezuela, Latvia|China, Korea|
|improvements<br>in<br>eco-|Venezuela,<br>Moldova,|Turkey,<br>Jordan,<br>Ar-|
|nomic<br>and<br>social<br>out-<br>comes|Spain,<br>Nigeria,<br>Ar-<br>gentina|gentina, Korea, South<br>Africa|
|improvements in quality|Estonia,<br>Bulgaria,|Turkey,<br>Jordan,|
|of life: general|Moldova,<br>Venezuela,<br>Slovenia|Russian<br>Federation<br>Ukraine, Moldova|
|improvements in quality|Nigeria, South Africa,|Moldova, Jordan, Ar-|
|of life: health|Mexico,<br>Bangladesh,<br>BiH|gentina, Indonesia, Mo-<br>rocco|
|peace|Bangladesh,<br>Latvia,|Bulgaria, Italy, South|
||India,<br>New<br>Zealand,<br>Macedonia|Africa, Chile, Mexico|
|Responsible<br>gover-<br>nance|||
|earning trust: executive|Venezuela,<br>Nigeria,|Turkey,<br>Iraq,<br>South|
|branch|New<br>Zealand,<br>Spain,<br>Albania|Africa, Argentina, Ko-<br>rea|
|earning trust: legislative|Nigeria, New Zealand,|Morocco,<br>Turkey,|
|branch|Venezuela, Spain, Ger-<br>many|South<br>Africa,<br>Korea,<br>India|
|Accountable<br>gover-<br>nance|||
|access to information, in-|Bangladesh,<br>Germany,|Bulgaria,<br>Morocco,|
|dependent mass media -<br>press|Slovenia, Sweden, India|Vietnam, Jordan, India|
|access to information, in-|Albania,<br>India,|Morocco,<br>Iraq,<br>Viet-|
|dependent mass media -<br>television|Bangladesh,<br>Nige-<br>ria, Venezuela|nam, Jordan, Egypt|
|judicial integrity and in-|Macedonia,|India, Japan, Morocco,|
|dependence|Bangladesh,<br>Nige-<br>ria, Venezuela, Latvia|China, Turkey|

Note: Top performers - in each governance outcome (as defined in the Table A1) 5 countries with the biggest mean difference between corresponding waves 15
<!-- Start of picture text --> World EU-15<br>0.6 0.6 —<br>o4 - o4 -<br>0.2 - 0.2 -<br>a a<br>fin.situation peace conf.gov't conf. courts cel fin.situation peace conf.gov't conf. courts cel<br>wave 1 Mwave 2 wave 2 Mwave 3 wave 1 MB wave 2 wave 2 Mwave 3<br>CEE countries EastAsia<br>8.6 HHO.AH 0.28 TT<br>a4 a — a) 6 =<br>o4 -<br>02 - 02 -<br>a a<br>fin.situation peace conf.gov't conf. courts cel fin.situation peace conf. gov't conf. courts cel<br>wave 1 Mwave 2 wave 2 Mwave 3 wave 1 MB wave 2 wave 2 Mwave 3<br><!-- End of picture text -->
<!-- Start of picture text --> w<br>Ger Ym Italy<br>eaa a :<br>ma 29:<br>MN 0.60-0.65<br>: WN 0.55-0.6<br>| oo |<br><!-- End of picture text -->
resolve some of them. First, we provide some arguments in favor of our aggregation procedure and overall analysis of the data. Second, we make a critical assessment of the data we have available.

# 5.1 Alternative aggregation techniques

Transparency, simplicity and possibility to tailor the assessment procedure for one’s research agenda are the main reasons behind adopting our aggregation procedure - taking weighted averages of citizens’ responses. Besides, some questions are relatively more important and comprehensive for assessing governance, which cannot be detected by mechanized data mining algorithms. In addition, many of our findings and conclusions concern directly separate governance outcomes (responses on a separate question), which does not depend on aggregation procedure.

Nevertheless, we use alternative aggregation techniques to test the robustness of our results. In particular, we apply uniform weights to our data, as well as we use averaging over percentile rankings (the way it is done in the Doing Business project - Djankov, 2007). Naturally, both methods produce slightly different rankings comparing to our main methodology. In particular, European countries lose some positions and East Asian countries gain - the result of increased reliance on the governance outcomes, which are related to trust and confidence in governmental institutions. However, only 11 of 51 countries in case of uniform weights (10 out of 51 in case of averaged percentile rankings) significantly change their standing (according to classification provided in the Figure 1, wave 3 - when country changes one of three categories).

# 5.2 Adjusting the data

In our estimation we use survey data from countries around the world, and the public opinion in a country - especially about the issues related to the government - might be influenced by factors, which we would definitely like to account for. One of the factors is so-called ”intimidation” effect, when people are afraid to express their true - negative - opinion about their government, because they think they could be punished for that. Another factor, frequently mentioned in the literature, is the ”indoctrination” effect, when mass media in a country praise the government so much, that it has a significant positive impact on public opinion. Another factor is the degree of citizen activism and perceived role of government in a country. In particular, Norris (1999) argues about the emergence in the 70s in developed countries of the class of so called ”critical citizens” - people, who were becoming more and more critical and demanding towards their governments despite their obvious successes.

Taking into account 3 factors mentioned above (”intimidation”, ”indoctrination”, ”critical citizenship”) we conclude that in general a response on a question about governance outcome of an individual might be affected not only by the quality of governance in a country. The true model can be rewritten in the following way: 18
where similarly to the notation from Section , sijk is a response of an individual j in a country i on a question k, gi is the quality of governance in a country i, , and ǫijk is a citizen-, country- and question-specific error. intij, indij, cr ~~c~~ itij are the degrees of intimidation, indoctrination and critical citizenship of an individual j in a country i. γik, ηik and μik - depending on country and question - are the coefficients of our interest.

The estimation of γik, ηik and μik is not possible from the model above, since we do not observe governance gi (this is in fact what we are trying to assess). However, the problem can be resolved if we note, that for some questions (governance outcomes) there are no effects of intimidation, indoctrination or critical citizenship, and for some there are. For instance, when an individual is asked about the satisfaction with her/his health, it is likely that she/he will not be intimidated to say true. At the same time, questions like ”Do you have confidence in your government?” are most probably subject to all above mentioned effects. Therefore, by taking the difference between the answers on these questions we can get rid of the governance on the right-hand side while intimidation, indoctrination and critical citizenship effects remain. The estimation model than become: where sijk, k = 1, .., K1 are the citizens’ answers on the questions, which are exposed to the biasing effects (intimidation, indoctrination, critical citizenship), sijk, k = K1 + 1, .., K2 are the answers on the questions with no role for above mentioned effects. Therefore, the left-hand side of our model is the difference between the averages of the two groups of questions (governance outcomes). Assuming that these groups of governance outcomes explain governance to the same degree (average βk’s are the same) we get rid of the quality of governance in the right-hand side, and can test for γik, ηik and μik directly. After taking into account these effects the estimator for the quality of governance can then be expressed as: gi is now the weighted average of people’s responses (the formula we adopted in the main body of the paper) less the effects of intimidation, indoctrination and critical citizenship - averaged over all residents of a country surveyed and multiplied by the weight of the questions in the survey, which are exposed to these effects.

We assume the following questions (governance outcomes) to be independent from the bias

- How satisfied are you with the financial situation of your household? (improvements in economic and social outcomes)

- All things considered, how satisfied are you with your life as a whole these days? (improvements in quality of life: general)

- All in all, how would you describe your state of health today? (health)

19
- How serious do you consider poor water quality, air quality, sewage and sanitation to be here in your own community? (environment)

- Taking all things together would you say you are [happy, unhappy]? (happiness)

On the opposite, the following questions (governance outcomes) are assumed to be exposed to bias effects:

- How much confidence do you have in government? (trust: executive branch)

- How much confidence do you have in parliament? (trust: legislative branch)

- How much confidence do you have in press? (trust: press)

- How much confidence do you have in television? (trust: television)

- How much confidence do you have in courts? (trust: courts)

# 5.2.1 Testing for the intimidation, indoctrination and ”critical citizenship” effects

We use 2 types of estimation procedures to extract γi, ηi and μi - effects of intimidation, indoctrination and ”critical citizenship” in a country i. First, we test for indoctrination (ηi) on an individual level, since there can hardly be any proxy for biasedness of mass-media (indoctrination) on a country-level. On a contrary, it is hard to come up with the proxies for personal intimidation and ”critical citizenship” (this effect was in fact defined only for countries as a whole). That is why we use country-level regressions to identify these effects.

As the proxy for indoctrination we take the frequency, with which an individual exposes her- or himself to media - TV and press. Specifically, we use questions ”Did you watch TV during the last week?” and ”Did you read newspapers last week?” from the World Values Survey. The more people watch TV or read newspaper the more they are exposed to possible indoctrination (or excessive criticism of massmedia). The exact estimation model then becomes: where tvij, pressij are the dummies for watching TV and reading newspapers last week (as it was posed in the questions of the survey), demogrij is a set of individual demographic variables (we take respondent’s education, income, age, marital status, political activism - participation in demonstrations, boycotts, signing petitions).

We report the results in the Table 3. The main conclusion from it is that even though developing countries, especially those in Middle East and East Asia, seem to be indoctrinated, the mass media bias is also present in many developed countries - Japan, Sweden, Switzerland, USA, France. This might be the outcome not of state monopoly (or dictate) on mass media, but of too optimistic or patriotic news coverage in these countries. The magnitude of the indoctrination effect ranges from 0.02 (except for Ukraine and Rwanda, where those who watch TV are actually more 20
||Table 3: Mass media bias in public opinion||
|---|---|---|
|Media<br>bias,|||
|magnitude<br>(η1i, η2i)|TV|Press|
|0.08 - 0.12|Japan,<br>Mexico,<br>India,<br>Slovenia,<br>Cyprus,<br>Ethiopia|Thailand, Cyprus|
|0.04 - 0.08|Sweden, Switzerland, Brazil, Turkey, Peru,<br>Moldova, Indonesia, Vietnam, Serbia, Egypt,<br>Andorra, Burkina Faso, Zambia, France|Jordan, Malaysia|
|0.02-0.04|China|USA,<br>Mexico,<br>Brazil,<br>Romania,<br>Egypt|
|≈0|Argentina, Australia, Bulgaria, Chile, Taiwan<br>Germany, Ghana, Italy, Republic of Korea, M<br>lands, Poland, Russian Federation, South Af<br>and Tobago, United Kingdom|, Colombia, Finland,<br>ali, Morocco, Nether-<br>rica, Spain, Trinidad|
|-0.08 - -0.02|Ukraine, Rwanda|Indonesia|

Note: First column - ranges for point OLS estimates are reported. For each range, only the countries, for which coefficients are different from 0 at a significance level less than 5%, are reported. ”≈ 0” range - countries with no significant TV or press bias. Sample of the countries used - WVS wave 3 (except Iran, Iraq, Hong Kong, New Zealand, where questions about mass media were not asked) critical towards the government) to 0.12, which combined with on average 75% of respondents watching TV or reading newspaper, may lead for some countries to a decrease in our estimates of governance by 0.005-0.03 points.<sup>5</sup> Intimidation and ”critical citizenship” effects are estimated on a country level. Specifically, as a proxy for the intimidation level in a country we use the average score of the country in the ”Freedom in the World” ranking - an annual publication of the Freedom House, where political and civil rights of the citizens are assessed. As for the ”critical citizenship” effect, we follow Pippa Norris (Norris, 1999) in her definition of a ”critical citizen”, and define the country to be in the stage of ”critical citizenship” if it had been classified ”free” by the Freedom House for at least ten years before the survey was conducted (long period of stable democracy), and the GDP per capita in this country (taken from IMF) was more than 10000 US dollars (wealthy population). Most OECD countries together with Slovenia and Chile enter the group. The estimation model than becomes:

> 5Note that our estimates of governance are assessed on a scale from 0 to 1.

21
|Table 4: Efect|s of indoct|rination|and ”critic|al citizenship”|
|---|---|---|---|---|
|Dependent vari-|Coef.|Std.|P>t|95% conf. int.|
|able - dif||Err.|||
|freedom|−0.03<sup>∗∗∗</sup>|0.007|0.000|-0.05<br>-0.02|
|cr<br>~~c~~it|0.09<sup>∗∗∗</sup>|0.025|0.001|0.04<br>0.14|
|F(6,157)|17.65||||
|Prob¿F|0.00||||
|R-squared|0.4||||
|Adj. R-squared|0.38||||
|No.<br>of observa-|164||||
|tions|||||

Note: *** - significant at less than 1% level. Method of estimation - OLS. Sample - countries surveyed by World Values Survey during all 3 waves.

where freedomi is an index of Freedom House, cr ~~c~~ iti is the ”critical citizenship” dummy defined above, and demogri is a set of demographical country-specific variables (average level of education, share of married population, share of males, average age).

The estimation results are presented in the Table 4. As one can see from the table, both freedom of the county and its being in the stage of ”critical citizenship” are highly statistically significant in explaining biases on responses on trust-related questions in the WVS surveys. The directions of the effects are what would be intuitively foreseen. In the Freedom House ranking a country has the higher score the less civil and political rights it’s citizens have: 1 is the best score, 7 is the worst. Therefore, negative γ in our estimation means that the intimidation effect plays a greater role in less free countries. 1 score up in the Freedom House ranking of a country makes the citizens of this country to be more cautious in answering governmentrelated questions in a survey, and consequently overestimate their governments in trust-related questions by 0.03 points. For a completely depressed country (with the score 7) the effect on our governance estimate would be -0.07 points. From the other side, residents of the countries, which are in a stage of ”critical citizenship”, do have significantly less confidence in their governments then they should have had. If not too ”critical”, residents of these countries would give their governments score 0.09 points higher, which would be reflected in the increase of citizen-centric indicator on about 0.03.

Even though we find statistically significant effects of indoctrination, intimidation and ”critical citizenship” in some countries, the magnitude of these effects is not particularly immense. For example, Vietnam with our score of 0.72 is not a free country based on criteria of Freedom House (it had rank 6 in 2005), and there is a moderate (0.05) effect of indoctrination on television. Together these effects would cut citizen-centric governance indicator in Vietnam by 0.07 points. New indicator would be 0.65 - still in the highest 20th percentile of the sample. Apparently, there are other reasons for some governments to score so high in the public opinion polls. In case of East Asia the main of them is probably last decade’s stable economic 22
growth and development in the region (as it is argued for China by Wang, 2005). At the same time, poor economic performance, political conflicts and corruption in the 90s (and for many countries up until today) in Central and Eastern European countries keep the scores the governments in this regions extremely low.

# 6 Concluding remarks

This paper has provided a conceptual framework for measuring governance quality using citizens’ evaluations consistently across countries and over time. It further provided empirical illustration - using data from the World Values Survey Association - of the usefulness of the methodology by developing governance quality rankings for 120 countries. These rankings significantly differ from those provided by available indicators that mostly capture foreigners’ (mostly interest groups) or arm-chair experts’ opinions.

The surveys of the WVS project are certainly subject to important limitations. They are not conducted in the same year for all countries, and the questionnaires may slightly differ from country to country, which may produce significant departures from objective estimation. It is also possible that in spite of the claims to the contrary by the survey organization, the survey may not be based on stratified random sampling for some countries due to practical difficulties (for instance, WVS for Vietnam).

Notwithstanding these limitations, the dataset constructed by us has important merits. The governance-related questions and answers are reported on the level of individual respondents in our dataset, which gives researchers a great flexibility in composing the rankings. In particular, it is possible to compose rankings among the people with higher education, different genders, income, etc. Most importantly and contrary to many other indicators, the data used in our estimation are freely accessible, and can be easily used by other researchers to replicate or modify our estimation procedure.

Ideally, our theoretical framework should be implemented using a world poll with stratified random sampling employing a uniform questionnaire across countries and over time. The World Gallup Poll or a similar instrument might offer such an opportunity in the near future.

23