metadata: authors: 0: author: Savedoff,William D.
count: World
docna: 0: docna: Governance in the health sector: a strategy
            for measuring determinants and performance
docty: Policy Research Working Paper
subtopic: National Governance,Government Policies,Health Care Services Industry,Public Health Promotion,Health Service Management and Delivery,Youth and Governance,Governance Indicators
teratopic: Governance,Health, Nutrition and Population
abstracts: cdata!: Many different strategies have been
            proposed to improve the delivery of health care services,
            from capacity building to establishing new payment
            mechanisms. Recent attention has also asked whether
            improvements in the way health care services are governed
            could make a difference. These approaches ask which factors
            -- such as rules and institutions -- influence the behavior
            of the system in ways that are associated with better
            performance and outcomes. This paper reviews the concept of
            governance as it is used in the literature on private firms,
            public administration, international development and health.
            It distinguishes between indicators that measure governance
            determinants from those that measure governance performance
            in order to propose a framework that is analytically
            coherent and empirically useful. The framework shows how
            these indicators can be used to test hypotheses about which
            governance forms are more useful for improving health system
            performance. The paper concludes by proposing specific
            measures of governance determinants and performance and
            describes the instruments available to collect and interpret them.
repnme: repnme: Governance in the health sector: a strategy
            for measuring determinants and performance
display_title: Governance in the health sector: a
            strategy for measuring determinants and performance
listing_relative_url: /research/2011/05/14168663/governance-health-sector-strategy-measuring-determinants-performance
topicv3: Health Services Delivery m1327294 1305,Governance Indicators m1327488 834,Public Health m1327519 635,Health m1504661 3480,Private Health Care Service Providers
            m1582905 4583
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Governance,Health, Nutrition and Population
no_of_pages: 46
author_keys: 0: author_key: 778910
model_extractions: WPS5655 Policy Research Working Paper 5655 Governance in the Health Sector A Strategy for Measuring Determinants and Performance _William D. Savedoff_ The World Bank Human Development Network Office of the Chief Economist May 2011
Policy Research Working Paper 5655

# **Abstract**

Many different strategies have been proposed to improve the delivery of health care services, from capacity building to establishing new payment mechanisms. Recent attention has also asked whether improvements in the way health care services are _governed_ could make a difference. These approaches ask which factors— such as rules and institutions—influence the behavior of the system in ways that are associated with better performance and outcomes. This paper reviews the concept of governance as it is used in the literature on private firms, public administration, international development and health. It distinguishes between indicators that measure governance determinants from those that measure governance performance in order to propose a framework that is analytically coherent and empirically useful. The framework shows how these indicators can be used to test hypotheses about which governance forms are more useful for improving health system performance. The paper concludes by proposing specific measures of governance determinants and performance and describes the instruments available to collect and interpret them.

This paper is a product of the Office of the Chief Economist, Human Development Network. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at savedoff@socialinsight.org _The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Governance in the Health Sector: A Strategy for Measuring Determinants and Performance**

by William D. Savedoff* JEL: I18, D01, D23 and H11 Keywords: Governance, Health, Public Sector, Performance, Indicators ________________________________________________________________________

*Senior Partner, Social Insight and Senior Fellow, Center for Global Development. The author gratefully acknowledges Ariel Fiszbein and Sebastian Martinez for commissioning and guiding this work. Conversations with April Harding were essential to developing the framework on authority, information and motivation and bringing the work to completion. Ethan Joselow provided important research assistance by reviewing a wide range of material on governance and indicators. Andrea Feigl prepared a background note on surveys and methods for measuring indicators, contributing a significant part of Section 5. Feedback from World Bank staff at a workshop in September 2009 was also gratefully received. All errors remaining in the work are the author‟s sole responsibility.
**Governance in the Health Sector: A Strategy for Measuring Determinants and Performance** W. Savedoff

# **1. Introduction**

How can countries improve access to health care services? This question is asked around the world in different ways – sometimes focusing on making adjustments to existing systems, sometimes by considering whether to introduce particular programs or policies. More recently, international attention has shifted toward improving the way health care services are _governed_ . Rather than asking whether a particular health care system has the right inputs or produces the right outputs, questions about governance aim to identify factors that influence the behavior of the system, such as rules or procedures that are expected, in turn, to be associated with better performance and outcomes.

While the term “governance” is increasingly being used to draw attention to a number of factors that affect the quality, effectiveness and reach of social services, no consensus has emerged on definitions, frameworks and, in particular, how it applies to the health sector. This paper seeks to draw on existing work from the health sector and broader governance literature to advance the discussion by providing a framework that should be both analytically and operationally useful. Such a framework will also make it possible to assess the usefulness of different measures of governance and to identify promising alternatives. Finally, the paper proposes specific measures of governance determinants and performance and describes the instruments available to collect and interpret them.

# **2. What Is Governance in the Health Sector?**

A key difficulty in finding a consensus definition for governance in the health sector is that governance operates at many different levels. Governance can be analyzed at the broadest level in terms of political actors who contest and collaborate to establish each society‟s particular public policies. Governance can also be analyzed at a secondary level in terms of the forms of these specific public policies, that is, the resulting rules, institutions, laws and enforcement mechanisms. However, governance can also be analyzed at the level of particular organizations, for example, the governance of a social security institute, a district health system or a hospital.

Those concerned with governance in its broadest sense tend to focus on normative principles that guide political and social debate. For example, the United Nations (UN) argues that a country has good governance when its public sector operates according to principles of transparency, accountability, predictability, responsiveness and participation (Welch and Nuru 2006). While the UN asserts that following these principles will help countries achieve social goals (in this case, the Millennium Development Goals – MDGs), this is not really the basis for arguing that these principles are part of good governance. Rather these principles are promoted because they are valued in and of themselves and not because they are instrumental toward achieving particular social outcomes. By contrast, the second and third levels of governance are generally analyzed ______________________________________________________________________________________

-2-
with direct concern for performance and outcomes. The questions being asked are whether particular rules and institutions are beneficial to citizens or shareholders or whether particular organizations perform well (in terms of service or profit).

# **_Definitions of governance_**

The term “governance” has entered international health policy debates from a variety of sources but primarily from the literature on private corporations, public sector agencies, and international development.

The literature on private corporations initially focused on governance in terms of the relationship between shareholders and managers. This literature sought to understand what “governs” the behavior of corporate managers who, over the courses of the 19<sup>th</sup> and 20<sup>th</sup> centuries, displayed increasing autonomy from their companies‟ owners. In addition to exploring the legal delineation of ownership rights and decision-making powers, researchers elaborated the implicit and explicit incentive contracts under which corporate managers function and investigated the formal and informal checks on their actions (e.g. shareholder voting). The literature expanded beyond the shareholder-manager relationship to consider the broader context in which corporations operate – legal provisions required by the state to preserve recognition as independent juridical entities, to sign contracts, or raise capital; regulatory provisions concerning reports to shareholders and disclosure to the public; and, eventually including analysis of cultural and social pressures that can be brought to bear on corporations via implicit or explicit norms. These latter efforts seek to influence corporate behavior in pursuit of broader societal goals such as fair treatment of workers or environmental protection and are often established as voluntary codes of conduct (for examples, see the European Corporate Governance Institute‟s website at http://www.ecgi.org/codes/all_codes.php).

The OECD defines corporate governance as: ... a set of relationships between a company‟s management, its board, its shareholders and other stakeholders. Corporate governance also provides the structure through which the objectives of the company are set, and the means of attaining those objectives and monitoring performance are determined. Good corporate governance should provide proper incentives for the board and management to pursue objectives that are in the interests of the company and its shareholders and should facilitate effective monitoring.<sup>1</sup> This definition summarizes a number of key conceptual features of the corporate governance literature. First, it emphasizes governance as a set of _relationships_ among actors who jointly influence the behavior and actions of a defined entity – in this case, the corporation. Second, it asserts that governance includes the process by which performance goals and objectives are determined. Finally, it notes that governance operates through incentives, that is, mechanisms that influence behavior.

> 1 Definition of Corporate Governance from the Preamble (p. 3) of OECD 2004.

______________________________________________________________________________________

-3-
The literature on governing public sector agencies mirrors the private corporation literature in many ways, with the roles of shareholders and managers replaced by citizens and public officials. Like the literature on corporations, these studies also focus on governance as a set of relationships, goal-setting processes, and incentive structures. However, public sector agencies face a number of additional issues because they respond to a larger number of interest groups. When an organization answers to multiple groups, the ability of any one group to achieve its goals is diluted and the resulting organizational performance is influenced by contesting principals (see, for example, Spiller 1990 and Spiller & Urbiztondo 1994). The existence of multiple principals can create space for the organization to pursue goals that are beneficial to its own staff, promoting their vested interests, rather than the agency‟s public purpose. The agency is also vulnerable to capture by particular groups who have a strong interest in the agency‟s actions when other groups are less strongly affected and, therefore, less likely to organize and pursue their interests (Olsen 1971).

A recent study in this literature defines public governance as, “The ways in which stakeholders interact with each other in order to influence the outcomes of public policies.” (Bovaird and Löffler 2003).This definition emphasizes two key aspects of the public sector governance literature: that governance involves the interaction of multiple stakeholders and that its relevance depends on how it influences outcomes.

A third literature focusing on governance concerns international development. The World Bank has played a central role in developing an entire research agenda around the question of how governance affects a country‟s ability to grow and benefit its citizens. Studies in the late 1990s were concerned with poor performance of many countries receiving aid and found the explanation might lie in the quality of how the recipient countries were governed. Over the last decade numerous studies have looked at governance indicators – ranging from perceptions of corruption to the strength of property protections – to see if they influence economic and social outcomes.

A notable aspect of this literature has been the effort to systematically measure governance at the country level. In the early 1990s, few measures were available, but today, a number of organizations – including the World Bank, the World Economic Forum, Transparency International and Freedom House – publish and update such indicators regularly. In this work, the World Bank defines governance as: ...the traditions and institutions by which authority in a country is exercised, This includes the process by which governments are selected, monitored and replaced; the capacity of the government to effectively formulate and implement sound policies; and the respect of citizens and the state for the institutions that govern economic and social interactions among them.<sup>2</sup> This definition is utilized in the Governance Matters series (which now includes annual data on countries from 1996 through 2006, see Kaufmann, Kraay and Mastruzzi, 2007) and it draws attention to the difficulty of measuring governance. How can we measure “processes,” “capacities” and “respect?” Kaufmann and Kraay (2008) describe the effort

> 2 Kaufmann, Kraay and Zoido-Lobatón (1999) appears to be the first statement of this definition.

______________________________________________________________________________________

-4-
to measure governance as divided into two classes: rules-based measures (e.g. does a country have anti-corruption legislation?) and outcome-based measures (e.g. is the anticorruption legislation enforced?). The Worldwide Governance Indicators (WGI) project<sup>3</sup> gathers both rules-based and outcomes-based measures from multiple sources (310 variables from 30 different sources in the latest iteration); clusters them into six dimension of governance (voice and accountability; government effectiveness; control of corruption; regulatory quality; rule of law; and political stability and absence of violence); and then aggregates and normalizes them. Using this wide range of data sources, it is argued, maximizes the use of information and reduces bias.

The work on governance and development is an ongoing process of refinement, new data collection, testing hypotheses and learning. The creators of these various indicators regularly warn against misusing their governance indices; however the measures are nonetheless consistently misused by investors, journalists, and academics. Some of the most common mistakes are to interpret cross-country differences as if there were no measurement error or to judge a country‟s change in its ranking over time as significant when the underlying measures are only relative (Arndt and Oman 2006).

Even when used correctly, the implications for public policy are not straightforward. Improvements in a governance mechanism can be both cause and consequence (i.e. reverse causality), the result of other changes that simultaneously improve country performance, or necessary but not sufficient conditions for improving country performance. Studies of governance that utilize cross-country data, in particular, are subject to an ongoing and vigorous debate and scrutiny over data, methods, and interpretation.

Three lessons can be drawn from the international development literature on governance. First, governance can be measured. While many measures suffer drawbacks, advances in how measurements are made and how they are used (e.g. addressing bias, reporting errors) show that this is still a fruitful endeavor. Second, governance measures can be useful both for advocacy and analysis. Organizations like Transparency International show how creating an index can keep an otherwise vague issue like corruption visible as a public policy issue. At the same time, an extensive research literature is using governance indicators to further our understanding of how different aspects of governance affect a country‟s performance. Finally, misuse and misinterpretation of governance measures are common. By recognizing particular errors related to misattribution of causality and considering particular mechanisms to be separable from their context, it should be possible to use these measures more effectively.

The term “governance” has entered the health sector literature in at least three different ways, paralleling and influenced by these other bodies of work. As in the corporate and public sector literature, part of the health sector literature has looked at governance from the perspective of relationships among actors as they influence the behavior of specific organizations such as hospitals (Harding and Preker 2003) or mandatory health insurance institutions (Savedoff and Gottret 2008). Much as the corporate and public sector literature was extended to include broader social goals, another stream of work in the

> 3 <u>http://info.worldbank.org/governance/wgi/index.asp</u>

______________________________________________________________________________________ -5-
health sector has emphasized the broader notion of governance, particular using the concepts of stewardship or steering (WHO 2000, Saltman and Ferrousier-Davis 2000). Finally, as in the international development literature, researchers have begun to try measuring governance of the health sector – in some cases as part of the broader governance measurement effort (e.g. the World Bank‟s CPIA includes a component that specifically measures the quality of health sector governance) and in other cases as a focus in its own right (Murray and Evans 2003).

Brinkerhoff and Bossert (2008) provide one of the more complete recent efforts to characterize governance in the health sector in a report they prepared for USAID. They present a framework that bridges the public sector approach and the broader stewardship approach. They define governance in terms of “the rules that distribute roles and responsibilities among societal actors and that shape interactions among them” (Brinkerhoff and Bossert 2008). As in the public sector approach, they emphasize relationships among actors and focus on incentives that encourage actors to perform in accordance with social goals. Like the stewardship literature, they give attention not only to how governance affects the efficiency and outcomes of social services but also to the fairness, legitimacy and openness of the process of policymaking itself.

* * *

Throughout the corporate, public sector, and international development literature, studies can be found that fall into a number of common pitfalls. First, many articles confuse descriptive and normative definitions of governance. For example, identifying secure property rights as an institutional feature that governs economic behavior is different from demonstrating that secure property rights are the best or even a desirable feature for governing an economy. Second, some of the literature confuses governance mechanisms with specific approaches. For example, assessing whether a government exercises regulatory authority to control which medications can be produced, distributed and sold is different from verifying whether a country has adopted a specific form of regulation such as an essential drugs list. A third pitfall is to fall into tautologies by assuming that a sector that performs well must be governed well. Thus several studies propose outcome measures or output measures as indicators of good governance. While outcome and output measures are certainly useful, they cannot be conflated with good governance without undermining the notion that governance arrangements are to some extent independent or exogenous factors that can be manipulated with the goal of influencing performance. The health sector literature concerned with governance struggles with these same problems.

From this literature review, we can identify a number of features for an effective definition of governance in the health sector. First, governance is more operational when understood as a set of relationships that influence the behavior of a clearly identified unit of analysis – a corporation or agency rather than a country or sector. Secondly, relationships matter through their impact on motivation – whether intrinsic and extrinsic, financial or otherwise. Third, governance is more useful analytically and for policy if it is assessed in terms of its impact on an organization‟s performance rather than a set of normative principles. Fourth, we need to measure governance mechanisms directly and not just assess governance in terms of outcomes. Finally, the lessons on how measures ______________________________________________________________________________________

-6-
are commonly misused must be kept in mind at all times, especially when measures are first being developed, collected and assessed.

In light of these strands of the governance literature, this paper defines governance as the <u>combination of political, social, economic and institutional factors that affect the behavior of organizations and individuals and influence their performance.</u> While this definition is still fairly broad, it excludes two important ways in which governance is sometimes discussed. First, it excludes definitions of governance that do not identify a specific organization or individual as the unit of analysis. Under this definition, it would not make sense to talk about governance of “the health sector” because this lacks clarity regarding whose behavior is being influenced. Rather it becomes necessary to specify whether the focus is governance of private health care providers, a ministry of health or a social security institute.

The value of identifying a unit of analysis other than a country or sector is the ability to specify which relationships are likely to make a difference to the unit‟s performance and how they operate. For example, if the unit of analysis is a health insurance agency, questions regarding the authority and impact of boards of directors, regulators and even health insurance competitors are relevant (See Figure 1). By contrast, none of these relationships are likely to be relevant for a Ministry of Health which is likely to be influenced more by parliament, political parties, and the press and, if it directly provides health care services, to face competition from private health care providers.

**Figure 1: Example of Governance for a Specific Unit – a Mandatory Health Insurer**

# **<u>Accountability to:</u>**

**Beneficiaries Government Employers & other Supervisors non-beneficiary contributors Regulators**

<!-- Start of picture text --> Mandatory Health Insurer<br>Forms of Payment Competition With:<br>& Negotiation With:<br>Other mandatory health insurers<br>Providers Other insurers outside the<br>mandatory system<br>Source: Savedoff and Gottret 2008.<br><!-- End of picture text --> ______________________________________________________________________________________

-7-
Second, defining governance in terms of how it influences performance excludes definitions of governance that are concerned with processes independent of their impact on outcomes. Many approaches using moral, philosophical or political grounding establish principles of good governance that are judged on the basis of whether they are fair, equitable, or fulfill rights to participate or be treated as full human subjects. While these are legitimate approaches to governance, the alternative route mapped here is to look at governance instrumentally in terms of the performance of the governed organization. Circumscribing the definition of governance in this way is not proposed as a substitute for definitions which are concerned with legitimacy and social processes that intrinsically value participation, voice or fulfillment of rights. Rather, it is proposed as a complementary approach that is appropriate when the task is to analyze how governance arrangements influence the behavior of organizations that have missions which are explicitly framed in terms of impacts and outcomes (e.g. to halt the spread of infectious disease, treat people with illnesses, provide financial protection against high medical costs).

# **_Distinguishing governance determinants from governance performance_**

The next step in operationalizing this definition of governance is to distinguish elements along the causal chain from governance factors that influence an organization‟s behavior<sup>4</sup> to the effectiveness of the governance mechanisms, the organization‟s performance, and, ultimately, outcomes for clients or citizens. Furthermore this causal chain needs to be elaborated in a way that does not oversimplify the multidimensional system in which organizations function by assuming the process to be linearly additive.

Kaufmann and Kraay (2008) distinguish rules-based and outcomes-based measures of governance. Rules-based indicators describe whether a law, regulation, procedure, or similar form of authority exists, while an outcome-based indicator would measure the degree to which the rule is actually implemented or enforced. As an example, they note that the existence of an anti-corruption law would be a rule-based indicator, while a measure of whether that law is enforced would be an outcome-based indicator. They recognize three limitations of rules-based indicators that demonstrate the need for them to be complemented by outcome-based indicators: “the inevitable role of judgment even in „objective‟ indicators, the complexity and lack of knowledge regarding the links from rules to outcomes of interest, and the gap between rules on the books and their implementation on the ground.” (Kaufmann and Kraay, 2008, p. 9) The World Bank‟s Actionable Governance Indicators (AGI) Initiative makes this distinction between rules-based and outcomes-based indicators more precise (Reid 2009). Using a formal model, it distinguishes between “performance determinants” (rindicators), “governance system performance” (Y-indicators), and “exogenous factors” (s-indicators).

The only caution to offer for this terminology is to maintain a clear distinction between governance performance (e.g. are organizational rules followed?) and organizational

> 4 This paper will refer to organizations but the analysis equally applies to segments of the health sector, such as a market of health care providers or a class of public health insurers.

______________________________________________________________________________________

-8-
performance (e.g. does the hospital have low infection rates?). This latter distinction is important because many measures of governance performance are closely related to organizational performance and, yet, different factors intervene at each stage. For example, absenteeism is a good measure of governance performance – it measures the degree to which governance arrangements promote managerial actions to recruit, motivate, supervise, and discipline staff to comply with their formal work obligations. Improving governance performance in this area, i.e. reducing absenteeism, may be a necessary condition to improve organizational performance but it is certainly not sufficient. Other factors intervene between governance performance and organizational performance, such as worker productivity, availability of complementary inputs, and client behaviors. Organizational performance could subsequently change in any direction, depending on how governance interacts with these other factors.

Depicting the AGI Initiative framework, and showing how external factors can intervene in many different stages, distinguishes governance determinants (referred elsewhere in this paper as governance factors or mechanisms) from governance performance and further distinguishes governance performance from organizational performance (see Figure 2).

**Figure 2: The AGI Initiative Model of Governance Determinants**

<!-- Start of picture text --> Other<br>Other<br>Governance<br>Factors<br>Factors<br>Determinants<br>Governance<br>Other Performance<br>Factors<br>Organizational Health<br>Performance Outcomes<br>Organization<br>Inputs Other<br>Factors<br><!-- End of picture text --> Source: Author‟s visual interpretation of the formal model presented in Reid 2009.

To illustrate this distinction, consider how governance of primary care health facilities affects health outcomes. One particular line of analysis could ask about governance _determinants_ that influence the performance of primary health care facilities. For example, we can ask whether public primary health care facilities owned by local governments and in which the local government can discipline workers is more or less successful than those in which workers can only be disciplined by higher levels of ______________________________________________________________________________________

-9-
government. Whether or not these governance mechanisms actually function would be a measure of governance performance, such as the share of workers who are absent from work without legitimate excuses. We can then ask whether the increased presence of staff has improved the facility‟s performance, say in the number and quality of consultations, and, in turn, whether this has contributed to better health outcomes in the local population (e.g. fewer individuals dying from acute respiratory infections). (See Figure 3.)

**Figure 3: An Illustration of Governance Determinants and Performance**

<!-- Start of picture text --> Local Labor<br>Government Market Adherence<br>Discretion<br>Disciplining<br>Training,<br>Staff for<br>Labor Laws<br>Absenteeism<br>Consul- Healthy<br>tations Individuals<br>Primary-Level<br>Health Unit<br>Staff Effective<br>Demand<br><!-- End of picture text --> <!-- Start of picture text --> Source: Author.<br><!-- End of picture text -->

# **_Complete and incomplete measures of governance_**

The AGI initiative notes that the most commonly available and best-elaborated measures are focused on service inputs, outputs and outcomes (Reid 2009). The initiative draws attention to what it terms the “missing middle,” that is, measures of governance determinants, governance performance, and the exogenous factors that condition how they work.

The search for governance determinants and performance measures follow substantially different logic. Governance performance measures can be identified by asking participants about problems that an organization faces in providing services. For example, when facilities pay higher than market price for supplies, it can restrict their ability to serve clients. Consequently, the ratio of supply prices paid relative to an average market price would be useful for assessing whether the governance of a facility is ______________________________________________________________________________________

-10-
performing efficiently. Similar performance measures are suggested by problems such as drug stock-outs and absenteeism.

By contrast, identifying measures of governance determinants requires statement of a hypothesis and underlying theory of behavior. For example, people have variously advocated for the advantages of centralized and decentralized procurement. Underlying theories give emphasis to economies of scale and efficient use of limited institutional capacity in the case of promoting centralization, and to responsiveness to local variation and accountability to local communities in the case of promoting decentralization. The degree of centralization in procurement can be measured, e.g. percentage of total procurement conducted at the facility level, but this measure has no normative or prescriptive content unless, at some point, a strong empirical link can be made between it and some outcome (hopefully both improved governance performance and organizational performance).

In this regard, governance determinants have more in common with input and output measures while governance performance measures are similar to outcomes. In particular, a high score for a governance determinant indicator is not necessarily good or bad. Like inputs, having more can be beneficial but it can also be wasteful. By contrast, outcomes and governance performance measures can generally be measured against a scale that is easily interpreted from worse to better. (See Box 1.) ______________________________________________________________________________________

-11-
# **Box 1: Is more always better?**

Health status is an outcome that can be measured against relatively objective standards of what is good and bad – rates of mortality, length of life, days without illness are outcome indicators that can be interpreted straightforwardly. Lower rates of mortality, longer life, fewer days without illness are generally better. Similarly, governance performance measures usually have straightforward interpretations: lower absenteeism rates are better than higher rates; smaller shares of counterfeit drugs in the market are better than larger shares.

By contrast health sector input and output measures cannot be interpreted so simply. The problem arises because the production function for health is not fixed. While in resource constrained situations (as in many developing countries) it is tempting to view more inputs as better, achieving the same health status for the population with _fewer_ inputs (e.g. less money, fewer workers) would actually be better because it would release resources for application to other important public, social or private uses. Only if we assume that the production function is fixed, e.g. we need a specific number of doctors per population, does the input measure become interpretable – approximating the ideal ratio is good, exceeding or not reaching it is bad.

For example, a health system would be more efficient if it were able to achieve the same health outcomes with fewer health professionals. Therefore, just having more doctors per population is not clearly better, outside of a particular context. There is no objective measure of the “right” ratio and with different approaches to training and management, this number could vary considerably. Similarly, holding performance constant, societies would probably prefer to spend _less_ on health rather than more.

Similarly, output measures demonstrate that a health system is functioning, but is a system better if it produces more health care services? The number of health care services provided could be excessive (e.g. unnecessary cesarean surgeries are common in many countries), the wrong services could be provided (e.g. treating viral infections with antibiotics), or avoidable services could be provided (e.g. amputation of the limbs of people with advanced diabetes when earlier interventions might have made this unnecessary).

Most governance determinant measures are similar to inputs and outputs with regard to interpretation. Knowing that a health care service is highly decentralized does not, in itself, tell us whether this is good or bad. In fact, governance determinants need to be treated more like hypotheses. They pose a question regarding whether or not the governance factor is associated with better performance. Interpretation, then, ultimately depends on evidence.

To be complete, governance determinants have to be characterized fully. Too frequently, hypotheses are framed in ways that lack coherence. For example, it does little good to ask about the impact of a new technology for information systems without considering who would use that information or about contracting private-sector health care providers without considering the public sector‟s capacities to supervise contracts.

______________________________________________________________________________________

-12-
Characterizing a governance determinant fully requires answering three questions. Who has authority over which decisions? How is information generated and used? And, what forms of motivation are in place? These three questions provide information about authority, information and motivation which can be defined in the health sector as:

- _Authority_ : The scope of decision-making and action by health sector actors related to allocations of funding, deployment and management of staff, mobilizing revenues, and publishing information among others.

- _Information_ : The generation by or transmission of information to actors who have an interest in and an ability to use it (e.g. providers, funding agencies, patients, civil society groups, suppliers).

- _Motivation_ : Extrinsic (e.g. financial incentives) and intrinsic (e.g. social status) factors that motivate actors to behave in ways that further or hinder their activities in delivering or supporting good quality health care and health system services.

These concepts fully characterize the notion of accountability in service delivery.<sup>5</sup> By defining authority, it is possible to delineate an actor‟s range of responsibilities (and simultaneously delineate those areas for which they lack discretion and cannot assume responsibility). The generation and use of information about that actor‟s performance indicates whether that actor‟s behavior is observable, to whom, and if it results in any consequences. Finally, the identification of what motivates the actor will provide insights into how they are likely to respond to potential consequences of their behavior. Most successful reforms address all three of these elements. (See Box 2.) The importance of this kind of full characterization – addressing authority, information and motivation – can be illustrated with reference to the common debate over the relative effectiveness of private and public health care provision. A simple approach of defining the governance determinant as the share of public funding that contracts private providers is only a very rough approximation of what the implicit theory of behavior would require. Those who advocate private-sector contracting would probably still agree that this approach is not necessarily going to succeed where contracts have ill-defined service requirements, where those who are contracting lack the capacity to negotiate contracts or monitor contract fulfillment, and where the threat of nonpayment or penalties is not credible. In other words, the hypothesis that private sector contracting will improve organizational performance is incomplete without specifying a large range of associated governance determinants which form a complete institutional whole.

> 5 For alternative but complementary treatments of accountability, see Kotalik and Rodriguez 2006, World Bank 2004, and Amin et al 2008.

______________________________________________________________________________________

-13-
# **Box 2: Authority, Information and Motivation Go Together**

A series of studies in Europe looked explicitly at the role of setting targets in national health policies within a scheme of good governance (Wismar et al 2006). Such targets “are viewed as a means of defining and setting priorities, creating high-level political and administrative commitment to particular outputs, and providing a basis for follow-up and evaluation.” (European Observatory 2006).

Targets can play an important role in defining, generating and collecting useful information for governing the health sector. However, without concomitant changes in authority and motivation, it is not clear what impact targets can have.

England‟s experience with setting health service targets in the mid-1990s was combined with changes in authority and motivation. For one thing, the National Health Service‟s targets were defined in terms of outcomes rather than operational activities. In addition, these targets were introduced with a range of associated financial incentives (motivation) and increased autonomy (authority). Performance on targeted measures of care improved in England compared to Scotland and Wales where such policies were not implemented and even though spending levels are lower in England.

The case study in Hungary demonstrated how target-setting has been accompanied by improved data collection over a 10-year period, including the ability to disaggregate health status data by important social and geographical categories. An important advance in this process came with the coordination of data collection between the country‟s national statistical office and its national public health system. While data collection and reporting continued to improve, it was not possible to demonstrate any impact on health policy, health service performance, or health outcomes. While the use of such data in national policy debates may eventually have some impact, this case suggests that efforts to improve information without accompanying changes in authority and motivation may not bear the desired results.

Source: European Observatory 2006.

In light of this definition of completeness for governance determinants, the share of private sector contracting is not a very complete measure. It can still be useful as a proxy for efforts to incorporate the private sector in publicly-financed service provision, but it is only indicative of what the underlying theory suggests. Rather, a series of questions would need to be answered regarding the authority of those who do the contracting (e.g. what discretion they have to issue contracts, what rules circumscribe their discretion); the information that is collected and transmitted (e.g. does the contractor provide administrative information on services provided or does an independent agent verify outputs?); and the motivation for fulfilling contractual obligations (e.g. what are the consequences of success or failure in delivering contracted services?).

Using the concepts of authority, information and motivation is useful for developing measures of governance determinants in several ways. First, if a variable does not have some bearing on authority, information, or motivation, it is not likely to tell us much ______________________________________________________________________________________

-14-
about governance. Secondly, the concepts provide an important qualification for interpreting studies of broad governance indicators (e.g. decentralization) by insisting that such measures are incompletely characterized. Third, they suggest ways in which broad governance measures can be refined and specified.

In sum, this paper defines governance as the combination of political, social, economic and institutional factors that affect the behavior of organizations and individuals and influence their performance. It explicitly requires the identification of which organizations or individuals are the units of analysis and judges governance determinants against their impact. Fully characterizing any given governance determinant requires specifying the distribution of authority, the generation and use of information, and the ways actors are motivated. The remainder of this paper applies these concepts to developing an agenda of measurable governance indicators in the health sector.

# **3. Why Measure Governance?**

Before discussing how to measure governance determinants and performance, it is necessary to ask about the purpose of such measurements. The answer to this question is critical to a number of strategic decisions, most importantly about the degree of priority given to finding indicators that are comparable across countries and the appropriate focus of such indicators in terms of the unit of analysis.

# **_The purpose of measuring governance_**

Throughout the literature, authors propose measuring governance with the aim of improving it. However, the path by which they believe measurement will lead to improved governance varies significantly. Those who think our understanding of good governance is sufficiently advanced tend to propose measuring governance in order to determine which countries have adopted the appropriate practices and to advocate for implementation of good governance mechanisms where they are lacking. Those who are skeptical of current expert views regarding good governance tend to propose measuring governance as part of a strategy for learning which governance mechanisms are effective or at least robust across different contexts. It is the author‟s judgment that this latter view more accurately reflects the current evidence base for governance in the health sector and, therefore, that the appropriate measurement strategy should lean in the direction of learning which governance arrangements are most effective.

# **_Comparing within or between countries?_**

Following this line of reasoning questions about governance determinants and performance are necessarily relative. They are not so much about what works as they are about what works better. In this regard, it is debatable whether the most fruitful lines of research will involve comparing different forms of governance within countries or between countries. Within-country research benefits from two very important advantages. First, differences in context are less likely to confound findings. Second, policy recommendations are more likely to be relevant and embedded in institutions that are comparable to the studied cases. For example, comparing different forms of hospital ______________________________________________________________________________________

-15-
governance within a large state in Brazil has provided evidence useful for policy toward hospital governance throughout that country in ways that are more relevant than if the findings came from comparisons between different countries (see La Forgia and Couttolenc 2008).<sup>6</sup> Comparisons between countries are also useful, but require much more careful consideration of the package of institutional and contextual factors that are likely to support any given governance arrangement. For example, the independence of health insurance institutions can improve performance when it limits undue political interference and can worsen performance when it insulates the institution from accountability for results – the implications of this kind of independence are contingent on interactions with other factors which may vary significantly across countries (Savedoff and Gottret 2008).

Cross-country indicators are easier to identify for governance performance than for governance determinants because the former are more easily defined against an objective standard. For example, an objective standard for unexcused absences might be “below 3%” but a similar objective standard cannot be defined for the degree of decision-making autonomy for health facilities. A recent effort to select health governance indicators (Lewis and Pettersson 2009) implicitly confirms this observation by mostly citing crosscountry studies when proposing performance measures such as informal payments (Lewis 2006) and absenteeism (Chaudhury et al 2006), but citing case studies when discussing determinants such as community monitoring (Bjorkman and Svensson 2007), performance pay (Solon et al 2009), and autonomy (PATHS 2008). The evidence base for assessing governance determinants relies almost entirely on within-country comparisons – either between different governance arrangements or assessing changes over time that can be attributed to a particular governance reform.

# **_Choosing units of analysis_**

Regardless of whether we are looking at within or between country governance measures, it is necessary to identify which units of analysis will be addressed. Health sectors vary so much around the world that it is difficult to make comparisons, let alone conduct research, without defining the organization or system that is being analyzed.

Countries are the unit of analysis in many initiatives, including the Governance Development Initiative and WHO‟s _2000 World Health Report_ . Interpreting differences between countries, however, is complicated by the myriad institutional arrangements that characterize their health systems. Leading approaches for classifying countries may distinguish their health systems _functionally_ – separately analyzing stewardship, resource generation, financing, and provision (WHO 2000) – or in terms of _policy instruments_ – such as health system financing, provider payment, organization, regulation, and social marketing (Roberts et al 2004). Others try to classify them by _institutional arrangements_ , for example, distinguishing those with social health insurance from those with national health services. Another approach is to focus on _roles and accountability_ , identifying actors and how they interact with one another (World Bank 2004, Brinkerhoff and

> 6 Savedoff 1998 takes a similar approach, using within-country institutional comparisons.

______________________________________________________________________________________

-16-
Bossert 2008). The latter approach is more consistent with our definition of governance. In fact, by focusing on roles and accountability, it is possible to move away from discussing aggregate systems and see that distinct parts of the health system are governed in significantly different ways, particularly between those that rely principally on political and market processes (Emmanuel and Emmanuel 1996).

Questions about governance, then, can be raised in any part of the health sector. Among health _system_ services, like assuring the quality of health care provision or addressing environmental health risks, some countries rely more on government agencies, while others rely more upon professional associations or private rating agencies. For health _care_ services, like hospital care, some countries rely primarily on government-owned and operated facilities, while others rely on private non-profit organizations, private for-profit firms, or hybrids (e.g. government-owned but privately managed facilities).

Choosing an appropriate unit of analysis is critical for interpreting measurements. First, interpretation will be facilitated by selecting units for which we have behavioral models or hypotheses. This suggests either choosing organizations that have coherent decisionmaking structures such as health facilities, insurance firms, and public agencies or choosing segments of the health sector that can be characterized as markets (e.g. private health care providers).

Relevance is another criterion for selecting units of analysis. An initiative to measure governance across countries could give greater weight to those health care services used by more people – in some countries this would imply giving greater emphasis to public providers while in other countries the emphasis would go to private providers. This would be an appropriate choice if the goal is to measure how overall population care is affected by national policy. An alternative approach might aim instead at understanding how a particular governance determinant affects performance, in which case, the share of provision is less relevant than the amount of knowledge that can be learned from analyzing that particular organization or segment of the health sector.

It is beyond this paper to propose governance measurements for all potential units of analysis. Therefore, the remainder of this note focuses on governance of direct public <u>provision of health care services because the experience of seeking governance indicators</u> for this segment is more extensive than any other part of the health system. This is not meant to imply that it is either the most important or even the central feature of a health care system. In fact, in many countries, direct public provision is quite marginal relative to social insurance plans or private providers. Similar exercises could be undertaken for the governance of public health insurance organizations (Savedoff and Gottret 2008), non-governmental or commercial health care providers, and public health regulation (e.g. of medical and pharmaceutical supplies, medical education).

# **4. What to Measure?**

A complete data collection effort on governance in the health sector requires gathering a range of standard information in addition to specific governance measures. Data on health service inputs and demographics are necessary for judging the constraints and ______________________________________________________________________________________

-17-
demands on health services. Data on health service outputs – measuring organizational performance – and health outcomes are needed to assess the effectiveness of the services, including their governance arrangements. Rather than reviewing the extensive work on inputs, demographics, outputs and outcomes that are addressed elsewhere, the rest of this note focuses on data that is specific to understanding governance, namely measures of (1) governance performance, (2) governance determinants, and (3) governance context.

# **_Measuring governance performance_**

Work on governance performance measures in health is much further advanced than on governance determinants. This is due, in large part, to the fact that governance performance can be measured against reasonably clear standards. Most governance performance indicators in public health care provision involve measuring the gap between expected and actual behaviors such as unexcused absences from work or diversion of funds from intended uses.

WHO 2008 provides a good list of governance performance measures for direct public health care provision:<sup>7</sup>

- Health worker absenteeism in public health facilities

- Proportion of government funds reaching district-level facilities

- Stock-out rates for essential drugs in health facilities

- Proportion of informal payments within the public health care system

- Proportion of pharmaceutical sales that consist of counterfeit drugs

These five indicators measure the gap between expected and actual performance of the public health care services and highlight critical aspects – human resource management, financial management, supply management, drug regulations and enforcement – that affect the outputs and impact of public health care services.

Another recent study, Lewis and Pettersson (2009), presents a longer and more comprehensive list of governance performance measures (See Table 1). For example, they not only propose to measure budget leakages, but also suggest separately reporting payroll leakages and in-kind leakages from financial leakages. Similarly, in addition to tracking absenteeism, they recommend collecting information on job purchasing and health worker performance.

> 7 A sixth measure is also proposed as a governance performance measure, namely, voice and accountability. As discussed earlier, voice and accountability are treated here as governance _determinants_ and not as governance _performance_ indicators.

______________________________________________________________________________________

-18-
**Table 1: Proposed** **<u>Governance Indicators for Health in Lewis and Pettersson (2009)</u>**

Budget and resource management _PEFA Indicators track budget credibility, comprehensiveness, transparency, execution, reporting, recording, and external audits and scrutiny_ Discrepancy between public budgeted health funds and the amounts received by health <u>providers</u> Irregularities associated with government payroll for health workers Differences in prices paid for similar medical supplies/equipment across health facilities Human resources Frequency of illegal side payments/bribes influencing hiring decisions and of payments for particular assignments _Existence of licensing requirements and of continuing educations programs and their operation_ Fraction of physicians or nurses contracted for service but not in site during the period(s) of observation _Types of incentives and accountability mechanisms facing_ _<u>public providers</u>_ Institutional providers _Incentives and accountabilities in hospital payments_ Average length of stay and bed occupancy rates Informal payments Frequency of illegal charges for publicly <u>provided health services</u> Perceptions of corruption and institutional quality Fraction of households, experts or public officials perceiving corruption in health; relative ranking of health sector on corruption index The Country and Policy Institutional Assessments (CIPA) for health Note: Governance determinants are italicized. Other items are governance performance measures. Source: Adapted from Tables 1 and 2 in Lewis and Pettersson 2009.

The list of five governance performance indicators proposed by WHO 2008 is a good starting point because it is parsimonious yet covers the full range of important public health care service functions (financial management, personnel management, drug management, client relations, and regulation). In addition, these indicators are attractive because they can be measured quantitatively and objectively and can be interpreted consistently. If sufficient resources were available to expand beyond these five governance indicators, then items from Lewis and Pettersson 2009 could be considered as additional candidates.

Most of these indicators, with some refinement, could be collected periodically to assess changes over time and across countries in ways that permit meaningful comparisons. Four of the indicators can be measured by cross-sectional surveys, the one exception ______________________________________________________________________________________

-19-
being budget leakages which also require calculating how much money was _supposed_ to reach a given health facility.<sup>8</sup> A country with high marks on all five of these governance performance measures has a functioning public health care service, while a country with low marks does not. Thus, independent of any effort to gather information on governance determinants across countries, these five governance performance measures could be used to benchmark the performance of public health care provision across countries and identify areas that are problematic. If worker absenteeism is high, then the personnel management system needs to be investigated. If counterfeit drugs are common, drug regulations and enforcement need analysis.

Additional governance performance measures that appear in different studies and which could also be considered include:

- share of births and deaths that are registered (HSM )

- disparities in health care service coverage between high and low income groups (HSM)

- share of health facilities that receive supervisory visits in accordance with national guidelines (HSM)

- share of workers who complete programmed training (GFATM)

- share of health facilities that are staffed according to national standards (GFATM, Tanzania)

- share of budgeted funds that are executed (PEFA, GFATM)

- share of positions that are vacant (GFATM SWEF)

- share of audited objections settled within expected time frame (Bangladesh)

- share of contracts awarded within initial bid validity period (Bangladesh)

- share of nationally distributed commodities that reach local facilities (Bangladesh)

- share of local governments submitting health information reports on time (Ethiopia, HSDP II, Mozambique and Tanzania)

- share of donor agency funds that are disbursed through the federal Ministry of Health without earmarking (Ethiopia, HSDPII and Ghana)

- share of maternal deaths that are audited (Ghana and Nicaragua)

- share of population aware of their rights to public health care services (Kyrgyz Republic)

- share of providers with satisfactory compliance of management agreements (Nicaragua)

- share of providers complying with medical protocols (WHO and IHME)

- share of health workers who leave service by cause (Mozambique)

# **_Measuring governance determinants_**

While the governance performance measures are useful for answering a range of questions, the suggested governance determinants address very specific approaches to governance. Each of the governance performance measures tells something about the

> 8 Surveys can reliably estimate the resources available at health care facilities, but it can be quite difficult to estimate the amount of resources that _should_ be at a given health care facility, particularly in countries with federated structures or in which facilities receive resources from a wide range of programs or funders.

______________________________________________________________________________________

-20-
extent to which public health care providers perform according to expected standards. By contrast, the governance determinants address one particular policy instrument.

For example, while adopting an essential medicines list may be a good health sector policy – because it is useful for guiding clinical practice, public procurement, or distribution – the existence of such a list does not, in itself, demonstrate much about authority, information, and motivation in the health sector. A more general governance indicator would document whether or not the public system has an effective policy regarding which medications are purchased, distributed and prescribed. This can be done by specifying who has authority to make such decisions (e.g. a single national procurement agency, district officers, health facilities), the capacities and discretionary powers of those in authority (e.g. through regulations, professional codes), who provides the information that is used to make decisions (e.g. pharmaceutical companies, public drug agencies), and what factors motivate decisions (e.g. profit, cost-effectiveness). Operationalizing this may be more useful but also requires greater effort. For example, WHO‟s Governance and Pharmaceutical Assessment Tool uses 66 separate questions to judge whether the system of purchasing, distributing and dispensing medications is procedurally effective.<sup>9</sup> All but six of the indicators presented in WHO 2008 are efforts to measure governance determinants. They include:

- Existence of a national health strategy

- Existence of an essential medicines list

- Existence of policies on drug procurement

- Existence of a national strategic plan for TB

- Existence of a national malaria strategy/policy

- Completion of the UNGASS questionnaire for HIV/AIDS

- Existence of a comprehensive reproductive health policy

- Existence of a comprehensive childhood immunization plan

- Existence and dissemination of key health sector documents

These proposed indicators for governance determinants are defensible: they are relatively easy to verify and measure with expert interviews, they represent an important international initiative to improve health sector governance, and they do describe a coherent set of procedures for changing authority and information. On the other hand, they achieve this at the expense of missing alternative procedures aimed at the same ends, e.g., the essential medicines list is only one way of establishing procedures for drug selection, and have not been tested empirically. I would contend that expert opinions like this should be treated as hypotheses to be tested rather than evidence to be followed.

Lewis and Pettersson (2009) include relatively few governance determinants among their proposed list of governance indicators. These include the procedural measures tracked by Public Expenditure Financing Assessments (PEFA) and the existence of licensing

> 9 See WHO‟s Good Governance in Medicines initiative at: <u>http://www.who.int/medicines/ggm/en/index.html</u>

______________________________________________________________________________________ -21-
systems for health care professionals. The other two governance determinants that they mention are payment and accountability mechanisms for health care professionals and institutional health care providers – determinants that have demonstrated effects in the health care literature but which have never been measured on a system-wide basis (e.g., What share of payments to doctors are paid as wages, fee-for-service and capitations?).

As noted earlier, choosing which governance determinants to measure requires beginning with a set of hypotheses about a particular organization and so we will continue to focus on direct public health care provision. Reviewing WHO (2008), Lewis and Pettersson (2009) and other literature on governance of public health care provision in developing countries reveals at least five broad categories of hypotheses related to (1) ownership, (2) decentralization, (3) formal procedures, (4) stakeholder participation, and (5) contextual factors. From these governance determinants, it is then possible to derive a number of associated governance performance measures.

# _Ownership_

Ownership is a governance factor that has been analyzed extensively in the corporate governance literature and has entered health sector discussions primarily in debating the relative merits of private and public forms of health care service provision. Here we are considering its role within publicly financed health care services. When the public sector contracts and manages private health care services, the contractual relationship generally divides authority between responsibilities of the payer and payee, specifies information and how it will be used, and includes explicit incentives for fulfilling contract terms. By contrast, when the public sector directly manages its own health care services, public regulations and procedures may or may not explicitly establish clear lines of authority, information requirements and motivations for performance.

This characterization is, however, merely a hypothesis. It is reasonable to assume that a private health care provider whose capital is at risk will respond to financial incentives or competitors by raising productivity and quality, but providers may have a number of other ways to protect their incomes or not respond to incentives at all. Thus, we have reasons to expect that different forms of ownership will affect health care service providers‟ behaviors but still need to test the question empirically.

A first step to analyzing ownership as a governance determinant in public health care provision would be to measure:

- the share of public health care service expenditures used to contract private health care providers, and

- the share of publicly-financed consultations provided by private health care providers.

These measures would give a rough sense of the scale of public contracting in a given public health care service and could be compared with other systems to assess their performance. It is also necessary to distinguish between private health care providers that operate commercially, for-profit, and those which operate as non-profit institutions ______________________________________________________________________________________

-22-
because empirical evidence suggests they behave differently under similar circumstances.<sup>10</sup> Interpreting any measured associations between the share of private contracting and the performance of publicly-financed health care services would require more precise information on how this contracting is implemented. To do this, a more elaborate index would be necessary, identifying a series of questions related to:

- specificity of contracts (in terms of service definitions and payment conditions),

- the existence of specialized public sector staff to supervise contracts,

- functioning information systems to independently assess contractual compliance, and

- application of sanctions in the event that contracts are not followed.

Additional measures that have been used in the literature related to ownership and which might be useful to consider include:

- share of publicly-financed hospital beds contracted out to NGOs (Bossert and Beauvais 2002),

- share of publicly-financed inpatient bed-days in NGO hospitals (Bossert and Beauvais 2002),

- expansion of hospital autonomy (Bangladesh)

# _Decentralization_

Decentralization has also been analyzed extensively in the literature on public administration and health sector policy in developing countries. Many studies argue that decentralization can improve health care service performance by allowing health care providers to make decisions that are appropriate to local variations in health conditions, service demands, and resource availability. Other studies argue that decentralization makes public health care services accountable to local stakeholders, citizens, and patients in ways that can improve health service performance.

While these effects have been demonstrated in some cases, other examples show the advantages of centralization. Centralized systems can produce more services by taking advantage of scale economies in purchasing, better utilizing scarce managerial skills, and optimizing resource allocation. Centralized systems can sometimes be more efficient and equitable by allowing certain locations to specialize in particular services, by pooling risks and by smoothing out resource consumption across different locations.

One of the better treatments of this subject can be found in Bossert and Beauvais (2002) who characterize the degree of decentralization by examining the amount of “decision space” accorded to local and central actors. The authors map decision space according to whether the “range of choice” for local decision-makers is viewed as narrow, moderate or wide, assessing the degree to which ministries of health and education prescribe how services are to be delivered. Bossert and Beauvais then survey indicators of finance,

> 10 The literature on this difference is most extensive in the United States. However, studies comparing the US and other wealthy countries (e.g. Mobley and Magnusson 1998) and a systematic review that includes developing countries (Hollingsworth 2003) demonstrate that the hypothesis of a difference between forprofit and non-profit organizations should be tested.

______________________________________________________________________________________

-23-
service organization, human resources, access rules, and governing structures seeking correlation between decision space and social service outcomes.

A first step to analyzing decentralization as a governance determinant in public health care provision would be to measure:

- share of public health care expenditures made by local governmental authorities

- share of public health workers hired by local governmental authorities

These measures would approximate the level of decentralization in public health care provision, but would be difficult to use for cross-country comparisons since countries vary significantly in size and in the subnational structures of public administration. Furthermore, these broad measures do not make it possible to distinguish the degree of decentralization which may vary across different health service functions. For example, a public health service might decentralize personnel management while retaining highly centralized drug procurement and distribution systems.

Interpreting any measured associations between the share of local government responsibility for public health care provision and performance would require data that distinguishes different aspects of health care provision, such as share of supplies purchased centrally and share of drugs purchased centrally; more refined measures of the scope of decision-making authority by local offices, such as the share of local government expenditures financed by local tax revenues; and contextual information such as the average and range of population sizes served by local government unit.

Additional measures that have been used to assess decentralization which could also be considered include:

- share of user fees retained by facility or local government unit,

- local authority to determine user fees,

- local authority to hire and fire workers,

- share of funds transferred to local health systems and autonomous regions (Mozambique and Nicaragua)

# _Formal procedures_

Formal procedures have been the focus of many efforts to improve public services in developing countries, with the goal of rationalizing public sector policy- and decisionmaking. The presumption behind this work is that the existence of formal procedures can improve public service provision by clarifying expectations and tasks assigned to different staff and units; by keeping those with vested interests from unduly influencing decisions; and by increasing accountability to higher-level authorities and citizens.

Assessing whether or not formal procedures are in place makes up a large part of the governance literature, outside as well as within the health sector. For example, the Public Expenditure Framework Assessments (PEFA) contain numerous questions related to the existence of different procedural arrangements (e.g. separation of authorization and expenditure functions, provisions for selecting procurement committees). As noted above, most of the indicators in WHO‟s draft proposal for monitoring health sector governance are such rules-based measures.

______________________________________________________________________________________

-24-
The most common critique of assessing formal procedures is that they may exist on paper but not be implemented – a situation that would be visible as a gap between the formal requirements (a governance determinant) and the practice (a governance performance measure). Formal procedures can also be followed in ways that allow them to be circumvented by other means. Again, this can be revealed by measuring the gap between the governance determinant and performance measures. But fundamentally, unless such procedures have a demonstrated and robust effect on performance, they must be treated as hypotheses. For example, competitive bidding may be a very effective means of reducing corruption in situations where single-sourcing is part of a systematic network of kickbacks, or it might simply reshape the network of corruption in other ways. The effectiveness of competitive bidding as a formal procedure is a question to be investigated.

A first step to analyzing formal procedures as a governance determinant in public health care provision would be to measure:

- the existence of formal budget planning and execution policies,

- the existence of formal human resource management policies, and

- the existence of formal drug procurement policies.

Just knowing that such policies exist is not likely to be of much use without further specification. Fortunately, a number of initiatives have already elaborated tools for assessing formal procedures that rely on indices composed of a large number of underlying questions. An example of such an assessment tool has been developed for governance of pharmaceuticals (WHO 2006) and could be used to summarize the quality of formal procedures in drug regulation, procurement, distribution and dispensing.

Similarly, the PEFA has a large number of questions that can be used for assessing financial management in public health care services, either using the PEFA exercise itself in those cases where public health care services follow the same procedures as other governmental agencies or adapting the PEFA questions to the public health care service financial management system when it is handled differently.

Guidance for assessing human resource management in the health sector was published by WHO in 2007 (Bossert et al 2007). It provides a framework for assessing the state of human resources in health, identifying problems, and considering the available policy levers. The report not only proposes a series of governance determinants and benchmarks, but also outlines methods for collecting, reviewing and reporting on findings.

Other assessment exercises have proposed measures of formal procedures such as:

- Share of facilities with user fee guidelines and exemption procedures (Peters et al 2007)

- Share of authorized providers with annual performance agreements (World Bank 2006)

- Implementation of specific health strategies (for preventing non-communicable diseases, emergency health response, etc.)

- Existence of a regulatory framework for non-public providers

______________________________________________________________________________________

-25-
- Existence of an accreditation system for all health care providers

- Existence of a Performance Monitoring Agency for supervising contracts

- Medium Term Expenditure Framework

- regulatory framework for pharmaceuticals

- improved planning and budgetary procedures

- performance audits linking financing to performance

- performance-linked staff incentives system in place

- Share of cooperating agencies using a single set of procurement procedures

- Human resource strategic plan operationalized (Zambia)

- Existence of Monitoring and Evaluation plan for human resources (Zambia)

- Provincial Human Resource Development Committee meets quarterly (Zambia)

- Procurement function is defined

- drug supply budget line policy implemented

- capital investment plan approved with funding

- national malaria control strategic plan operationalized

- Health care financing policy operationalized

# _Stakeholder participation_

Stakeholder participation is considered to be a critical factor governing the delivery of health care services. In many public health care services, the primary channels of accountability are hierarchical – from the public providers to higher levels of public authority. But public health care services are also accountable to citizens indirectly through the mechanisms by which political authorities are elected or appointed. This “long route of accountability” (World Bank 2004) can be problematic and many countries have experimented with policies that involve interested parties more directly in the management or supervision of public health care services.

Studies have argued that creating channels for stakeholder participation in managing or supervising public health care services can improve performance in several ways. Giving citizens a role in supervising public health care services or soliciting their feedback on service quality can provide useful information and apply pressure to improve public health care service quality and productivity. Creating opportunities for unions to participate in planning and check arbitrary management decisions can provide improve the quality of management and boost staff morale. Opening policy decisions to professional associations can safeguard the medical quality of care and protect health professional interests.

A variety of health sector policy documents have advocated for greater stakeholder participation as a worthwhile element of health sector governance in and of itself. In such cases, greater stakeholder participation would be considered to be better governance regardless of the impact on health care service performance. This is different from the approach presented here which would empirically test whether participation improves or worsens performance and assess the value as a governance determinant accordingly.

In fact, many studies argue that greater stakeholder participation can harm public health care service provision. As demonstrated in the literature on public administration, ______________________________________________________________________________________

-26-
opening public agencies to stakeholder participation can distort decisions in favor of particular groups. Citizen representatives can influence decisions to their own benefit in ways that may be detrimental to others in the community. Similarly, union representatives might defend workers against legitimate disciplinary actions or lobby for benefits to their members even when they reduce the quality or quantity of available care. Professional associations can use their involvement in policy decisions to restrict the number of practitioners and boost incomes even when this might not be in the social interest.

A first step to analyzing stakeholder participation as a governance determinant in public health care provision would be to measure:

- share of hiring decision bodies that include union representatives,

- share of facilities with citizen representatives on governing or advisory boards, and

- share of planning boards with representatives of planning associations.

These measures would give an indication of whether these three key stakeholders play a significant role in public health care provision. Nevertheless, they are only approximations and might miss a large number of alternative ways that stakeholders can participate. For example, unions might not have a formal role in hiring decisions, but they may exert public pressure effectively through strikes or political lobbying; citizens might be regularly interviewed or surveyed for feedback on service quality; or professional associations might not need formal representation to influence policy decisions. Therefore, a fuller characterization of stakeholder participation might require measurements like:

- dissemination of public expenditure data to the public,

- public advertising of jobs, selection criteria and outcomes of hiring decisions,

- share of representative bodies that file minutes, and

- numbers of meetings per year of representative bodies.

Additional indicators of stakeholder participation that have been proposed in other studies and that could be considered include:

- Active hospital health boards as indicated by number of meetings

- Participation in local health organizing committees

- Institutional arrangements for community and stakeholder participation

- number of regional coordinating meetings that are held each year (Ethiopia)

# **_Measuring the context for governance_**

Contextual Factors influence public health care provision in many ways, interacting with the governance determinants described above. Competition from private health care providers can sometimes motivate public health care providers to improve their services. Higher wages in the labor market, and specifically for health care workers, can make it difficult to recruit and retain qualified staff in the public sector. Furthermore, the scale and quality of professional training for health care workers can be an important constraint to public health care services and the general quality of public service institutions, whether courts, police, or public auditors, can influence the performance of public health care institutions.

______________________________________________________________________________________

-27-
Some of these may seem self evident, but they also require empirical testing. For example, competition might not influence public health care providers unless their budgets or incomes are affected by the number of services they provide; and market wages might have limited influence on the ability to recruit qualified staff if non-financial motivation for joining the public health care service is strong.

As a first step to analyzing governance of public health care provision, it would be useful to control for:

- share of consultations by public and private health care providers

- ratios of public nurses‟ and doctors‟ average monthly income relative to high school and college graduates

- number of annual graduates from nursing and medical schools relative to public health care service vacancies

- World Bank measures of governance of public institutions

More detailed information might be necessary. For example:

- average number of private health care providers operating within 10 km of each public health care facility

- ratios of health workers incomes to specific alternative professions (e.g. teaching, law) or statistically controlled analyses of wages relative to the larger labor market

- country governance indicators that are directly relevant to the health sector (e.g. the CPIA health sector component).

Additional contextual factors that have been measured and used in other studies include sociodemographic data (e.g. female literacy rates, income distribution, ethnic and religious fragmentation), labor market indicators (e.g. unemployment rates among high school graduates), and quality of public institutions (e.g. the World Bank‟s CPIA).

# **5. Ways to Measure Governance in the Health Sector**

This section addresses practical issues for measuring governance in the health sector. It describes leading approaches to measuring the five governance performance measures identified above. It concludes with a discussion of strategies for measuring governance determinants. The appendix provides links to websites for a number of survey instruments and measurement initiatives.

# **_Governance performance_**

The strategy for measuring governance performance measures in ways that are comparable across countries is reasonably clear. This section describes how to measure governance performance with a focus on the five indicators identified by WHO‟s task force, namely:

- Health worker absenteeism in public health facilities

- Proportion of government funds reaching district-level facilities

- Stock-out rates for essential drugs in health facilities

- Proportion of informal payments within the public health care system

______________________________________________________________________________________

-28-
#  Proportion of pharmaceutical sales that consist of counterfeit drugs

One way to calculate these indicators would be to use administrative data on local health facilities that is known to be reliable. This is not the case in most countries and, therefore, a more fruitful approach is to use representative facility surveys. In order to be most useful to public policymakers, these surveys should ideally be expanded beyond the more common exclusive focus on public health care services to include surveys of all health care providers in proportion to their share of clients served.

Fortunately, work on health facility surveys has advanced significantly in recent decades. Two particular initiatives that deserve mention are the Public Expenditure Tracking Surveys (PETS) and Quality of Service Delivery Surveys (QSDS). Together, they can be used to measure three of the five core health governance performance indicators (absenteeism, funds reaching facilities, and stock-out rates).The proportion of informal payments can be calculated from questions asked in household surveys, but more reliable measures can be extracted from patients who have just left a facility. Detecting counterfeit medicines, by contrast, requires special surveys that may include visits to border crossings, warehouses, and pharmacies, as well as health facilities.

_Share of funds reaching facilities and Public Expenditure Tracking Surveys (PETS)_ For measuring the share of funds reaching health facilities, the Public Expenditure Tracking Survey represents the best model. Public Expenditure Tracking Surveys (PETS) follow the flow of government resources through the often-complex system of delivering public services. These studies commonly gather information about the funds that are budgeted, authorized and disbursed through different levels of the public service system and then survey a representative sample of facilities to determine how much of those funds are actually available in the form of services. When these studies are successful, they can measure the share of funds that fail to reach their intended goal and identify how much is lost at different points in the system. This information is useful for focusing investigations of diverted funds but more often is used to improve public policies (e.g. reallocating resources), financial management (e.g. monitoring, reporting and controls) and administration (e.g. simplifying procedures) (Reinikka & Svensson 2006). In countries with sophisticated and reliable financial administration, the share of funds reaching facilities can be extracted from public sector data. However, such systems are lacking in most countries, leaving the PETS approach as the only viable method.

More than two dozen PETS have been conducted in a range of low- and middle-income countries and the experiences demonstrate that the instrument is useful but difficult to compare across countries, particularly in the health sector (Lindelow 2006). PETS differ from country to country precisely because they are implemented with different policy concerns and because the traceability of resource flows differs in each setting. PETS tend to be most useful where they have been repeated in the same country and can be used to monitor the effects of policy initiatives over time (Dehn 2003). For those interested in conducting a PETS, the 2004 PETS/QSDS instrument used in Chad (that was largely based on the Mozambique 2002 PETS/QSDS) is a good template to begin with.

______________________________________________________________________________________

-29-
_Absenteeism, stock-outs, informal payments and Quantitative Service Delivery Surveys (QSDS)_ Absenteeism, stock-out rates and informal payments can be measured by studies that rely on detailed facility surveys. The Quantitative Service Delivery Survey (QSDS) represents such an approach. A QSDS examines the performance of service providers by collecting data on finances, inputs, outputs, pricing, quality, and supervision from a representative sample of facilities, often combined with interviews of staff and patients.

A QSDS provides a more thorough picture of how money and resources are utilized at the service provision level than do most PETS. The QSDS approach permitted Chaudhury and Hammer (2004) to estimate absenteeism through random visits of health care facilities in multiple locations in Bangladesh over a period of months. To the extent that local health care provision is similar across countries, the measures of absenteeism, stock-outs and informal payments from the QSDS could provide reasonably comparable estimates of governance performance.

The cost of implementing a combined PETS and QSDS is approximately US$200,000.<sup>11</sup> This figure includes a survey – applied in a random representative sample of health facilities – along with a review of documentation and interviews to track the flow of funds through the public health care system.

_Issues in measuring health care worker absenteeism_ Accurate measurement of absenteeism that is comparable across countries requires attention to a number of factors. The basic facility sampling strategy should assure that the survey is representative of the chosen type of facility (e.g. all public primary health care facilities). Unlike normal facility surveys, gathering information on absenteeism also requires that the timing of the facility visit be unannounced. Finally, appropriate information has to be gathered to distinguish excused from unexcused absences. As a governance performance measure, only unexcused absences are of interest. This means it is necessary to gather enough information about facilities to know which staff members are actually supposed to be present and determine, for those who are not available, whether they have legitimate reasons for being absent – such as authorized vacations, offsite assignments, or training courses. Follow-up visits are recommended to determine the reliability of the survey.

One six-country study has shown that international comparisons of absenteeism rates is possible given a rigorous study design and standardized questionnaire (Chaudhury et al 2006). The costs for absenteeism studies by themselves without a complete QSDS are closer to US$150,000 per country.

_Issues in measuring stock-out rates_ Several approaches are available to measure stock-out rates (the absence of particular medicines from a point of health care provision). Examples include a study by Reinikka

> 11 Cost estimates are based on conversations with World Bank staff who conducted or supervised studies.

______________________________________________________________________________________

-30-
and Svensson (2006) that measured stock-out rates in combination with the PETS/QSDS in Uganda, and another study in Kenya that followed the PETS methodology and reported on stock-outs at health care facilities (reviewed in Gauthier 2006). An African initiative of civil society organizations to monitor and pressure governments to reduce stock-outs can be found online (www.stopstockouts.org). This initiative includes periodic visits to facilities in Kenya, Uganda, Zambia and Zimbabwe to determine whether essential medicines are available.

A joint effort by WHO and Health Action International (HAI) in 2008 created a survey tool that is the best currently available for measuring the prices and availability of drugs. The survey tool gathers information on prices paid for key medicines, as well as differences between brand and generic prices, differences between national and international reference prices, the availability of drugs at facilities and pharmacies, and the variation of prices and availability across sectors and regions within countries. The tool is specifically designed to allow cross-country comparisons and has been conducted in more than 55 countries.

In the survey, data are collected on the availability of a selected group of medicines from a sample of outlets in the public and private sectors (both for- and non-profit) in six regions of a country or – in the case of large countries – of a state or province. Up to 50 medicines are chosen, generally comprising: 14 global core medicines; 16 regional core medicines; and 20 supplementary medicines. The global and regional core medicines lists are part of the standard methodology provided by WHO and HAI and are chosen to assist countries in making international comparisons. Price data can be collected quite readily in a randomized representative survey of sites at which medicines are sold. However, measuring stock-out rates is only possible where good stock records exist – which is somewhat more common in the public sector than in the private sector, according to HAI. Applying the survey as designed by HAI and WHO typically costs between US$10,000 and US$20,000.<sup>12</sup>

# _Issues in measuring informal payments_

While informal payments vary significantly across countries, it is still feasible to measure them whenever surveys can identify a clear gap between what patients pay for public health services and what they are legally required to pay. Measuring informal payments poses special challenges in that patients and households are often uncertain about how much they paid for health care services and are often even less knowledgeable about how much they are legally obligated to pay. Different institutional, legal and cultural contexts also create significant variation across countries that can confound cross-country comparisons (Lewis 2000; Gaal et al 2006).

Generally, patients and households are considered to be better sources for data on informal payments. In particular, exit surveys at facilities are best suited to getting reliable answers from patients as to how much they paid for health care services in a setting where the researcher can compare what the respondent paid to an official fee schedule for the services provided. By comparison, questions in household surveys

> 12 Author‟s estimate based on interviews.

______________________________________________________________________________________ -31-
regarding informal payments tend to be less precise and more vulnerable to misinterpretation by respondents (Lewis 2000).

Other than exit surveys, data on informal payments has been extracted from national expenditure surveys, such as the Hungarian household budget survey, and the World Bank‟s Living Standard Measurement Surveys (LSMS Team 2006). A list of survey results and different study designs on informal payments can be found in a review of informal payment surveys in Lewis (2000).

# _Usefulness of population surveys_

In practical terms, facility level surveys are likely to be the main source of information in most developing countries, but they can be usefully complemented by population surveys in a number of ways. One of the most important would be to rely on population surveys to determine the relative weight of different kinds of facilities in providing health care services. Beginning from a sample frame constructed from answers to health care utilization in a population survey is the most reliable way of developing a clear picture of what kinds of facilities are most important to health care provision. Developing a sampling frame for health care facilities based on the population‟s use of those services can give a fully representative picture of governance performance across the entire health sector, across different forms of provision and across different geographic and socioeconomic categories.

Two important population survey initiatives include the World Bank‟s Living Standards Measurement Survey (LSMS) project and the Demographic and Health Surveys (DHS) initiated by USAID. The LSMS tends to have relatively few questions on health care utilization and health status compared to the DHS. By contrast, DHS tends to have relatively few questions on socioeconomic characteristics and concentrates on the health of mothers and children.

If the effort to measure governance performance were to include additional measures – such as satisfaction or perceptions of quality and corruption – then public opinion surveys can be helpful. For example, Afrobarometer is an example of a large-scale household survey that measures governance performance in many political dimensions as well as with regard to service delivery. Afrobarometer tracks trends in public attitudes towards the institutions that serve them on a regular basis, asking questions about whether public health care services are affordable, whether medicines are available, whether individuals feel they are treated respectfully, whether doctors are absent, whether waiting times were long, and whether illegal payments were demanded (Afrobarometer 2009). The answers to these questions are comparable across countries in terms of being representative of the national sample; however, they do not account for differences in the structures of health care services across countries and ways in which the units of analysis – the facility or health care organization – may differ across contexts. That is, the questions are often too general to link respondents‟ views to specific health care service providers. Such population perception surveys can be useful complements to facility surveys but are decidedly second best options.

______________________________________________________________________________________

-32-
_Issues in measuring the proportion of counterfeit medicines_ The true scale of the counterfeit drug market remains unknown since no one has conducted a worldwide standardized effort to measure the proportion of counterfeit drugs in the global market (Newton et al 2009; WHO 2009). A counterfeit medicine is “a medicine, which is deliberately and fraudulently mislabeled with respect to identity and/or source. Counterfeiting can apply to both branded and generic products and counterfeit products may include products with the correct ingredients or with the wrong ingredients, without active ingredients, with insufficient active ingredients or with fake packaging” (WHO 2009). The share of counterfeit medicine being sold and distributed in a country indicates how well the market is regulated, generally through public rules, monitoring and enforcement but also generally including the effects of competition and suppliers‟ reputations. Both the proportion of shops selling counterfeit medicines and the average share of counterfeit medicines in any given outlet are useful measures for this aspect of government performance (Newton et al 2009).

Most studies have used convenience sampling to assess the scope of counterfeits, but for comparability and precision, initial random lot quality assurance sampling with subsequent large scale, formal random sampling is the appropriate method (Newton et al 2009). While convenience sampling is inherently biased, it can serve as an initial exploratory tool, but only a representative random sample can provide accurate assessments of the scope of the problem.

Existing studies of counterfeit medicines lack comparability for a number of reasons. One source of confusion is studies that many studies fail to distinguish between counterfeit, substandard, and expired drugs.<sup>13</sup> Other shortcomings of existing studies include the use of insufficiently large samples, announced sampling, or biased sampling. Apparently, only three peer-reviewed studies have used randomized sampling (Newton et al 2009).<sup>14</sup> For recommendations on how to objectively and systematically assess the prevalence of counterfeit medicines, see Newton et al 2009.

Until recently, the high cost of chemical analysis has inhibited large-scale analyses in developing countries. However, new technologies are sharply reducing these costs. For example, Africa Fighting Malaria recently conducted a survey of anti-malarial drugs in six major African cities to determine whether effective drugs were available. Their experience suggests that reliable representative results can be obtained in a given country for as little as US$5,000, and would rise in line with the number of medicines tested and the complexity of the sampling required to obtain a representative sample (author‟s estimate based on Bate et al 2008).

> 13 According to the International Pharmaceutical Federation (FIP – www.fip.org), substandard drugs are “genuine products which do not meet quality specifications set for them.” Expired drugs are those for which the manufacturer can no longer guarantee full potency and effectiveness.

> 14 USAID, in collaboration with the Pharmaceutical Security Institute (PSI) has compiled a list of all published studies and data on counterfeit and substandard medicines in USAID-assisted countries (USP DQI 2009). The accuracy of the data differs from study to study, yet the list itself demonstrates that counterfeit drugs are a serious problem.

______________________________________________________________________________________

-33-
In sum, the prevalence of counterfeit drugs can be measured in a systematic random sampling approach, both in country and across countries. As such, this measure could form the basis for a cross-country indicator on how well given countries are regulating (and also monitoring) the drug market.

# **_Governance determinants_**

The strategy for collecting information on governance determinants in ways that are comparable across countries is much less clear. Most governance determinants are context-specific despite efforts to identify objective and comparable indicators of ownership, decentralization, formal procedures and stakeholder participation. (See Box 3.)

# **Box 3: Assessing Governance Determinants with Expert Informants**

WHO has developed an assessment tool for governance of pharmaceuticals (WHO 2006). It provides a research protocol and survey instrument with 66 questions that address a wide range of formal procedural and performance measures in the regulation, procurement, distribution and dispensing of medications. The protocol provides guidance to researchers regarding the number and types of key informants who should be interviewed. The assessment is currently underway in more than 10 countries.

In 2000, the Pan American Health Organization (PAHO) convened a committee of public health experts who identified 11 Essential Public Health Functions (EPHF) and subsequently developed a survey instrument designed to assess a country‟s ability to carry out those functions. The EPHF survey is administered to individuals who are considered knowledgeable about the health sector (PAHO 2008). The intent of the survey is to answer questions about the capacity of the public health system to fulfill its essential functions in a yes-or-no format, asking for example whether the public health surveillance system is capable of analyzing a public health threat. With this series of binary questions, the EPHF assesses the public health system‟s strengths and weaknesses relative to this expert view of what a public health system should do.

Both of these initiatives, by relying on expert responses to a series of detailed questions parallel the approaches found in a number of measurement approaches being used in broader governance studies, such as the World Bank‟s CPIA and Freedom House.

One strategy would be to follow the approach of Kaufmann and Kraay, identify a limited number of governance dimensions, classify data gathering efforts by as many organizations as possible, aggregate and then normalize the measures. The advantage of such an approach is that it can reduce errors and bias by combining multiple sources of information on the structural features of health sector governance. The key disadvantage is that the quality of poor data gathering efforts – in terms of both validity and reliability – might overshadow better efforts and generate more confusion than information.

Another strategy would be to cull the existing range of indicators and select those that fulfill some basic criteria: that they are available, accurate, independent, comparable across countries and over time, and relevant. The primary difficulties here are likely to be ______________________________________________________________________________________

-34-
related to finding indicators that demonstrate both cross-country comparability and relevance.

A final strategy is to develop a database of within-country studies of governance determinants and classify them according to common avenues of inquiry. These can then be subjected to procedures applied by synthetic reviews – clearly identifying the question about governance that is being asked, establishing criteria for the quality of the underlying evidence and assessing findings accordingly.

Ekman (2004) demonstrates how such synthetic reviews can be conducted on a governance determinant. In this case, the focus of the study was the impact of community health insurance schemes on utilization of health care services, out-of-pocket expenditures and health status. By clearly identifying the hypotheses under consideration, pre-establishing criteria for assessing the quality of evidence, and conducting a systematic search for studies, Ekman was able to take studies conducted in specific contexts and determine what evidence was available for generalizable lessons.

It is this latter strategy that is most likely to generate the kind of evidence that policymakers are seeking when they ask how to improve health service provision by improving health sector governance.

# **6. Conclusion**

After reviewing the many ways governance has been used and studied, this paper argues for defining governance as the combination of political, social, economic and institutional factors that affect the behavior of organizations and individuals and influence their performance. Applying this concept to applied research in the health sector requires clarifying which organizations are the subject of analysis (e.g. health insurers, public health services, regulatory agencies) and how their performance is measured (e.g. what are their missions in terms of service provision or health outcomes). It also requires distinguishing governance determinants, such as ownership, decentralization, formal procedures and stakeholder participation, from governance performance (e.g. whether formal procedures are implemented, workers fulfill their responsibilities, or stakeholders have substantive input into decision processes).

Measures of governance performance in public health care provision can be gathered through facility and population surveys. Facility surveys can directly measure certain aspects of governance performance, such as the share of workers who are absent without legitimate excuses. While population surveys are less direct, they can still elicit information about the performance of health facilities in terms of absenteeism or informal payments. The existing literature contains a manageable number of governance performance measures for public health care provision that could form the basis for a ______________________________________________________________________________________

-35-
regular process of data collection that is increasingly comparable across countries, relevant, valid and reliable.<sup>15</sup> By contrast, measures of governance determinants are more difficult to choose and collect. Unlike governance performance indicators, which measure the congruence between actions and expectations, governance determinants vary highly across contexts. They are best conceived as hypotheses that require testing: does private ownership of health care services encourage better organizational performance? Does stakeholder participation improve policy-making? Do establishing formal procedures reduce corruption?

While it is possible to initiate a process of measuring governance determinants in the health sector and creating a large cross-country database, much like the ones developed for governance and economic development research, this paper argues instead for researching governance determinants with case studies followed by synthetic reviews. In this way, it is hoped that the advantages of case-study research can be emphasized while reaping the benefits of cross-country evidence in a synthetic review process.

> 15 Future work needs to identify governance performance measures that would be appropriate to private health care providers given that they account for the majority of health care services in many low- and middle-income countries, including Nigeria, China and India.

______________________________________________________________________________________

-36-
# **Appendix: Survey Tools and Websites**

# <u>PETS/QSDS (proportion of funds reaching designated health facilities)</u>

- World Bank PETS/QSDS Survey Tools <u>http://go.worldbank.org/1KIMS4I3K0</u>

- PETS/QSDS instrument used in Mozambique <u>http://go.worldbank.org/I2TPOUU6J0</u>

# - <u>Drug availability (stock out rates)</u>

- Health Action International Survey Instruments: Prices, Availability, Affordability, and Price Components

- <u>http://www.haiweb.org/medicineprices/manual/documents.html</u>

- IMAT (Inventory Management Science for Health) Tool, Indicator 3: Percentage of Medicines in Stock <u>http://www.msh.org/resource-center/inventory-management-assessmenttool.cfm</u>

- International Medical Products Anti-Counterfeiting Taskforce (IMPACT) <u>http://www.who.int/impact/en/</u>

# <u>Counterfeit drugs</u>

- United States Pharmacopeia

- <u>www.usp.org</u>

- Pharmaceutical Security Institute

-

- <u>http://www.psi inc.org/index.cfm</u>

- TruScan

- <u>http://www.ahurascientific.com/material verification/products/truscan/index.php</u>

- Securing Pharma <u>www.securingpharma.com</u>

# <u>Other Links</u>

The World Bank‟s Country Policy and Institutional Assessment (CPIA): http://go.worldbank.org/7NMQ1P0W10 WHO‟s World Health Report 2000: http://www.who.int/whr/2000/en/index.html WHO‟s International Health Regulations (IHR) http://who.int/topics/international_health_regulations/en/ ______________________________________________________________________________________

-37-
WHO‟s National Health Accounts: <u>http://who.int/nha/en/</u> WHO‟s Good Governance in Medicines initiative: <u>http://www.who.int/medicines/ggm/en/index.html</u> ______________________________________________________________________________________ -38-
# **References**

Afrobarometer (2009). “Poverty Reduction, Economic Growth and Democratization in Sub-Saharan Africa.” Briefing paper 68. May.

Amin, Samia, Jishnu Das, and Markus Goldstein (2008). _Are You Being Served? New Tools for Measuring Service Delivery_ . Washington, DC: World Bank.

Arndt, Christiane and Charles Oman (2006). “Uses and Abuses of Governance Indicators.” Paris: OECD Development Centre Studies.

Bate, Roger, Philip Coticelli, Richard Tren, and Amir Attaran (2008). “Antimalarial Drug Quality in the Most Severely Malarious Parts of Africa – A Six Country Study.” _PLoS ONE_ 3(5): e2132. doi:10.1371/journal.pone.0002132 Bjorkman, Martina and Jakob Svensson (2008). “Efficiency and Demand for Health Services.” Washington, DC: World Bank. Mimeo.

Bossert, Thomas J., Till Bärnighausen, Diana Bowser, Andrew Mitchell, and Gülin Gedik (2007). “Assessing Financing, Education, Management and Policy Context for Strategic Planning of Human Resources for Health,” Geneva: World Health Organization.

Bossert, Thomas J. and J.C. Beauvais (2002). “Decentralization of Health Systems in Ghana, Zambia, Uganda and the Philippines: A Comparative Analysis of Decision Space,” _Health Policy and Planning_ , 17(1):14-31.

Bovaird, Tony and Elke Löffler (2003). “Evaluating the Quality of Public Governance: Indicators, Models and Methodologies.” _International Review of Administrative Sciences_ , 69(3):313-328.

Brinkerhoff, Derick W. and Thomas J. Bossert (2008). “Health Governance: Concepts, Experience, and Programming Options.” Health Systems 20/20. Washington, DC: USAID. February. http://www.healthsystems2020.org.

Chaudhury, Nazmul, Jeffrey Hammer, Michael Kremer, Karthik Muralidharan and F. Halsey Rogers (2006). “Missing in Action: Teacher and Health Worker Absence in Developing Countries.” _Journal of Economic Perspectives_ , 20(1): 91–116.

Chaudhury, Nazmul and Jeffrey S. Hammer (2004). “Ghost Doctors: Absenteeism in Rural Bangladeshi Health Facilities.” _World Bank Economic Review_ 18(3):423-441.

Dehn, Jan, Ritva Reinikka and Jakob Svensson (2003) “Survey Tools for Assessing Performance in Service Delivery,” in François Bourguignon and Luiz A. Pereira da Silva, eds. _The Impact of Economic Policies on Poverty and Income Distribution: Evaluation Techniques and Tools_ . New York, NY: Oxford University Press. pp. 191–212.

Ekman, Björn. (2004). “Community-Based Health Insurance in Low-Income Countries: A Systematic Review of Evidence.” _Health Policy and Planning_ . 19(5):249-270.

Emanuel, Ezekiel J. and Linda L. Emanuel (1996). "What Is Accountability in Health Care?" _Annals of Internal Medicine_ 124(2):229-239. January.

______________________________________________________________________________________ -39-
European Observatory on Health Systems and Policies (2006), “Health targets and (good) governance,” _Euro Observer: The Health Policy Bulletin of the European Observatory on Health Systems and Policies_ , 8(1), Spring.

Gaal, Peter, Paolo Carlo Belli, Martin McKee, Miklós Szócska (2006). “Informal Payments for Health Care: Definitions, Distinctions, and Dilemmas.” _Journal of Health Politics, Policy and Law_ , 31(2):251-293.

Gauthier, Bernard (2006). “PETS-QSDS in Sub-Saharan Africa: A Stocktaking Study,” HEC Montreal and World Bank, AFTKL. Washington D.C: World Bank. September.

Harding, April and Alexander S. Preker (2003), “A Conceptual Framework for the Organizational Reforms of Hospitals,” in A.S. Preker and A. Harding, eds. _Innovations in Health Service Delivery: The Corporatization of Hospitals_ . Washington, DC: The World Bank. pp. 23-78.

Hollingsworth, Bruce (2003). "Non-Parametric and Parametric Applications Measuring Efficiency in Health Care.” _Health Care Management Science,_ 6:203–218.

Kaufmann, Daniel and Aart Kraay (2008), “Governance Indicators: Where Are We, Where Should We Be Going?” _The World Bank Research Observer_ , 23(1):1-30.

Kaufmann Daniel, Aart Kraay and Massimo Mastruzzi (2007), “Governance Matters VI: Aggregate and Individual Governance Indicators for 1996–2006.” World Bank Policy Research Working Paper 4280. Washington, DC: World Bank. July.

Kotalik, Jana and Diana Rodriguez, eds. (2006). _Global Corruption Report 2006: Corruption and Health_ . London: Pluto Press and Transparency International.

La Forgia, Gerard M. and Bernard F. Couttolenc, eds. (2008). _Hospital Performance in Brazil: The Search for Excellence_ . Washington, DC: World Bank.

Lewis, Maureen (2006), “Governance and Corruption in Public Health Care Systems,” CGD Working Paper 78, Washington, DC: Center for Global Development.

Lewis, Maureen (2000). “Who Is Paying for Health Care in Eastern Europe and Central Asia?” Human Development Sector Unit, Europe and Central Asia Region. Washington, DC: The World Bank.

Lewis, Maureen and Gunilla Pettersson (2009), “Governance in Health Care Delivery: Raising Performance,” Draft, Washington, DC: World Bank. Mimeo. June 4.

Lindelow, Magnus (2006). “Tracking Public Money in the Health Sector in Mozambique: Conceptual and Practical Challenges,” East Asia Human Development Unit. Washington, DC: The World Bank. Draft.

Murray, Christopher J.L. and David B. Evans, eds. (2003). _Health Systems Performance Assessment: Debates, Methods and Empiricism_ . Geneva: World Health Organization.

Newton, Paul N, Sue J Lee, Catherine Goodman, Facundo M. Fernández, Shunmay Yeung, et al. (2009) “Guidelines for Field Surveys of the Quality of Medicines: A Proposal.” _PLoS Medicine_ 6(3): e1000052. doi:10.1371/journal.pmed.1000052 ______________________________________________________________________________________

-40-
Olsen, Mancur (1971). _The Logic of Collective Action: Public Goods and the Theory of Groups._ Cambridge: Harvard University Press.

Organization for Economic Cooperation and Development (2004), “OECD Principles of Corporate Governance.” PAHO (Pan American Health Organization) (2008). “The Essential Public Health Functions as a Strategy for Improving Overall Health Systems Performance: Trends and Challenges Since the Public Health in the Americas Initiative 2000-2007.” Working document.

PATHS (2008), _Celebrating Success. PATHS in Nigeria (2002-2008). Final Programme_ - _Report_ , Partnership for Transforming Health Systems. http://www.lath.com/Our <u>Projects/12</u> Reid, Gary J. (2009), “Actionable Governance Indicators – Concepts and Measurement,” AGI Initiative, Washington, DC: World Bank. Mimeo.

Reinikka, Ritva and Jakob Svensson (2006), “Using Micro-Surveys to Measure and Explain Corruption,” _World Development,_ 34(2): 359–370. February.

Roberts, Marc J., William Hsiao, Peter Berman, and Michael R. Reich (2004). _Getting Health Reform Right: A Guide to Improving Performance and Equity_ . New York: Oxford University Press.

Saltman, Richard and Odile Ferrousier-Davis (2000). “The concept of stewardship in health policy,” _Bulletin of the World Health Organization_ , 78(6):732-739.

Savedoff, William D. and Pablo Gottret, eds. (2008). _Governing Mandatory Health Insurance: Learning from Experience_ . Washington, DC: World Bank.

Savedoff, William D. ed. (1998). _Organization Matters: Agency Problems in Health and Education in Latin America_ . Washington, DC: Inter-American Development Bank.

Solon, Orville, Kimberley Woo, Stella A. Quimbo, Riti Shimkhada, Jhiedon Florentino, and John W. Peabody (2009). “A Novel Method for Measuring Health Care System Performance: Experience from QUIDS in the Philippines,” _Health Policy and Planning_ 24:1-8.

Spiller, Pablo S. (1990). “Politicians, Interest Groups and Regulators: A Multiple Principals Agency Theory of Regulation, (or Let Them Be Bribed),” _Journal of Law and Economics_ , 33:65-101.

Spiller, Pablo S. and Santiago Urbiztondo (1994), “Political Appointees vs. Career Civil Servants: A Multiple-Principals Theory of Political Institutions,” _European Journal of Political Economy_ .

USP DQI (U.S. Pharmacopeia Drug Quality and Information Program) (2009), “Matrix of Drug Quality Reports Affecting USAID-assisted Countries,” Washington, DC: U.S. Agency for International Development and United States Pharmacoepia. <u>http://www.usp.org/pdf/EN/dqi/ghcDrugQualityMatrix.pdf</u> ______________________________________________________________________________________

-41-
Welch, Gita and Zahra Nuru, eds. (2006). “Governance for the Future: Democracy and Development in the Least Developed Countries.” UN Development Program and UN Office of the High Representative for the Least Developed, Landlocked Developing Countries, and the Small Island Developing States. http://www.un.org/specialrep/ohrlls/ldc/Governancereport.pdf Wismar, Matthias, Martin McKee, Reinhard Busse, Divya Srivastava, eds. (2006). _Good Governance for Implementing Health Targets: Defining Targets, Exerting Influence and Using Intelligence._ Copenhagen: WHO Regional Office for Europe on behalf of the European Observatory on Health Systems and Policies.

World Bank (2004), _World Development Report 2004: Making Services Work for Poor People_ . Washington, DC: World Bank.

World Health Organization (2009). “General information on counterfeit medicines.” Geneva: Switzerland. Downloaded October 31, 2009. <u>http://www.who.int/medicines/services/counterfeit/overview/en/print.html</u> World Health Organization (2008), “Toolkit on monitoring health systems strengthening: Health Systems Governance,” Geneva: World Health Organization.

World Health Organization (2006). “Measuring Transparency to Improve Good Governance in the Public Pharmaceutical Sector: Draft Assessment Instrument,” Working draft for field testing and revision. Geneva: World Health Organization. January.

World Health Organization (2000). _The World Health Report 2000: Health Systems – Improving Performance_ . Geneva: World Health Organization.

______________________________________________________________________________________

-42-
# **Also consulted:**

Berglöff, E. and E. L. Von Thadden (1999). “The Changing Corporate Governance Paradigm: Implications for Transition and Developing Countries _._ ” Conference paper, Annual World Bank Conference on Development Economics.

Bouckaert, Geert and Steven van de Walle (2003). “Comparing Measures of Citizen Trust and User Satisfaction as Indicators of „Good Governance‟: Difficulties in Linking Trust and Satisfaction Indicators.” _International Review of Administrative Sciences_ , 69(3):329-343.

Brinkerhoff, Derick W. (2005). “Accountability and Good Governance: Concepts and Issues” in A.S. Huque and H. Zafarullah, eds. _International Development Governance_ . New York: CRC Press.

Brinkerhoff, Derick W. and Arthur A. Goldsmith (2005). “Institutional Dualism and International Development: A Revisionist Interpretation of Good Governance _._ ” _Administration and Society_ , 37(2):199-224. May.

Cohen, Jillian Clare, Monique F. Mrazek and Loraine Hawkins (2007), “Corruption and Pharmaceuticals: Strengthening Good Governance to Improve Access,” in J. Edgardo Campos and Sanjay Pradhan, eds. _The Many Faces of Corruption: Tracking Vulnerabilities at the Sector Level_ , Washington, DC: The World Bank. pp. 29–62.

Das Gupta, Monica, Varun Gauri and Stuti Khemani (2004). “Decentralized Delivery of Primary Health Services in Nigeria: Survey Evidence from the States of Lagos and Kogi.” Africa Region Human Development Working Paper 70. Washington, DC: The World Bank.

de Ferranti, David, Justin Jacinto, Anthony J. Ody and Graeme Ramshaw (2009). “How to Improve Governance: a New Framework for Analysis and Action.” Washington DC: Brookings Institution, 2009.

Denis, Jean-Louis, François Champagne, Marie-Pascale Pomey, Johanne Préval and Ghislaine Tré (2005). “Toward a Framework for the Analysis of Governance in Health Care Organizations and Systems.” University of Montréal. Ottawa: Canadian Council on Health Services Accreditation. October.

Devarajan, Shanta (2008). “Comment on „Governance Indicators: Where Are We, Where Should We Be Going?‟" _The World Bank Research Observer_ , 23(1):31-36.

Di Tella, Rafael and William D. Savedoff, eds. (2001). _Diagnosis Corruption: Fraud in Latin America’s Public Hospitals_ . Washington, DC: Inter-American Development Bank.

Gauthier, Bernard and Waly Wane (2008). “Leakage of Public Resources in the Health Sector: An Empirical Investigation of Chad,” _Journal of African Economies_ , 0: ejn011v132.

Gottret, Pablo and George Schieber (2007) _. Health Financing Revisited._ Washington, DC: World Bank.

______________________________________________________________________________________

-43-
Gray-Molina, George, Ernesto Pérez de Rada and Ernesto Yañez (2001). “Does Voice Matter? Participation and Controlling Corruption in Bolivian Hospitals,” in Rafael Di Tella and William D. Savedoff, eds. _Diagnosis Corruption: Fraud in Latin America’s Public Hospitals_ . Washington, DC: Inter-American Development Bank. pp. 27–56.

Gupta, Sanjeev, Hamid R. Davoodi and Erwin R. Tiongson (2002), “Corruption and the Provision of Health Care and Education Services,” in George T. Abed and Sanjeev Gupta, eds. _Governance, Corruption, and Economic Performance_ . Washington, DC: International Monetary Fund. pp. 245–279.

Kaufmann, Daniel, Aart Kraay and Pablo Zoido-Lobatón (1999). “Aggregating ” Governance Indicators _._ World Bank Research Working Paper No. 2195, Washington, DC: World Bank.

Leonard, Kenneth, Melkiory Masatu and Alex Vialou (2006), “Getting Doctors to Do Their Best: the Roles of Ability and Motivation in Health Care Quality,” _Journal of Human Resources,_ 42(3):682-700.

Lindelow, Magnus and Adam Wagstaff (2008), “Assessment of Health Facility Performance,” in Samia Amin, Jishnu Das and Markus Goldstein, eds. _Are You Being Served? New Tools for Measuring Service Delivery_ . Washington, DC: The World Bank. pp. 19–65.

Padovani, Eric A. and Emanuele Scorsone (2007). “Challenges in the Comparative Analysis of Local Government Bureaucracy and Performance Measurement: The Case of Italy and the United States,” Working Paper, European Group of Public Administration Annual Conference.

Walford, Veronica (2007), “Indicators of Health System Performance: Report of a Review for DFID,” DFID Health Resource Centre, London: DFID. Mimeo.

______________________________________________________________________________________

-44-