metadata: authors: 0: author: Lindelow, Magnus
1: author: Wagstaff, Adam
docna: 0: docna: Health facility surveys : an introduction
docty: Policy Research Working Paper
subtopic: Health Care Services Industry,Disease Control & Prevention,Public Health Promotion,Reproductive Health,Early Child and Children's Health,Health Service Management and Delivery,Immunizations
teratopic: Agriculture,Health, Nutrition and Population,Communities and Human Settlements
abstracts: cdata!: Health facility surveys come in various
            guises. One dimension in which they vary is their
            motivation. Some seek to understand better links between
            households and providers. Others seek to understand better
            provider behavior and performance. Still others seek to
            understand the interrelationships between providers, while
            yet others seek to shed light on the linkages between
            government and providers. Health facility surveys differ too
            in the data they collect, in part due to the different
            motivations. Surveys also vary in the way they collect data,
            some relying on direct observation, some on record review,
            and some on interview. Some quality data are collected
            through clinical vignettes. Facility data have been put to a
            variety of uses, including planning and budgeting;
            monitoring, evaluation, and promoting accountability; and
            research. Lindel and Wagstaff review some of the literature
            under each heading and offer some conclusions regarding the
            current state of health facility surveys.
repnme: repnme: Health facility surveys : an introduction
display_title: Health facility surveys : an introduction
listing_relative_url: /research/2003/01/2120330/health-facility-surveys-introduction
topicv3: Education, Skills Development and Labor
            Market m1326135 1484,Health Workforce m1326573 631,Health Services Delivery m1327294 1305,Immunizations m1327505 1459,Health m1504661 3480
publishtoextweb_dt: 2003-02-07T00:00:00Z
historic_topic: Agriculture,Health, Nutrition and Population,Communities and Human Settlements
no_of_pages: 56
author_keys: 0: author_key: NA
1: author_key: NA
model_extractions: POLICY RESEARCH WORKING PAPER 2953

# Abstract

Health facility surveys come in various guises. One dimension in which they vary is their motivation. Some seek to understand better links between households and providers. Others seek to understand better provider behavior and performance. Still others seek to understand the interrelationships between providers, while yet others seek to shed light on the linkages between government and providers. Health facility surveys differ too **in** the data they collect, **in** part due to the different motivations. Surveys also vary in the way they collect data, some relying on direct observation, some on record review, and some on interview. Some quality data are collected through clinical vignettes. Facility data have been put to a variety of uses, including planning and budgeting; monitoring, evaluation, and promoting accountability; and research. Lindeldw and Wagstaff review some of the literature under each heading and offet some conclusions regarding the current state of health facility surveys.

This paper-a product of Public Services, Development Research Group-is a background paper prepared for the World Bank workshop on Health Facility Surveys, December 2001. Copies of this paper are available free from the World Bank, 1818 H Street NW, Washington, DC 20433. Please contact Hedy Sladovich, room MC3-607, telephone 202-473-7698, fax 202-522-1154, email address hsladovich@worldbank.org. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at mlindelow@worldbank.org or awagstaff@worldbank.org. January 2003. (47 pages)

> _The Policy Research Working Paper Series disseninates the findings of work_ **_in_** _progress to encourage the exchange of Ideas about development issues. An objective of the series is to get the_ **_findings_** _out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should_ **_be_** _cited accordingly The findings, interpretations, and conclusions expressed_ **_in_** _this paper are entirely those of the authors. They do not necessarily represent the_ **_view_** _of the World Bank, its Executive Directors, or the countries they represent._

Produced by the Research Advisory Staff
# **Health Facility Surveys:**

## **An Introduction**

### Magnus Lindelow

The World Bank, 1818 H St. NW, Washington, D.C. 20433, USA Centre for Study of African Economies, St Anthony's College, Oxford OX2 6JF, UK mlindelow(worldbank.org

### **Adam Wagstaff**

The World Bank, 1818 H St. NW, Washington, DC, 20433, USA The University of Sussex, University of Sussex, Brighton, BN 1 6HG, UK awagstaff(worldbank.org Background paper prepared for World Bank Workshop on Health Facility Surveys, 12 December 2001.
# **Contents**

|I.IN<br>T<br>R<br>O<br>D<br>U<br>C<br>T<br>IO<br>N<br>.........................................<br>1|
|---|
|**II.**MOTIVATIONSFORFACILITYSURVEYS.........................................<br>**1**|
|Provider-householdlinkages........................................<br>2|
|Measuringandunderstanding providerperformance ........................................<br>3|
|**III.WHAT**DATATOCOLLECTANDHOWTO**Do**IT........................................<br>7|
|Inputs ........................................<br>7|
|Costs ........................................<br>9<br>|
|Outputs,activitiesandthroughputs .......................................<br>10<br>|
|Quality.......................................<br>13|
|Staffperceptionsandbehavior.......................................<br>18<br> <br>|
|Theinstitutionalandorganizational enviromnent .......................................<br>19|
|**IV.**HowFACILITYDATA HAVE**BEEN**USED**........................................**<br>20|
|Planningandbudgeting.......................................<br>20|
|Monitoring,evaluation,and accountability.......................................<br>21|
|Research...........<br>24|
|**V.CONCLUSIONS...........**<br>28|
|APPENDIX:**A**REVIEWOFHEALTH FACILITYSURVEYS........................................<br>30|
|DemographicandHealthSurveys(DHS andDHS+)............<br>...........................30|
|TheLivingStandardsMeasurement Study (LSMS).......................................31|
|Situation Analysis.......................................<br>33|
|RAND Surveys .......................................<br>34|
|MEASURE Evaluation.......................................<br>35|
|WHOMulti-CountryEvaluationofIMCI.......................................<br>37|
|Public ExpenditureTrackingSurveys.......................................<br>38|
## I. **Introduction**

In the last 20 years, household surveys have considerably improved our understanding of health outcomes and health related behavior in developing countries. For example, data from surveys such as the Living Standard Measurement Study (LSMS) surveys and Demographic and Health Surveys (DHS) have shed light on the nature and determinants of health problems in particular contexts, while also providing information on health related behavior, including household expenditure on health care and the utilization of health services. Although these and other surveys have considerably strengthened the basis for policy design and implementation in the health sector, they have also highlighted the need to better understand the supply side of service delivery. The supply side has also received increasing attention as a consequence of many health systems having to grapple with real or perceived problems of inefficiency, low quality, inequalities, and unsustainable financing. In this context, health facility surveys have come to comprise an important source of information, both about the characteristics and activities of health facilities, and the financing and support systems that serve them.

Although these surveys all have the health facility surveys as their focus, they vary along at least four important dimensions. First, they have different motivations. Some aim to understand better how health facility characteristics influence health-seeking behavior and health outcomes, while others make the facility the focus of analysis, emphasizing issues such as cost, efficiency, and quality. In addition, some surveys are designed to shed light on the broader context of service delivery, including links between different providers, or between providers and government. For example, some surveys have tracked public expenditures from the central ministry to rural facilities. A second difference concerns the scope and nature of data collected. For example, while many collect data on inputs, not all collect data on costs or on clinical dimensions of health care quality. Third, surveys have collected data in different ways, and adopted different approaches to measurement. For example, quality is sometimes assessed indirectly through data on inputs, sometimes directly, e.g., through the use of clinical vignettes or consultation observation. Finally, surveys vary in the uses to which the data have been put. In some cases, the focus has been on research. In others, the principal use has been to design interventions, or to monitor and evaluate programs.

In this paper, we seek to provide an introduction to health facility surveys, and to the methodological approaches that underpin them. The paper is organized as follows. In section II we discuss what has motivated different health facility surveys. Section III provide detail on the type of data that have been collected in different surveys, and on approaches to measurement. Section IV outlines how facility data have been used. Finally, in section V, we offer some conclusions, focusing on lessons learnt and emerging themes.

# **II. Motivations for Facility Surveys**

There are two broad motivations for health facility surveys. These can be illustrated with reference to Fig. 1 below, which illustrates how health facilities interact with one other, 1
and with other elements in the health system, in the production of health outcomes. First, health facility data can be collected with a view to better understand how health services affect health related behavior and outcomes, in particular by collecting data on health care quality, access, and user charges. Second, health facility surveys can be used to monitor and analyze the performance of health care providers. In this case, the focus is on measuring different dimensions of health facility performance, as well as institutional and organizational determinants of performance. This may concern characteristics of the facility itself, or its interaction with other providers-whether between different levels of the public system, or between public and private providers. Facility performance may also depend on features of the administrative and support system "upstream" from the facility, such as government spending, policy, and regulation.

IFAguire i - Thoe routes to better nealRth outcomnes <!-- Start of picture text --> ~gProviders<br>Goverimment<br>o expenditure -------- _ ---- E[ouseholdth<br>• policy ~~~~ be h orv outcoom2<br>o reguatnion @ 0 \7/<br><!-- End of picture text --> In many surveys, the motivation for collecting health facility data is to understand better the link between health facilities, on the one hand, and the health-seeking behavior and health outcomes of households, on the other (see Fig. 2). Indeed, a number of health facility surveys have been implemented with the explicit purpose of feeding into the analysis of household level data. Although household-level behavior and outcomes are shaped in large part by factors within the household (income, education, location, etc.), health care qualitye.g., input availability, amenities, and clinical and inter-personal skills of health workershave an impact not only on utilization of health services, but also on the effectiveness of health care interventions and client perceptions. As a consequence, poor quality can drive a sizeable wedge between actual and potential cost-effectiveness of interventions (Wouters 1991).

2
_Figure 2 - Provider-household linkages_ <!-- Start of picture text -->
| Public |<br>Government<br>* expenditure ----------- Household Health<br>* policy outcomes<br>* regulation<br><!-- End of picture text -->

Work on the link between households and providers dates back to the 1970s, when the World Fertility Survey (WFS) started collected data to measure the impact of health service availability on fertility and mortality (Turner and others 2001). Initially, data were collected at community level through interviews with key community informants. This practice was continued in the context of the DHS surveys, which took over from the WFS in 1984.' Many LSMS surveys also included community modules that collected information on, _inter alia,_ the availability of public services.

From collecting community data on service delivery, it was a natural extension to visit the actual service delivery units. In the late 1980s, a number of LSMS surveys (e.g., C6te d'Ivoire and Jamaica) experimented with health facility and/or school surveys to complement the household data. A more systematic approach-Situation Analysis (SA)-was introduced by the Population Council in 1989, where the focus was on family-planning and reproductive health services. At least in part, the approach was motivated by findings emerging from the DHS surveys, which indicated that weaknesses on the supply side were important in explaining low contraceptive prevalence (Miller and others 1997, 1998). Unlike the LSMS facility surveys, the SA did not also collect household data, although client interviews were carried out at the sample sites. More recently, facility surveys have been implemented in conjunction or coordination with DHS household surveys. These so called Service Provision Assessments (SPAs) are ambitious in scope, and seek to provide information about the characteristics of health services, including extensive information about quality, resource availability, and infrastructure.<sup>2</sup>

# _Measuring and understandingproviderperformance_

In some surveys, the facility rather than the household is the object of interest. Facility data can serve monitoring or program evaluation purposes, but may also provide the basis for empirical work on the determinants of facility performance. In other words, how do we

> ' In the context of the DHS surveys, these conununity surveys were referred to as the Service Availability Module.

> 2 Linking household and facility data raises important sampling questions. For example, if data are only collected from the facility that is closest to the community, **it** may only be possible to capture the local health care market in a very limited sense. This issue is particularly problematic in urban and peri-urban setting. Also, analysis of bypassmg and choice between providers is precluded. The alternative is to collect data from a more comprehensive sample of health care providers in the community. However, the survey can impossibly cover the "full" market for health care, and the appropriate interpretation of empirical results on the effect of facility characteristics on health outcomes and behavior is often far from clear. Moreover, as a consequence of the inevitable clustenng of facilities, the precision of point estimates of facility variables is likely to be reduced. These issues are discussed in considerable detail in Turner et al. (2001).

3
account for differences across providers in key dimensions of performance such as quality, costs, and efficiency? 3 The facility surveys designed to explore provider-household linkages have generally offered little insight into these issues. Fig. 3 illustrates the complex institutional and organizational environment that influences the performance of health care providers. This operates through the financing, support, regulation, and oversight provided by administrative levels "upstream" from the facility, through the competitive environment in which the facility is operating, and through the oversight, accountability, and influence activities by households and communities.

_Figure 3 - Provider performance_ <!-- Start of picture text --> Providers<br>Quality, costsand efficienc<br><!-- End of picture text --> Some surveys have focused explicitly on the analysis of costs and efficiency, often with a view to say something about differences between goverment and private providers (see, e.g., Bitran 1995; Wouters 1993). There are also examples of surveys that have sought to measure and analyze the impact of different management indicators on performance, in particular costs (Somanathan and others 2000). In principle, data on inputs, costs, and outputs should be provided by a well-functioning management informnation system. However, where such systems are weak or otherwise deficient, a survey approach can be valuable.

# HeaRth woirkePrs

The perceptions and behavior of health workers have received increasing attention in recent years. This is not surprising, given the central role that health workers play in the delivery of care. Moreover, there is increasing evidence that "micro-level" issues, such as low levels of effort and opportunistic behavior, is hampering both quality and efficiency in service delivery.<sup>4</sup> Standard principal-agent theory provides a framework for analyzing worker incentives.<sup>**5**</sup> Health workers can be seen as agents for multiple principals, including patients,

> 3Thse concept of "performance" is discussed fuirther below.

> 4 See, e.g., Ferrinho et **a]** (1998) for a discussion of absenteeism; Ensor and Witter (2001) and Killingsworth et al. (1 999) for evidence on informal payments; McPake et al. (1 999) provide a detailed and quantified assessment of the extent **and** implications of informl economic activities of health workers in Uganda.

> 5Recently, the pernciples of agency theory have motivated reforms in public sector management, which emphasize

> **performance** measurement and incentives (see, e.g., Goddard, et al. 2000; Martinez and Martineau 1998; Mills 1997). These

4
communities, and government agencies with responsibilities for the delivery of health services. In general, the challenge is to induce health workers to exert effort in a range of different areas: clinical tasks (diagnosis, treatment, follow-up, outreach activities, etc.), psycho-social interaction with patients, and administration and maintenance of hygienic standards. In addition, there is a need to restrain opportunistic behavior (e.g., overcharging, stealing) by health workers. Agency problems in the health sector is the subject of a considerable theoretical and empirical literature.<sup>6</sup> Due to the nature of health care, including multiplicity of tasks and principals, as well as vague and poorly observable objectives, it is very difficult to establish clear contracts. In this context, factors such as merit pay, group incentives and career concerns gain importance (Dixit 1997, 2001; Wilson 1989). In addition, "intrinsic motivation" (Deci 1975) in the form of professional ethics or norms may also be important, in effect resulting in a self-enforcing contract.<sup>7</sup> The importance of intrinsic motivation has also been emphasized by a multidisciplinary literature that seeks to provide a more comprehensive picture of worker motivation. For example, Franco, Bennett, and Kanfer (2002) propose a framework in which worker motivation has many and complex influences, including non-monetary factors. They highlight the deeply psychological and cultural nature of worker motivation. This perspective on health worker motivation presents a more complex picture, where the effect of policy changes, including a softening of financial incentives, are highly context-contingent. Drawing on the psychological and sociological literature on worker motivation, some contributors have suggested that intrinsic motivation is not only an essential factor in resolving deep agency issues in the health sector, but that increasing reliance on extrinsic incentives may serve to undermine internal sources of motivation.<sup>8</sup> These issues raise a number of important conceptual, empirical, and methodological questions. For example, what is the relative importance of extrinsic and intrinsic motivation, and how do they interact? How do professional norms, commitment, or trust affect the motivation and behavior of health workers? How can we approach the measurement of intrinsic motivation and their impact on behavior? Is intrinsic motivation stronger or more important in certain types of organizations? There is currently little evidence on these issues, particularly for developing countries. Some research has tried to measure professional commitment through the use of self-administered questionnaires with put a broad range of questions concerning the worker's level of identification with the organization (e.g., hospital) efforts have tried to remedy a perceived lack of incentives in the public sector by introducing systems of rewards and sanctions in the form of "performnance management systems".

6A sizeable literature has dealt with the incentive issues that arise due to the asymmetry of information between patients and providers. (see Arrow (1963) for an early contribution). A related literature has studied the effect of provider payment systems on the incentives and behaviour of health workers. Chaix-Couturier and others (2000) and Gosden, Pederson, and Torgerson (1999) discuss the effect of different forms of physician payment-capitation, shared financial risk, fee-for-service, and salary-on medical practice. See also Barnum, Kutzin, and Saxenian (1995) for a discussion of the effect on payment systems on hospital performance.

7Kreps (1997) has suggested that in many cases what is referred to as intrinsic incentives may in fact be workers' response to fuzzy extrinsic motivators such as fears of discharge or career concems, but he also acknowledges that "true" intrinsic motivation may be an important factor m many contexts.

a For example, Segall (2000, p. 11) notes: "Market relationships present health care providers with perverse incentives and can do violence to the professional ethos of caring." See also Kreps (1997).

_5_
and its goals, willingness to exert effort on behalf of the organization, and general job satisfaction.<sup>9</sup>

# Links between provAiders

Another potential motivation for health facility surveys is to examine the interactions between providers. This can include the interactions between different levels within the public system, and the interactions between public and private providers. In OECD countries, the impact of competition on provider behavior has received attention in recent years. For example, Propper (1996) and Propper, Wilson, and Soderlund (1998) explore the effects of competition on prices in the NHS hospital sector in the United Kingdom. Soderlund and others (1997) explore the effects of competition between hospitals on hospital costs (rather than prices). Some work on competition and strategic interaction in the health care market has also been carried out in the US. For example, the Center for the Study of Health System Changel' has carried out a number of "Community Tracking Studies", some of which address these issues. Similar research, which could clearly benefit from health facility surveys, appears to be very limited in the context of developing countries."

# Government-prTovdeir linkages

> In other cases, health facility surveys have been undertaken to examine the interface

> between government and providers. The Public Expenditure Tracking Surveys (PETSs), for

> example, have been motivated by a desire to be able to account better for the flows of public expenditures from central government to regions and districts and thence to providers in the

> periphery. These studies hence relate to recent research concerning the link between public

> spending and health outcomes. Cross-country evidence suggests that total public spending on

> health has had much less impact on average health status than one might expect (for a review,

> see Filmer, Hammer, and Pritchett 2000). Tracking surveys and related studies have argued that the negligible effect from social sector spending to human development outcomes is

> likely to reflect problems of _identification,_ broadly defined. In other words, more spending

> does not necessarily imply more or better public services (Pritchett 1996, Reinikka 2001, and Reinikka and Svensson 2001). From the supply side one can identify two general explanations. First, there may be low efficacy in the transfer of funds within the public sector; for example, leakage of funds may prevent spending from reaching the intended frontline

> service provider (for instance, the health unit). Second, there may be low efficiency in the

> production of frontline goods and services (even if funds reach the frontline provider).

The need for this type of tool has increased in recent years, in the wake of debtreduction initiatives and a move towards sector and budget support under a "country-owned"

> **9** See, e.g., Bennet et **al.** (2000). See also Mowday and Steers (1979) for an early approach. This line of investigation can also take a more clinical perspective, by looking at the determinants of health worker performance in the management of

> cases. Some of this research has highlighted the important role that caregivers and the "community" can have on, e.g., prescribing practices (Paredes and others 1996; Rowe and others 2001).

> 10 http://www.hschange.org/.

> X l There is some evidence on the link between providers in developing countnes. For example, Gertler and Molyneaux (1997) report an increase in fees in private sector providers in response to the demand effect of an increase In user fees in public facilities. However, in general, the surveys are not explicitly designed to address these issues.

6
policy framework.<sup>12</sup> The premise of these initiatives is that sustainable development impact requires local governance capacity, and an effective public expenditure process. In practical terms, this means that governments now face new pressures to demonstrate that resources are being channeled into increased and improved delivery of public services.

On other occasions, the motivation has been to better understand the influence that governments have-through the various policy instruments-over provider performance. There are few examples of surveys specifically addressed to these questions. A recent case is a survey implemented in Paraguay, carried out with support from MEASURE Evaluation, to identify and quantify the effect of transferring management control for basic health services provision from central to municipal government in Paraguay on costs, efficiency, basic health service quality, and patterns of health service use and equity (Angeles and others 1999).

# **III. What Data to Collect and How to Do It**

As is clear from the previous section, health facility surveys have served widely differing purposes. The choice of approach, as well as the scope and precise content of the survey instruments, reflect these differences. Depending on the survey, data have been collected on inputs, costs, outputs, quality, staff behavior and perceptions, and the institutional and organizational environment. Different approaches have been used to collect the data, including facility questionnaires; health worker questionnaires, sometimes including tests of clinical skills; direct observation of facilities or facility "audits"; direct observation of patientclient interactions; record reviews (e.g., stock cards, material requisitions, patient registers); and, exit polls, possibly including a re-examinations. In addition, some facility surveys have also collected data from administrative units "upstream" from the facility. This section will review the types of data that have been collected in facility surveys, and the different approaches that have been applied.

## **_Inputs_**

Data on facility inputs are a standard feature of most facility surveys. There are three primary reasons for collecting these data. First, information about the availability and quantity of inputs can be used to assess the adequacy of resources at facility level. Second, and related to resource adequacy, the availability and characteristics of inputs at facility level affects the range, quality, and effectiveness of services provided. Finally, in combination with price information, data on inputs are required for the analysis of costs. Data on some facility inputs are, of course, often available from administrative records at national or sub-national level. However, a survey approach is motivated by a lack of confidence in these records, or by insufficient richness and detail.

> 12 In large part, this shift **in** the approach to aid has resulted from increased appreciation of the fungibility issue and from finding that lack of government control over projects and the budget process more generally have senously undermined long-term impact of aid. The World Bank (1998, p. 12) notes: "Aid agencies have a long history of trying to 'cocoon' their projects using free-standing technical assistance, independent project implementation units, and foreign experts - rather than trying to improve the institutional environment for service provision. ..They have neither improved services m the short run nor led to institutional changes in the long run."

7
Inputs may be divided into capital and recurrent inputs. The former are often defined as inputs that last for more than one year-e.g., vehicles, equipment, buildings, and other infrastructure. Many surveys collect some data on the characteristics of basic facility infrastructure, such as type of building, source of water and electricity, and availability of a toilet or latrine. Information about the availability and state of repair of means transport (van, car, motorcycle, bicycle, etc.) is often collected, both because these may be significant contributors to facility costs, and because the availability of transport serves to reduce access time in emergency cases, permits outreach services, and may also reduce interruptions in supplies. Similarly, on the premise that the absence or non-functioning of a certain piece of equipment has an adverse effect on the ability of the facility to provide effective health services, some surveys collect data on the quantity or availability of medically-related equipment. Sometimes attention is restricted to a limited list, including items such as refrigerator, sterilizing equipment, blood pressure monitors, scales, microscope, thermometer, stethoscope, and beds. However, in other surveys, a more comprehensive list of equipment has been considered, covering, for example, lab equipment and clinical tools.

Recurrent inputs primarily comprise personnel, supplies (drugs, vaccines, other medical supplies, non-medical supplies), and operations and maintenance (primarily for vehicles and buildings). The number and mix of staff are important characteristics of the health facility, having an impact on the range of services that can be provided, the technical competence with which these services are delivered, as well as overall cost. With this in mind, most surveys have collected data on the number of staff in different categories, sometimes with additional information on, qualifications, education and training, and terms of employment.1<sup>3</sup> For the purposes of costing, data on salaries and other benefits are also required. Beyond staffing, many surveys collect information about the availability of "essential" drugs and vaccines. Sometimes other medical and non-medical consumables have also been considered. Information about current availability or the frequency and duration of stock-outs can comprise an important quality indicator. Further quality information can be gleaned from data on drug prescription practices and drug charging. However, for the purposes of costing, data collection must go beyond questions about stocks, prescriptions, and charges, to consider the flow of drugs and other inputs to the facility.

Given the physical nature of inputs, approaches to data collection are not particularly controversial. Basic input data-e.g., on infrastructure characteristics, the availability of certain types of equipment, or the frequency and duration of drug stock-outs-can be gathered through an _interview_ with the facility director, although information reported by facility staff may be inaccurate. _Direct observation_ of inputs may therefore be advisable. This permits verification of what drugs are actually in stock at the time of the interview, as well as an assessment of the state of repair and level of functioning of essential inputs (e.g., temperature of refrigerator). A _record review_ is required if data on the flow of resources to the facility (e.g., number of drug kits received in a six month period) or historical data are required. However, this approach relies on the availability and reliability of records (e.g., stock cards)

> 13 In addition to staff numbers and basic characteristics, some surveys have sought to assess the clinmcal and interpersonal skills of health workers, as well as the perceptions of patients of the process of care. These are clearly important factors for the effectiveness of the care provided, as well as for the propensity for mdividuals to seek care from the respective health care provider. Approaches to collecting these types of data are discussed further below.

8
at facility level, a condition that is often not met. Finally, because of the paucity of records at facility level, some input data are best collected at administrative levels above the actual facility. This may include payroll data, as well as information on the supply of medical and non-medical supplies to the sampled facilities. The collection of comparable data from different levels (e.g., facility and district administration) also provides a basis for verification. Although this type of cross-checks can be very useful, data from different levels are often difficult to reconcile. This may be due to irregularities, or simply due to poor record keeping. If these issues are not anticipated, they can lead to serious problems of interpretation ex post.

# _Costs_

Costs arise from the use of various scarce resources in the production of health services-including patients' time, volunteer services, in-kind donations, etc. Information on costs is an important management tool, and is also useful in determining resource requirements in the process of planning and budgeting. In addition, cost information is a necessary input into the analysis of cost-effectiveness of health care interventions, and may serve as a guide for cost-recovery policies. Finally, information on costs is also important for the evaluation of service delivery performance, in particular efficiency.

Many studies of costs have relied on administrative cost data.<sup>14</sup> We may refer to this as a "top-down" approach. Top-down costing relies on data from central or lower level administration. These data can take many forms, including disaggregated budget allocations or financial records at hospital or sub-national administrative level. While this may be a convenient and relatively cheap way of gathering data, the scope for analysis can be limited by poor quality and insufficient disaggregation. These problems are likely to be particularly severe in developing countries, and for primary health care providers. Also, records of this nature may not exist or be available for NGO or for-profit providers.

In contrast to the top-down approach, the "bottom-up" approach proceeds by quantifying the use of resources at facility level.'<sup>5</sup> In general, it is possible to observe five categories of resources at facility-level: (i) staff time; (ii) supplies, medicines and other consumables; (iii) facility-level administration and overhead expenses (e.g., utilities); (iv) equipment; and, (v) buildings and other physical infrastructure. In addition, there are costs incurred at higher level (e.g., training, supervision, central and sub-national administration), which typically cannot be observed in the facility.

In "bottom-up" costing, a trade-off has to be made between detail and sample size. At one end of the spectrum, a survey can collect data on basic output indicators and on key input use in the production of services. At the other end, a full resource costing considers a more comprehensive range of inputs, and allocates resources used at provider level to specific service categories.' This typically includes some form of time-and-motion study, and a

> 14 Examples of studies based on admmnistrative data include Dor (1994), Bitran-Dicowsky and Dunlop (1993), and Anderson (1980). For a general review, see Barnum and Kutzin (1993) and Wagstaff and Barnum (1989a).

> 15 Costing is based pnmarily on input and price data. However, for some inputs, market prices may not be available, and some values must be imputed in order to find total cost estimates.

> 16 Lewis et al. (1996) comprise a rare example of this approach being used in a developing country. Through a detailed study, they estimate costs for outpatients, inpatient and emergency care, taking into account case mix, clinical norms, indirect and imputed costs, as well as depreciation of physical infrastructure and equipment. They find that although personnel

9
recordin of all goods and services (direct and indirect) associated with different categories of services. In this way, the resource costs needed to treat the average case can be calculated. This type of study can provide interesting perspectives on efficiency. However, there are also important limitations. In particular, detailed costing is very time intensive, and attention is usually restricted to one or a few health care providers. For example, Lewis, La Forgia, and Sulvetta (1996) studied only a single hospital, although in considerable detail.

Some surveys have set out to estimate service specific costs. In this case, the issue of "joint" inputs-i.e., inputs that are used in the production of more than one type of servicecomprise a considerable challenge. For example, most facility staff are involved not only in the provision of outpatient services, but also in immunizations, deliveries, and inpatient care. Similarly, certain pieces of equipment may be shared across many categories of services. In terms of costing specific services, this raises the question of how the respective inputs should be allocated across the services in which they are used. This can be dealt with on the basis of _ad hoc_ assumptions, or using information collected in the survey-for example on the proportion of time staff spends on different types of activities.'<sup>8</sup> There are however some fundamental methodological concerns associated with the analysis of this type of data. in particular, in a multi-input/multi-output context, where facilities can substitute between inputs depending on the scale and scope of activities, it is far from clear how differences in unit costs across different facilities (based on an accounting approach to costing) can be interpreted.

# _oipits, activides and toug/pMes_

It is possible to conceptualize service delivery as a production process, where inputs are transformed into outputs. The "technology of production" refers to the relationship between inputs and outputs, and also determines costs. Information on outputs is clearly important from a management perspective, for example in establishing input requirements. However, the planning and management function is typically filled by routinely collected administrative data. In contrast, survey data on outputs are often collected with a view to analyzing costs and the relationship between inputs and outputs.

How should the output of a health care provider should be appropriately measured? A first issue concerns the distinction between "readiness-to-serve" capability and rendered services. At one level, we may think that the appropriate facility output is the capacity to provide services-e.g., outpatient care, immunizations, antenatal care, deliveries-rather than the actual number of cases seen. This is because households, as demanders of health services, play a part in influencing the level of utilization, so the amount of services delivered is not entirely under the control of providers. For example, low levels of utilization may be due to

> represent only 2.5 11.5 and 5.1 percent of total hospital costs for emergency patients, outpatients, and inpatients respectively,

> 84 percent of hospital budget in the Dominican Republic pays salaries. This indicates a high degree of waste in the system, and clearly demonstrates the possible divergence between expenditures and actual use of resources in the delivery of specific services.

> **17** Even in more comprehensive costing exercises, capital costs are often excluded due to problems of measurement. Capital costs arise from the use of assets in the production of services, and include the opportunity costs of the initial investments and the depreciation of the asset over time.

> 18 "Step down analysis" is a standard methodology for distributing aggregate costs across department (cost centers), and, ultimately, to final service categories (with measurable outputs). For a review of issues in the allocation of overhead costs, see, e.g., Drummond et al. (1997)

10
low population density or a good overall health status in the catchment area. Insofar as variation in output across facilities is driven by these types of factors, service capacity would appear a conceptually appealing measure of output, at least for the purpose of efficiency analysis. For example, one might measure _potential_ inpatient days rather than actual days, just as one might measure school _places_ rather than school _enrolment_ or attendance.1<sup>9</sup> However, in many cases, a divergence between service capacity and rendered services is due to factors that are within the discretion of the facility. Here, of course, there is some scope for disagreement as to precisely what _is_ under the control of providers.<sup>20</sup> In general, both "readiness-to-serve" capability and rendered services are problematic as measures of output. Most surveys collect data on the number of units of services delivered, largely because these type of data are more readily available. However, for some types of analysis it is clearly important to try to remove the confounding effects of influences that are genuinely beyond the control of providers.

A second issue to consider in the measurement of outputs is the heterogeneity of service outputs. There are different sources of this heterogeneity. First, quality may differ considerably over time, across health care facilities, and even between individual clients at a given facility. For example, a thorough outpatient consultation with a doctor is a very different service than a rushed consultation with a poorly trained nursing aide. While this is more important for some types of services than others (e.g., immunizations may be considered reasonably homogenous in terms of quality), this issue can create serious problems of comparability across time and space. Although it is in principle possible to control for quality differences in analysis, this has proved difficult in practice. Second, within a particular service category, there can be a notable variation in case mix and case complexity. Again, this causes problems in terms of comparability.21 For example, inpatient days can range from cases requiring simple interventions and limited monitoring, to highly complex cases where a broad range of material and human resources are required. Differences in case-mix can arise from the differences in the socio-economic characteristics of the population in provider catchment areas, or because more complex cases may seek out providers with particular characteristics. The problem can be at least partly overcome. For example, output can be disaggregated into more detailed diagnostic categories. Alternatively, we can stratify providers by "servicecapability groups" and assume homogeneity in terms of case mix and severity. 22 Finally, in addition to problems relating to the comparability of output measures in specific categories, most health care providers offer a wide range of services (service mix). Even with a small number of aggregated categories of interventions/services-e.g., inpatient days and outpatient visits-the issue arises of how to compare the quantity of outputs across facilities with different service mixes. A standard technique for dealing with multi-output production is to construct an output index using market prices as weights. In general, this approach is not appropriate in the health sector, where output prices do not exist or do not

> 19 See Prennushi, Rubio, and Subbarao (2001).

> 20 For example, if households fail to take advantage of a free child imnmunization program, providers presumably can be blamed for a part of the non-utilization if they fail to publicize it and fail to employ delivery mechanisms that respond to the reasons for non-utilization (e.g., outreach rather than clinic-based services).

> 21 This was pointed out by Feldstein (1967), who found that differences in case mix was an important explanation for variation in ward costs per case among hospitals.

> 22 See Thatchell (1983) for a detailed discussion.

11
reflect some notion of market prices.<sup>23</sup> For certain types of analysis the issue can be avoided. For example, if we are concerned with the estimation of costs, multiple outputs can be included as independent variables. Similarly, for the estimation of a production function, Wouters (1993) suggests as an estimation strategy that output(s) be included as dependent variable(s). For example, the number of outpatients can be regressed on the number of inpatient days and other explanatory variables. In this framework, the number of outpatient visits is estimated _given_ the level of inpatient admissions at a facility. Although convenient, the approach imposes strong assumptions about the production technology.

In practice, most facility surveys have opted to measure output by the units of services actually rendered-i.e., outpatient visits, immunizations. While this may be unsatisfactory in some respects, it is often the only practical option. Moreover, many of the problems concerning the measurement and conceptualization of health facility outputs can, as noted, be overcome if the service data is sufficiently disaggregated. The level of disaggregation will however be largely determined by the way in which patients are registered at facility level and is therefore context specific. There is, however, some choice in the source of output data. Information on service outputs can be collected from either the patient register at facility level, or from administrative records at district or provincial level. In principle, these sources should be consistent. However, due to poor record keeping or incentive effects, there may be considerable discrepancies in practice. For example, results from a recent service delivery survey in Uganda indicate that districts have a tendency to over-report the number of patients seen in the facilities in the district (see Fig. 4).

lFigure 4 - Compaarnson of faciRlty and dAstirAct irecords in Uganda%

> 23 Composite output indices can, of course, be constructed on the basis of _ad hoc_ weights. If the limitations of the resulting output measure are squarely recognized, such output measures can provide a useful means of summarizing information.

> 24 Differences may arise due to low administrative capacity, or because district or facility staff know that the number will not be used or verified. Alternatively, if resources are allocated on the basis of reported patient numbers, or **if** these records are used to define user fee revenue targets against which the facility is held accountable, there may be incentives to over- or under-report patient numbers.

12
<!-- Start of picture text --> 4000<br>3500 --<br>3000- ___ . _-<br>2500<br>* 2000 .__ -__ _<br>*~1500 -- #<br>1000 _ ___ __ _ _ .<br>500<br>0<br>0 1000 2000 3000 4000<br>Outpatients - facility data<br><!-- End of picture text --> _Source: Lindelow, Reinikka, and Svensson (2001)._

# **_Quality_**

Health care quality is a vague and nebulous concept. Yet, however defined, it is a commonly held view that the quality of health services can have a big impact on health outcomes, health related behavior, and patient satisfaction. It is therefore not surprising that many facility surveys have tried to capture some dimensions of quality. In general, the need for the concept of "quality" stems from the considerable heterogeneity of health services, and from the fact that this heterogeneity is often difficult to observe and is not, in general, reflected in prices. For example, while health services differ along some observable dimensions-e.g., input availability, amenities, and clinical and inter-personal skills of health workers-other dimensions of this heterogeneity, such as clinical effort and competence, are harder to observe.

According to a commonly used definition, health care quality pertains to the management of a personal health problem by a health care practitioner (Donabedian 1980). Following previous literature, Donabedian distinguishes between a technical and an interpersonal domain of management, where the first refers to the application of the science of medicine and other health sciences, and the second the social-psychological interaction of the health care provider and the patient.25 In this framework, the basis for making assessment about the quality of technical care pertains largely to professional norms based on a body of scientific knowledge. It is more complicated to evaluate the quality of interpersonal care, but criteria must clearly relate to socially defined norms of acceptability and good practice. In other words, according to this view, quality must always be conceived with reference to some standards or norms, and is inherently a relative concept.

> **25** Donabedian underlnes that the two domains are interrelated and often difficult to distinguish. He also distmguishes the "amenities" offered by the health care provider as a possible element of quality, but considers this part of the

> interpersonal management of an illness episode.

13
Donabedian suggests that the most direct approach to assessing of health care quality is to focus on the "process" of care-i.e., the activities of the health care provider-and evaluate this against established technical, professional, and ethical norms. An alternative is to assess quality indirectly, through an analysis of "structure" and "outcome". Structure refers to the characteristics of the health care provider, and is relevant through the impact of the respective characteristic on the probability of a good process and outcome. Structural dimensions of health care include the tools and resources that health care providers have at their disposal, and the physical and organizational setting in which care is delivered. In contrast, outcome refers to the change in a patient's current or future health status that can be attributed to health care, as well as changes in patient attitudes (including satisfaction), health knowledge, and health-related behavior.

This "structure-process-outcome" trilogy has been influential in structuring both health sector research and operational approaches to assessing and improving the quality of health care in developing countries. Each of the three approaches to assessing quality has strengths and weaknesses. 26 However, the problem of the "trilogy" "apoc approach lies isi in theh tenuous links between different dimensions of quality. For example, quality of structural inputs by no means assures good process. Similarly, the link between process and outcome is often not clear, and may not be visible for a long time. Also, favorable outcomes are often affected by factors not directly under the control of the health worker. This then raises the question of how to weight different dimensions of health care quality in an overall assessment. Are some dimensions more important than others? Are there important complementarities? Gilson, Magomi, and Mkangaa (1995) argue that quality of primary health care in developing countries is best assessed by looking at structure and process dimensions of quality. This is because primary health care is largely based on interventions for which effectiveness has already been demonstrated. In other words, it is more important to know why clients do not attend facilities and why interventions are not properly executed than to know their potential impact on health status.

In terms of empirical work, quality can be defined to encompass all or most aspects of health care. Indeed, the "Quality Assurance Methodology" takes a very broad view of quality, and includes technical competence, access to services, effectiveness, interpersonal relations, efficiency, continuity, safety, and amenities as "dimensions of quality" (Brown and others 1992). This may make sense in an operational context, where the objective is to design practical interventions aimed at improving outcomes, and to promote comprehensive monitoring. However, this all-encompassing notion of quality is less helpful from an analytical perspective, where the emphasis is on coherent modeling of empirical phenomena, and on valid and reliable measurement. Narrowing down the definition of health care quality is however far from easy. For example, Bessinger and Bertrand (2001) describe the difficult process of reducing the number of indicators of quality in family planning services from 200

> 26 In a review of studies of quality of health care in developing countries, De Geyndt (1995) notes that outcome measures are generally absent in the literature. Ackerberg, Machado, and Riordan (2001) provide an exception m their comparison of the performance of different providers in Maine's alcohol abuse treatment program. In contrast, most studies have focused on structural or process indicators (or both). Notably, researchers from the economics discipline have tended to focus on quantifiable and measurable structural variables, while process variables are more often used by medical practitioners.

14
to 25.27 Even after this considerable simplification, the recommended approach requires data collection through consultation observation, a facility audit, and interviews with clients. More generally, facility surveys have measured quality through a combination of three different approaches: inputs, the consultation process, and beneficiary or client perceptions. Although it is not possible to reach full agreement on how to best measure quality, findings from research to date can provide some guidance.

# **Quality measurement through inputs**

The most common approach to measuring the structural dimension of quality is to collect data on the availability and quantity of inputs considered important for the delivery of services. Quality can then be proxied by focusing on particular inputs, for example drugs. Alternatively, composite quality indices can be constructed.<sup>28</sup> Quality data of this kind permit an analysis of the relationship between quality and health outcomes and behavior.<sup>2 9</sup> Observed variation in "structural quality" also raises the question of what explains these differences. While some of these indicators are important and have the benefit of being easy to measure, they are likely to be inadequate as measures of quality.<sup>30</sup>

# **Quality measurement through clinical vignettes**

Good health care quality, as pertaining to the overall management of a health problem, clearly requires more than material inputs. In this regard, clinical skills are of obvious importance, and many surveys have tried to collect data concerning the process of care. Here, the general principle is to assess case management against established diagnostic and treatment norms. The most common ways of assessing "process quality" is through clinical vignettes and consultation observations.<sup>31</sup> In addition, some surveys (e.g., DHS SPA) have asked questions about general facility procedures, for example in respect of sterilization, disposal of needles, etc.

I Clinical vignettes assess clinical skills by presenting health workers with hypothetical cases, and seek to elicit information about the way in which they would handle such a case. This approach has the advantage of permitting a considerable degree of structure in the survey instruments, thus limiting the need for technical skills in the enumerators. It has been used in a number of surveys, including the Jamaica LSMS, the IFLS, and DHS SPAs. A more ambitious approach entails using medically qualified enumerators, who observe the case management of health workers in the respective facilities. While the skill requirements for this type of survey can be prohibitive in many contexts, the approach has the advantage of permitting a more comprehensive assessment of the process of care. For example, beyond the strictly clinical aspects of the case management, the enumerator can assess the social-

> 2 7 The "Quick Investigation of Quality" (QIQ) approach is described **in** Bertrand et al. (Bertrand, et al. 2001).

**28** See, e.g., Gamer et al. (1990).

**29** Evidence from this literature is discussed below.

**30** One reason for this is that we may expect only a weak link between structural dimensions of quality and health outcomes. As noted by Gilson et al. (1993), process quality **is** more likely to be validated in terms of effectiveness than structure.

**31** Consultation observation is often combined with re-examination of the client. It is also possible to carry out ex post reviews of patient records, or chart abstractions. This approach however relies on records that are often inadequate in the case of outpatient care and/or unavailable **in** developmg countries (Peabody, et al. 2000).

15
psychological interaction between the health care provider and the patient. This can include the general attitude and courtesy of the health worker, but also communication of information about treatment regimes, medication, return visits, etc. On the other hand, direct observation of patient-provider interactions may suffer from bias due to the behavioral response of the health worker to being observed. In addition, differences in case mix across providers can confound the assessment of quality. Observation of case management has been a feature in surveys such as the WHO multi-country evaluation of IMCI, SPAs, and Situation Analysis.

What is the best approach-clinical vignettes or consultation observation? Either approach has its advantages. Clinical vignettes are easy to administer, and do not necessarily require enumerators with sophisticated medical training. Consultation observation, on the other hand, provides more direct evidence from a clinical setting and permits observation of non-clinical aspects of the provider-client interaction. That said, the presence of an observer may lead the provider to adjust his or her behavior. Assuming that the provider effectively applies clinical knowledge in case management, and that observation bias is not significant, we would expect a close correspondence between quality measures from vignettes and clinical observations. There is indeed some evidence from the US that this is the case. Peabody and others (2000) report evidence that vignette scores in an outpatient setting appeared to reflect physician practice as recorded in observation episodes. On the other hand, it is not clear that these findings can be generalized. As noted by Peabody and others, the costs-in terms of time and effort-of complying with diagnostic and treatment procedures in a clinical vignette are negligible. This is not the case in a clinical setting. As a result, we would expect vignettes to be biased in favor of high quality in a context where external and/or internal incentives for compliance are low. Reflecting this concern, Leonard et al (2000) report evidence from Tanzania that while most clinicians knew how to deal with certain tracer conditions (on the basis written examination), many did not actually do the right thing when observed in a clinical setting.

QualIty measurement througdh beneflciiry perceptonM The assessment of quality through structural and process characteristics is premised on a set of established norms for clinical effectiveness. An alternative approach to assessing quality of care is to ask beneficiaries about their perceptions. Patient satisfaction is important in its own right as an objective of health care, but also as an indicator of the structure, process and outcome of health care, and through the effect of satisfaction on client behavior. Patient satisfaction is typically assessed through interviews with actual or potential patients. Data on actual patients can be collected through exit polls, whereas a sample of households or individuals in the community must be surveyed in order to capture the views of non-users.<sup>32</sup> Multi-topic household surveys with added facility modules, such as the LSMS and CWIQ surveys, have typically not collected detailed data on patient perceptions, whereas exit polls with sections on patient perceptions have often been a feature of more focused health surveys (e.g., WHO MCE, SPAs, and Situation Analysis surveys).

There is some evidence to suggest that observations and client exit interviews provide similar results for many (family planning) quality indicators (Bertrand, Sullivan, and Rice

> 32 Data can also be collected through focus group discussions. This option is not discussed here.

16
2001). However, it is also known that there are considerable problems in interpreting subjective perceptions of health care quality. In part, this is due to "courtesy bias", such that individuals may provide responses that they deem socially acceptable.<sup>33</sup> However, it is also the case that subjective perceptions of health care quality depend on both needs and expectations, which in turn are underpinned by the explanatory models held by different individuals regarding the functioning of the body, etiology, and the appropriateness of different remedies and interventions in response to a given symptom (Atkinson 1993). There may be important systematic differences across demographic and socio-economic groups in respect of these explanatory models, possibly making client perceptions poor proxies for objective assessments of different dimensions of quality.

There are two other approaches relating to client perceptions and satisfaction that, despite not relying strictly on facility surveys, deserve mention in this context. First, following the publication of the 2000 World Health Report (WHO 2000), the concept of health system responsiveness has gained increased currency. The concept of responsiveness is different from patient satisfaction in that it is seen to pertain solely to non-health enhancing aspects of the health system, and in its focus on health systems rather than individual providers (de Silva 1999). In addition, the methodology for measuring responsiveness seeks to overcome some of the perceived weaknesses of measures of patient satisfaction, in particular in respect of relativity arising from differences in expectations and experiences.<sup>3 4</sup> While the methodology is far from uncontested (see, e.g., Williams 2001), the framework may contribute to improved measurement of client satisfaction in the context of facility surveys.

Second, there is a host of approaches that seek to gather "community-based evidence" on service delivery. This includes service delivery surveys, "report cards" on public services, and other related approaches.<sup>3 5</sup> These approaches cannot in general be described as facility surveys, nor are they always limited to the health sector. Rather, the purpose is to use a range of different tools - e.g., household interviews, focal groups, institutional reviews of service providers, interviews with service providers - to elicit information about awareness and experiences of services, and opinions about prices, quality, waiting times, courtesy, etc. Despite their simplicity, the approaches can usefully inform the formulation of hypotheses about service delivery, as well as the design of quantitative surveys. Moreover, they have proved effective (at least from a monitoring and advocacy perspective), and may in the future complement and support facility surveys.

**33** For example, Bitran (1995) found that regardless of facility scores on quality of diagnostic and treatment norms, and availability of drugs and supplies, patients generally reported being satisfied with the treatment they received, and mdicated that they would return for future treatment. Many other studies have noted an inconsistency in responses, where high levels of overall satisfaction are combined with widespread complaints about specific aspects of care or the health care system. With a view to reconcile these findings, it has been suggested that positive and negative comnments about health care are typically not reducible to satisfaction and dissatisfaction, and a more careful methodology is required to collect meaningful qualitative data on patient perceptions (Atkinson 1993).

**34** The different aspects of responsiveness considered in the WHO methodology are: dignity, autonomy, confidentiality, prompt attention, quality of basic amnenities, access to social support networks during care, choice of care provider (de Silva 1999).

**35** For service delivery surveys, see e.g., Cockcroft et al. (1999) and CIET international (1996). See also www.ciet.org. For report cards, see, e.g., Paul (1992; 2000) and World Bank (2001a).

17
_99affRercejgoins cand 1bevav or_ Notwithstanding the obvious importance of health worker behavior and motivation for both quality and efficiency, there are currently few examples of surveys that focus on<sup>these</sup> issues.<sup>36</sup> McPake and others (1999) comprise an important exception. They collect data on the informal economic activities of health workers in Uganda, where informal economic activities are defined as "...activities which yield income which fall outside<sup>official dutiesand</sup> earnings." On the basis of this definition, the study focuses primarily on drug and user charge leakage.<sup>37</sup> The survey team spent a full month in a small sample of facilities in order<sup>to permit</sup> health workers to get used to their presence. They use wide range of survey instruments, explicitly designed to address the sensitive issues involved. Structured interviews were carried out with both health workers and community members. In addition, key informant interviews and focal group discussions were perforrned to probe certain issues in greater detail. Finally, detailed record reviews and checklist instruments were administered at different<sup>stages to</sup> collect information on daily patient attendance, drug stocks and use, number of prescriptions, user charge and revenue information, expenditures financed by user charges, etc. Through<sup>the</sup> "triangulation" of information from different sources, it was possible to verify and crosscheck information on construct estimates of overcharging and the divergence between the amounts of drugs supplied to the facility and actually prescribed.

From a different perspective, Bennet, Gzirishvili, and Kanfer (2000) collect subjective data on health worker motivation and performance in hospitals in Georgia. Extensive research on this issue has been carried out in developed countries, in particular in non-health sectors. More recently, efforts have been made to design an appropriate methodology for developing countries (Bennet and Franco 1999; Kanfer 1999). It emphasizes motivational determinants<sup>at</sup> both individual, organizational (work context), and socio-cultural levels. Data are collected through contextual analysis and structured interviews, where individual assessments were designed to assess the determinants and outcomes of motivational processes. Specifically, self-administered questionnaires were completed by a sample of both health workers and supervisors. The questionnaires were based on psychometric scales used for studying<sup>work</sup> motivation in the US, which focus on individual perceptions about themselves, their work environment, and performance.

Both these studies are based on small samples and survey staff in considerable detail. They illustrate the inherent difficulty involved in measuring perceptions, behaviors, or outcomes. This is particularly the case when there are strong rules or norms prescribing or proscribing certain perceptions or behaviors. Yet, both studies point at the potential value of focusing on human resource issues in the health sector, either through in-depth studies and expanded facility surveys.

> 3 6 There is however some recent interest in the behaviour, motivation, and perceptions of public officials in general. See, e.g., Manning et al. (2002).

> 3 7 Drug leakage can arise from recording of ghost patients; recording of prescriptions for real patients that did not actually receive any drugs; removal of drugs from drug store without recording. User charge leakage can arise from nonrecording of revenues, or from a divergence between recorded revenues and expenditures that is not accounted for.

18
_The institutional and organizational environment_ Many facility surveys have found considerable variation in facility performance. Yet, there are few examples of surveys that have looked beyond the facility, to collect data on the determinants of performance, including the institutional and organizational environment in which the facility is operating. In large part, this reflects the difficulty in measuring both performance and its determinants, and in establishing an empirical relationship between specific determinants and chosen measures of performance.

Performance measurement of private enterprises is relatively straightforward in most sectors. "Firm value"-i.e., the present value of the future stream of profits-provides a unified criteria which captures both efficiency and quality. In contrast, performance measurement in much of the public sector is fraught with difficulty. In part, this is due to the fact that neither input nor output prices can be assumed to reflect "market prices". But there are also deeper issues, relating to the meaning of performance in the different spheres. In the case of the health sector and many other areas there the state has a prominent role, "society" has particular distributive concerns. In consequence, performance cannot be assessed by exclusive reference to some notional "bottom line".<sup>38</sup> In practice, this often leads to the application of several parallel criteria in the assessment of health sector performance, including allocative and technical efficiency, quality of care, and equity.<sup>39</sup> An even greater difficulty arises in identifying and measuring determinants of performance. In general, there are several features of this operational environment that can be expected to be important. First, both government and private facilities are generally subject to some mechanisms for oversight and accountability. This includes issues such as the hiring and firing of staff, the allocation of decision rights between facilities and district or provincial administration, sanctioning devices against private facilities that do not comply with government regulation, and mechanisms for community participation and redress (for example through facility management committees). There are also important issues to explore in relation to financing and support systems that operate "upstream" from the facilities. Government facilities, and in some cases NGO facilities, depend on district and provincial administration to provide them with salaries and material inputs. Finally, there are a number of factors that are internal to the facility-e.g., the characteristics of the facility managerthat may be important as determinants of facility performance.

Currently, there is little systematic empirical evidence on the impact that different features of the operational environment has on facility performance.<sup>40</sup> However, some recent surveys have made inroads into this area. For example, Somanathan and others (2000) hypothesize that differences in costs across facilities in Sri Lanka are related to differences in the way that these facilities were managed. To analyze this issue, they collect data on

> 38 This is based on the idea that performance by the extent to which an agent (m this case a health facility or the health sector as a whole) fulfils the objectives assigned by the agent ("society") (Pestieau and Tulkens 1993).

> 39 Sometimes other dimnensions of performance are included, e.g., effectiveness and accessibility (see Goddard, et al. 2000). Depending on the role of private providers m the health sector, different criteria for performance may be appropriate.

> 40 There is little guidance in the literature on how to best measure different aspects of the institutional and organizational environment. Price and Mueller (1986) offer a general review of different areas of organizational measurement. Over and Watanabe (2000) propose a number of indicators for organizational environment, organizational behavior, and outcomes.

19
characteristics of the managers, the systems they use to manage their key inputs, community support, competition, etc. Angeles and others (1999) address related issues in Paraguay by comparing performance (costs) before and after decentralization. A recent health facility survey in Uganda (Lindel6w, Reinikka, and Svensson 2001; Reinikka and Svensson 2000a) provide a detailed description of the complex relationships between facilities and the district administration. In this case, data concerning financing, support and supply systems, and supervision arrangements were collected both through interviews with both facility staff and district administrators. Related to this approach, public expenditure tracking studies have been implemented in a number of countries. Although the focus of these surveys has varied, the general principle has been to collect data on resource flows from central level to the service delivery unit, as well as on institutional arrangements and characteristics. Depending on the context and survey design, this permits not only an assessment of whether actual distribution is consistent with intended allocations, but also on how capacity and system weaknesses, information asymmetries, and local capture can impact on the budget execution process.41 These examples represent attempts to understand better what factors explain observed differences in performance by trying to measure variation in the institutional and organizational environment. They demonstrate the value in trying to look beyond the facility, but also illustrate the considerable conceptual and measurement challenges involved in moving this agenda forward.

RV. How Facefty Daft Haave Beeimn Used

# _lPlsanning and bdgefin_

Planning and budgeting in the health sector concerns decisions about the use of resources in the future - for example, in respect of mix of services or the location and means of provision. Although an inherently political process, information is a fundamental input in all the stages of planning and budgeting, including situation analysis, priority-setting, option appraisal, programming, implementation, and monitoring and evaluation. Much of this information will be generated by routing health information system. However, facility surveys can provide valuable non-routine information to support decision making.

First, facility surveys can provide a detailed picture of resource availability and adequacy and costs at facility level. The premise in assessing "resource adequacy" is that a certain range, quantity and quality of inputs is required in order to provide the intended service package. In this sense, resource or input adequacy can be assessed by analyzing how resource or input availability relates to resource requirements. The resource requirements are a function of the package of services that the respective facility should be delivering, as well as to the number of clients. Hence, in order to make a meaningful assessment of resource adequacy, a careful estimate must be made for each type of input on the basis of information about the service range and the activity level of the facility in question. For example, an assessment of staff adequacy may be based on a minimum standard for any facility, and on the patient load for staff (Hanson and Gilson 1996). Similarly, there are established methods for estimating drug requirements (WHO 1988). In addition to looking at specific inputs,

> 41 Evidence from PETSs are discussed below.

20
composite indicators can be constructed to provide information on the capacity of the health facility to perform basic clinical tasks. This type of exercise can focus on defined "tracer conditions".<sup>4 2</sup> In general, this type of information can be used to estimate total resource needs and to support decisions about the allocation of resources within the sector. In principle, surveys can be used to estimate unit costs for the delivering different types of services in different contexts. This could then form the basis for calculating total resource needs to meet certain service standards, and for making decisions about the allocation of resources within the sector. In practice, however, surveys may not be the most appropriate tool for costing basic service packages.<sup>43</sup> However, survey information on the availability of human and material resources at facility level can point at egregious imbalances in how resources are allocated across inputs.

Second, it should be noted that while resource inadequacy at the level of health care facility is likely to be the consequence of a general shortage of resources in the sector, these problems may be aggravated by weaknesses in the budget execution system, including systems of procurement and distribution of in-kind resources. For example, Public Expenditure Tracking Surveys have shown how delays and leakage in budget execution can result in budget outturns that depart considerably from intended allocations.44 In general, surveys can offer a perspective on support systems and administrative routines and procedures that is not otherwise available.

# _Monitoring, evaluation, and accountability_

A number of survey initiatives have for many years served as bases for continuous assessment of health status and living standards more generally. These include household surveys such as the Demographic and Health Surveys (DHS), the Multi-Indicator Cluster Surveys (MICS), and the Living Standards Measurement Study (LSMS). Data from these surveys have proved a useful supplement to data gathered from surveillance sites, such as the ICDDRB Centre for Health and Population Research in Bangladesh<sup>4 5</sup> and the Indepth Network<sup>46</sup> , as well as from more traditional sources, such as censuses and vital registration systems. The attraction of household surveys is that they can be stratified by, for example, gender, location, ethnicity, and household living standards.<sup>47</sup> In addition, surveys provide

> 42 Most facility surveys have taken a "checklist approach" to assessing resource availability, focusing on the availability of certain types of equipment, staff, drugs and other inputs. However, in the analysis of input data, the availability and quantity of inputs are often not related to output vanables, making it difficult to make any statements about the _adequacy_ of resources. There are only few examples of surveys that have used tracer conditions. One example is a survey of rural health facilities in Papua New Guinea, aimed at assessing the "potential quality" of each unit (Gamer, et al. 1990). In this case, a range of items on a checklist was scored to construct a "quality indices", reflecting the capability of each facility to perform certain common clinical tasks. The study analyzed differences in resource adequacy across, _inter alia,_ agency types and regions.

43 Comprehensive costing is typically beyond the scope of most surveys. In addition, survey based costing is likely to result in considerable variation in unit cost across unit. This variation may have many explanations, including differences in efficiency, quality, case mLx, and input prices.

44 Results from these surveys are discussed in greater detail below.

45 htto://www.icddrb.org 46 htto:H/www.indenth-network.or2/ 47 See htto://www.worldbank.org/novertv/health/data/index.htm for a 42-country study that disaggregates various key MCH outcomes by poverty status.

21
more representative information that data collected routinely at facility level in<sup>contexts</sup> where access to health care is limited.

Similarly to surveys carried out at household level, semi-standardized surveys of health care providers - for example, Service Provision Assessments (SPAs)<sup>or Situation</sup> Analysis (SA) - have provided valuable assessments of health care quality, infrastructure, utilization and availability. For example, a recent survey in Kenya (Macro International<sup>2000)</sup> found that although contraceptive services, drugs and materials are widely available, compliance with infection control procedures, including sterilization of<sup>equipment and</sup> sterilization of needles was often poor. Moreover, consultation observations revealed<sup>that the</sup> diagnostic process was often unsatisfactory, and that clients were frequently provided<sup>with</sup> incomplete or insufficient information to chose between interventions and<sup>to ensure</sup> compliance with treatment regimes. Aside from providing data for research on the determinants of health outcomes and health-related behavior, this type of<sup>informationis</sup> valuable in the design of health sector strategies and interventions. Moreover, repeat<sup>surveys</sup> permit the monitoring of changes in quality over time.

In other cases, household and/or facility surveys are implemented as components of the monitoring and evaluation of specific projects or programs. As noted by<sup>Mills (1997),a</sup> broad array of health sector reforms have been proposed in recent years, notwithstanding<sup>the</sup> fact that there is currently little empirical evidence to vouch for their effectiveness.<sup>In this</sup> context, efforts aimed at continuous assessment and ex post evaluation are of particular importance. Under ideal circumstances, this will entail (i) a baseline survey; (ii) subsequent policy change in a random sub-sample of units; and (iii) a follow-up survey of<sup>both treatment</sup> and control facilities. However, in many cases evaluations have to be carried<sup>out under less</sup> than perfect conditions.

There are several examples of surveys that have been implemented in the context of project or program evaluation. For example, two household and facility surveys have been carried out in Uganda (1995 and 1997) as components of the ongoing DISH project (reproductive health). The surveys have been implemented to measure changes in reproductive, maternal and child health knowledge and behavior (Kantende and others 1999). An ongoing Multi-Country Evaluation (MCE) of the Integrated Management of Childhood Illnesses (IMCI) comprises another example of the role that facility surveys can play in the monitoring and evaluation of health sector interventions. Worldwide there are 30 countries<sup>at</sup> different stages of implementation of IMCI, among which Uganda, Peru, Bangladesh and Tanzania will participate in the MCE. The purpose of the MCE is to document<sup>theeffects of</sup> IMCI interventions on health workers performance, health systems and family behaviors; determine the extent of measurable impact on health outcomes; and, describe costs of implementation (WHO 2001). Health facility surveys also formed part of the evaluation strategy for the Bolivian Social Investment Fund (SIF), which was established in<sup>1991to</sup> direct public service investments to poor communities. Surveys were collected in 1993 and 1997, and included households, communities, and service delivery units. The surveys demonstrated clear improvements in infrastructure and equipment, as well as<sup>increased</sup>

> 48 In many cases these efforts are ongoing. It is therefore difficult to determine how successful this approach has been in analyzing the determinants of facility performance. Many of these surveys are discussed in greater detail in the appendix.

22
utilization rates and reduced mortality (Newman and others 2000). As a final example, a survey was carried out in Paraguay in 1998 with a view to identify and quantify the effect of transferring management control for basic health services provision from central to municipal government on costs, efficiency, basic health service quality, and patterns of health service use and equity. The study uses pre- and post-decentralization design, with a control group, and focused on family planning, maternal health, and infant and child health (Angeles and others 1999).49 Monitoring information can also serve to strengthen accountability in the delivery of services. This can include the use of survey methods both to shed light on the systems through which budget resources are transformed into services, as well as to give users an opportunity to express their views on public services. The World Bank's Public Expenditure Tracking Surveys (PETS) and Quantitative Service Delivery Surveys (QSDS) provide examples of the former type of surveys. The focus of these non-standardized surveys have varied. On the one hand, some surveys have been concerned with the actual flow of resources-in financial or real terms-from central level administration to the actual service delivery unit. Indeed, it was in response to these concerns that the first Public Expenditure Tracking Survey (PETS) was designed and implemented in Uganda in 1996 (see Ablo and Reinikka 1998; Reinikka 1999). This survey asked why it was that service delivery outputs in the health and education sectors had failed to respond to considerable increases in public spending. Data problems were severe, particularly in the health sector (a finding of some interest in its own right). Yet, the survey provided powerful evidence of how weak governance and lack of accountability can hamper the process of budget implementation. For example, on average only 13 percent of the total yearly capitation grant from the central government reached the school in 1991-95. Eighty-seven percent either disappeared or was used for purposes unrelated to education. Interestingly, the access to funds varied in systematic ways with school size, income, and the extent to which teachers were qualified.<sup>5 0</sup> Information asymmetry was seen to be the primary factor behind these findings. In response to the survey results, the government began publishing the monthly transfers to the districts in newspapers, broadcasting on radio, and requmrnf primary schools to post information on received funds in public places in the school. 5 A number of tracking surveys with at least partial focus on the health sector have been implemented in other countries.<sup>5 2</sup> From these experiences, it is clear that there is no standard approach to doing a PETS, largely because the administrative and logistical systems on which

> 49 A second round of data collection is planned, but has been delayed due to slower than expected implementation of decentralization.

> 50 In the health sector records were poor or non-existing, and little quantitative data were collected. On the basis of interviews, it was however estimated that on average, almost 70 percent of medical supplies and drugs were expropnated by staff. Similar findings are reported by McPake et al. (1999)

51 Follow-up surveys have documented a considerable increase in the proportion of funds received by schools. This can at least in part be attributed to improved access to information, although other factors are also likely to be important.

52 Surveys in Tanzania looked at the allocation of resources in priority sectors at different levels of administration, and at delays and leakages in the transfer of resources (Price Waterhouse Coopers 1999; Research on Poverty Alleviation (REPOA) and Economic and Social Research Foundation (ESRF) 2001). A survey in Ghana sought to measure actual expenditures (including in-kind transfers) on basic education and primary health care to estimate the leakage of public funds in the transfer process from central government via districts to facilities (Ye and Canagarajah 2001). A survey in Honduras looked at ghost workers, absenteeism and "job-migration" in both health and education sectors (World Bank 2001b).

23
the facilities depend are highly complex and context specific. However, the studies have demonstrated the value of collecting data "upstream" from the facility. Specifically, the PETS have helped to show how inequities, inefficiencies and resource inadequacies observed at facility level are the product not only of low levels of financing, but also of the capacity constraints, incentives, and information asymmetries that characterize the systems upstream from the facility.

In other surveys the focus has been on the service delivery unit, although with an eye on the relationship between the facility and the administrative system upstream. This is in recognition of the fact that even if budget outturns reflect allocations, ultimate impact also requires that financial resources are efficiently transformed into appropriate services.<sup>53</sup> This can include facility surveys, but also citizen surveys or scorecard approaches. For example, the original "Bangalore Scorecard", designed and implemented by the Public Affairs Centre (PAC), a local NGO, questioned a sample of both rich and poor users on their perceptions of the city's public services. The survey was used to rate different agencies in terms of staff behavior, quality of service, and information provided, and proved a powerful tool to generate feedback on public services, identify weaknesses in service delivery, and advocate for change (Paul 2000).

Both tracking surveys and service delivery surveys are increasingly used as monitoring and accountability mechanisms in contexts where routing management information systems are weak. Indeed, Uganda intends to place the conduct of regular service delivery surveys at the center of its strategy for improving public services, with baseline surveys to establish strategies and targets, and public dissemination of survey findings.<sup>5 4</sup>

# _Research_

Qualaity and health outcomes As data on health care quality has become increasingly available, the body of empirical research on the link between health care quality and health outcomes has grown. There is now evidence that certain characteristics of health care providers and their relationship with clients are important determinants of health outcomes. For example, a number of studies have found a positive correlation between health service infrastructure and characteristics, such as the number of clinics or doctors per capita, and health indicators, including child mortality, fertility, nutritional status and anthropometric indicators (see e.g., Benefo and Schulz 1994; Frankenberg 1995; Frankenberg and Thomas 2001; Hossain 1989; Lavy and others 1996; Rosenzweig and Schultz 1982; Rosenzweig and Wolpin 1982; Thomas, Lavy, and Strauss 1996). There is also a literature based on DHS data, focusing specifically on family planning and contraceptive use (see, Beegle 1995; Feyisetan and Ainsworth 1996; Mroz and others 1999; Steele, Curtis, and Choe 1999).

> 53 These surveys have been referred to as Quantitative Service Delivery Surveys (QSDS) (Dehn, Remiikka, and Svensson 2000).

> 54 Government of Uganda, Letter of Development Policy for World Bank Poverty Reduction Support Credit, 2001.

> 55 See Wouters (1991), Frankenberg (1995), and Alderman and Lavy (1996) for discussion of the evidence.

24
There is plenty of survey evidence of low quality in the process of care. This relates both to clinical (clinical knowledge, adherence to diagnostic and treatment protocols, etc.) and non-clinical (inter-personal relations) aspects of the process. There is, however, less evidence on he effect of different aspects of the process of care on health outcomes and behavior. An exception in this regard is Peabody, Gertler, and Liebowitz (1998), who use the LSMS Jamaica data to study the relationship between quality and birth weight. While none of the structural measures of quality had a statistically significant effect, they found that women who had access to complete examination (process) had infants that weighed an average of 128g more at birth.

# **Quality and health seeking behavior**

One of the conduits through which health care quality affects health outcomes is client behavior, including both care seeking behavior (utilization of health services) and adherence behavior (i.e., compliance with treatment regimes, follow-up visits and referTals). However, concern with quality was limited in early studies of health care demand. 56 To the extent that health care quality was considered at all, it was treated as an unobserved variable, pertaining to the provider type rather than the individual provider (e.g., Gertler, Locay, and Sanderson 1987; Gertler and van der Gaag 1990). More recently, there have been studies that have used cross-sectional data to examine the effect of a series of structural quality variables on healthseeking behavior.<sup>57</sup> In some of these studies, multicollinearity across quality characteristics and service type prevented the examination of independent impact of each characteristic. Generally, the studies demonstrate a significant and sometimes large statistical correlation between quality and utilization. However, as Gertler and Hammer (1997) note, with the exception of the experimental study by Litvack and Bodart (1993), this may reflect the effect of utilization on pricing and quality policy rather than the other way around.

Researchers have also used facility and household data to study the phenomenon of bypassing - i.e., when patients reject a closer facility in favor of a more distant health care provider. There is only a limited literature on this issue, mostly focusing on the case where households bypass public facilities in favor of the private sector (Akin and Hutchinson 1999; Leonard, Mliga, and Mariam 2000). In large part, this reflects the complexity of the data required to analyze this aspect of health seeking behavior. In general, studies have found that bypassing is associated with facility quality (e.g., number of doctors, drug availability,

> **56** Several contributors (e.g., Akin, et al. 1984; 1986; Litvack and Bodart 1993) have noted that excluding quality from

> the purview of analysis is problematic on methodological grounds. Higher price may simply reflect higher quality, and, insofar as clients are willing to pay for quality, the failure to control for it in estimation of health care demand will result in

> underestimation of the price effect.

> **57** Mwabu and Ainsworth (1993) consider drug availability and staffing; Lavy and Germain (1994) measure the quality of various provider types through drug availability, staffing, infrastructure, and service range; Akin et al. (1995) use operational cost per capita as reported by facility as the preferred proxy for quality, but also consider measures relating to the

> physical conditions of facilities and availability of drugs (percentage of year) to control for differences in efficiency. Litvack

> and Bodart (1993) focus on drugs as the primary component of quality. Although the scope of the study is narrow, **its**

> methodological strength is due to the experimental design.

25
opening hours). However, the studies have also demonstrated that care seeking behavior is often a sophisticated response to the type and severity of illness of the client.' Efriciency ainalysis sand the economnetnric esm tion of coqt unncfons Cost functions can be useful in shedding light on a number of questions. Are health facilities under- or over-capitalized? Are facilities inefficient in other respects? Are they technically inefficient in that they fail to produce the maximum output from their inputs? Do they use inputs in the wrong proportions? Do facilities vary in their degree of inefficiency? Are public facilities less efficient than others? Should facilities specialize or provide a broad range of services? Could costs be reduced by concentrating cases in fewer facilities?

In the industrialized world, there is a large literature on these topics. A much smaller literature exists in the developing world. Wagstaff and Barnum (1992) review the four principal studies up to 1992 (Ethiopia, Kenya, Nigeria and Peru). Barnum and Kutzin (1993) report estimates of cost functions for hospitals in Colombia and China. The published literature during the last 10 years includes a large volume of cost function studies from industrialized countries but apparently none from the developing world. Some of the studies (e.g., Nigeria, Colombia, China) that have been done in developing countries-in contrast to those undertaken in the industrialized countries-have relied on survey data, which is unsurprising in view of the poor quality of data in most developing country health facilities (Lewis, La Forgia, and Sulvetta 1996; Mills, Kapalamula, and Chisimbi 1993).

Few of the studies to date shed light on the issue of whether health facilities have too little or too much capital. The test proposed by Cowing, H[oltman and Powers (1983) is to see whether the slope of the variable cost function with respect to capital is equal to the price of capital. Wagstaff and Barnum (1992) suggest an alternative test similar to Feldstein's (1967), namely to test the hypothesis that the slope of the short-run total cost function with respect to capital equals zero. Anderson's (1980) study of Kenyan hospitals is inconclusive on the overcapitalization issue. The results of the studies from Ethiopia (Bitran-Dicowsky and Dunlop 1993) and Nigeria (Wouters 1993) are consistent with over-capitalization, but Wagstaff and Barnum warn against taking the results too literally.

In many studies the primary focus has been on efficiency. Broadly speaking, the concept of efficiency pertains to the relationship between inputs and outputs.<sup>5 9</sup> In modem efficiency measurement it is customary to distinguish between technical and allocative efficiency (Farrel 1957).6? Technical efficiency refers to the maximization of output with a given set of inputs. In contrast, allocative efficiency reflects the substitution between inputs with different prices to achieve minimum costs. These two measures can be combined to provide a measure of total economic efficiency. However, applying the concept of allocative

> 58 The welfare implications of bypassing are not immediately obvious, but it is clear that under some circumstances, bypassing can lead to low utilizations of available pubhc facilities (resulting in low efficiency) and impose unnecessary costs on the clients.

> 59 In the broadest sense, efficiency can be seen to concern the relationship between inputs and _outcomes._ Using this broad concept of efficiency would however require data that are not typically available.

> 60 The efficiency concepts were originally developed in relation to firm performance. See Fned et al. (1993) for a comprehensive treatment.

26
efficiency in the public sector is often fraught with methodological difficulties.<sup>6</sup> 1 In consequence, many studies of efficiency in this context restrict attention to technical efficiency.<sup>6 2</sup> Much of the recent literature on efficiency of health care providers in the industrialized world has employed a statistical frontier model (see Dor 1994; F0rsund, Lovell, and Schmidt 1980; Li and Rosenman 2001; L6pez Casanovas and Wagstaff 1996; Rosko 1999; 2001; Vitaliano and Toren 1994; Wagstaff 1989b; Zuckerman, Hadley, and lezzoni 1994). The traditional cost function links costs to outputs, input prices, and-in the shortrun-capital stock. The cost frontier model extends this by introducing a term that represents the excess above the minimized cost, given the facility's outputs, input prices, and so on. Kutzin and Barnum use a frontier model to test for inefficiency in their sample of Chinese hospitals, but find none. Dor (1987), in his study of Peruvian hospitals, does not use a frontier model, but includes in his cost function a dummy variable to indicate whether the hospital is operated by the Ministry of Health (MoH) or the Social Security Institute (SSI). His findings suggest that the MoH hospitals are more efficient than those operated by the SSI. Inefficiency in a cost frontier model can be due to technical inefficiency, or input-mix inefficiency, or both. Wouters (1993), in her study of Nigerian facilities, explores the issue of input-mix inefficiency. She finds evidence in her sample of under-employment of health workers relative to non-health workers, and that inefficiency is greater in the private sector.

Another important issue confronting policy-makers is whether hospitals should be encouraged to specialize or to provide a broad range of services. Should hospitals, for example, provide both inpatient and outpatient care? Should hospitals providing inpatient services aim to treat most case types, or should they specialize? This is the issue of economies of scope-whether costs can be reduced by producing two or more products jointly. In many studies of health facilities, the functional form used in the regression analysis is too restrictive to explore this issue. Of the four studies surveyed by Wagstaff and Barnum (1992) only one employ a specification that is sufficiently general not to prejudge the issue of economies of scope. Both Anderson's study of Keynan hospitals and Wouters' study of Nigerian facilities employ a multiproduct Cobb-Douglas production, which, as Baumol, Panzar, and Willig (1982) note, implicitly assumes cost anti-complementarities and hence assumes diseconomies of scope unless there are sufficiently strong offsetting fixed costs. Dor's (1987) specification, in his study of Peruvian hospitals, is less rigid but is consistent with economies of scope only in the implausible case where the marginal cost of an outpatient visit is negative. Only in the Bitran-Dunlop study of Ethiopian hospitals is the model specification sufficiently general not to prejudge the issue of economies of scope. The results imply mild economies of scope-a result that is similar to that reported by Kutzin and Barnum in their analysis of Chinese hospitals.

> 61 First, the choice of inputs is often beyond the control of the individual facility individual facility (at least m the public sector). where there is discretion, price signals may be weak. Second, cost minimization may not be an appropnate behavioral assumption. For example, staff allocation may be driven by a policy to ensure minimum service standards. As a result, we should not necessarily expect to observe allocative efficiency.

> 62 In cases where the allocative efficiency in the health sector has been addressed, this has typically been done by comparing the ratio of marginal products to the ratio of staff remuneration for the respective categories. Marginal products are calculated on the basis of an estimated production function with, _inter alia,_ different categones of staff as its arguments (Goldman and Grossman 1983).

27
A further important issue facing policymakers in developing countries is whether the current number of facilities should be increased or reduced to cope with the existing workload. This depends on how far economies of scale have been exploited-the conventional wisdom being that unexploited economies of scale imply too many producers. In the multiproduct case (where, for example, hospitals are treating inpatient and outpatient cases), one needs to work with the concept of _ray_ economies of scale, which show how costs change when all outputs are increased in the same proportion. Ray economies of scale depend in part on product-specific economies of scale, and in part on economies of scope. Two of the studies reviewed by Wagstaff and Barnum (1992) prejudge the issue of economies of scale. Dor's (1987) specification, for example, implicitly assumes that the product-specific economies-of-scale index is equal to one for all outputs. Given this, the link between ray economies of scale, economies of scope and product-specific economies of scale, and that the specification in effect rules out economies of scope, it follows that the specification forces ray diseconomies of scale. Bitran-Dicowsky and Dunlop (1993) find slight product-specific diseconomies of scale but find ray economies of scale. The implication is that these stem from the economies of scope noted above. Wouters (1993) also finds ray economies of scale but in contrast to Bitran-Dicowsky and Dunlop finds product-specific economies.

# V. CGIindl olln$

This paper has reviewed facility survey approaches. From the foregoing discussion it is clear that "facility surveys" have varied extensively-in terms of motivation, type of data collected, methods of data collection, and use of the data. Some surveys have focused specifically on the health care provider, which receives resources, delivers services to the community, interacts with other providers, and reports on activities and outputs. In other surveys, the primary object of analysis has been the household, which is the ultimate beneficiary of the services, who may make payments to the provider, and who may have a role in holding the provider accountable. Finally, some surveys have sought to shed light on the administrative and logistical system "upstream" from the service delivery unit. This system is responsible for channeling resources to the service delivery unit, and provides the institutional framework for regulation, supervision, provider payment, etc.

What are the lessons that emerge from the experiences to date, and what should be the empirical agenda for the future? One point to emerge is that since many of the issues of interest concern the interaction between different units in the system, it is necessary to look beyond a single part of the system. Indeed, this is precisely the rationale behind the first community and facility surveys implemented in conjunction with the DHS and LSMS surveys. However, while these surveys have taught us a lot about how health behaviors and outcomes relate to service availability and provider characteristics, many questions remain unanswered. This is true not only for the relationship between households, the community, and the provider, but also in respect of the strategic interaction between different providers, and between the administrative and logistical system and providers.

Another point to emerge is that there are difficult methodological issues to resolve concerning the measurement of quality, costs, and outputs. For example, the paper has noted discussed the disparate approaches to measuring health care quality. Is there an emerging best 28
> practice? Will it be possible to design simple ways of collecting meaningful and consistent data on quality? Is it even reasonable to view health care quality as a facility characteristic? Actual client experience may be endogenous to client characteristics and previous client

> behavior. For example, both structural (e.g., attending staff, access to equipment and drugs) and process (e.g., staff attitudes, time spent with the patient) characteristics of care may depend on client characteristics such as income, social standing, education, assertiveness of the client, or on how much was paid. This would mean that different individuals do not only perceive their visit to the health care provider differently, but also receive different treatment. Progress in the measurement of quality is not only important for the purpose of monitoring, but also for increasing the value of research on health outcome and health related behavior. To date, the disparate approaches to measuring health care quality, and the sometimes inconsistent findings, largely reflect differences in data availability and quality. Similar methodological concerns exist in respect of the measurement of costs and outputs.

Surveys have been better at documenting variation in facility performance-both in terms of quality and efficiency-than in explaining differences across facilities and countries. We still do not know enough about the possible merits of private and NGO providers, or about whether general prescriptions can be made in respect of the institutional context in which public providers operate. There are a number of instructive examples of surveys that have focused on detailed performance measurement, including concerning staff perceptions and behavior. Some of these surveys have also made some progress in trying to systematically capture important aspects of the institutional and organizational environment that may be important determinants of performance. However, this is very much an open research agenda, and we are still far from a consensus about what features of the facility and its environment matter for performance, and about how we should go about measuring these features. Notwithstanding the inherent difficulties involved in researching this area, the widespread and well-documented problems of quality and efficiency in health care provision suggest that the potential payoffs from progress in this area are considerable.

Finally, what has been the impact of health facility surveys? This is very difficult to assess, not least because of the considerable differences in focus and motivation of surveys. Yet, it is probably safe to say that the impact has not been as great as it could have been. In many cases, data have been under-utilized and survey finding have not found their way into the policy process. Although this problem is by no means universal, it raises some important challenges for the future. Ultimately, impact requires a degree of involvement by policymakers in the design and implementation of surveys, as well as in the dissemination and use of survey findings. In part, this depends on the survey approach being attuned to the needs and requirements of a particular context. However, survey findings generally gain force if they can be generalized and compared with findings from other surveys. An important challenge is therefore to improve consistency across surveys - both over time and across countries. This would require agreement about some core methodological issues, possibly in the form of some core facility modules that will permit international analyses of health system performance.

29
Appeendix A Review of HeaRalth IFacfity Sirveyp This appendix provides an overview of health facility surveys. It offers brief descriptions of the respective surveys. This review does not purport to be comprehensive, but seeks to cover important and accessible facility survey initiatives in the health sector. Where appropriate, references to background reading and research output based on survey data are provided.

# _JD)emographic and Health Surveys (DHS and DH$_

The Demographic and Health Surveys (DHS) have been an important source of individual and household level health data since 1984. The design of the DHS drew on the experiences of the World Fertility Surveys<sup>63</sup> (WFS) and the Contraceptive Prevalence Surveys (CPS), but included an expanded set of indicators in the areas of population, health, and nutrition.64 Similarly to the some of the World Fertility Surveys, many of the DHSs included tools to collect community level data. These Service Availability Modules (SAM) were not facility surveys per se, but rather collected information from "community informants". They were aimed at collecting "objective" information on the facilities and services available to women in the community, focusing, in particular, on family planning services.

More recently, detailed facility _surveys-Service Provision Assessments (SPAs)-_ have been collected within the scope of _DHS+_ activities. The objective of the SPA is to provide information about the characteristics of health services including their quality, infrastructure, utilization, and availability. The surveys can be national or subnational and can be implemented as a stand-alone assessment of services or can be linked to household survey data. Also, the SPAs are not necessarily limited to government facilities, but may be used to compare services of various providers, including private for-profit, NGO, and community health workers. Examples of SPAs include Guatemala (1997), Kenya (1999), Bangladesh (1999), and Mexico (2000).

Exaumples of SPAs and IFindings: lKemys and Tanzaniia The 1999 Kenya SPA (KSPA), focused on the availability and quality of services in the areas of family planning, sexually transmitted infections, or maternal and child health. The data were collected through community interviews, visits to health facilities, and observations of consultations. The facility data complements household data from a DHS implemented in 1998. The survey found that although contraceptive services, drugs and materials are widely available, compliance with infection control procedures, including sterilization of equipment and sterilization of needles was often poor. Also, consultation observations revealed that the diagnostic process was often unsatisfactory, clients were frequently provided with incomplete

> 63 The WFSs were a collection of intemationally comparable surveys of human fertility conducted in 41 developing countries in the late seventies and early eighties. The project was conducted by the Intemational Statistical Institute (ISI), with fundmng from USAID and UJNFPA.

> 64 In 1997 DHS changed name to _DHS+_ to reflect the integration of DHS activities under the MEASURE programme. Under this mandate, _DHS+_ is charged with collecting and analyzing demographic and health data for regional and national family planning and health programs.

30
or insufficient information to chose between interventions, and to ensure compliance with treatment regimes. Other findings include poor preparedness to deal with STI sufferers due to drug stock-outs; lack of equipment to deal with obstetric complications; and poor prescription practices (Macro International 2000).

The Tanzania Reproductive and Child Health Facility Survey was carried out with assistance from MEASURE DHS+. The objective was to provide information on the availability and use of selected reproductive and child health services in Tanzania. The survey included government, NGO and for-profit providers, and covered hospitals, health centers, and dispensaries. Questionnaires were administered to collect information on the community, facility, facility inventory, service provider (staff), pharmacy inventory, and district health management team. The sample was designed to capture the "market of facility services". This was done by sampling all facilities within a concentric ring of the enumeration area, rather than the nearest facility within 30 km of the EA (as in previous surveys) (National Bureau of Statistics (Tanzania) and MEASURE Evaluation 2000).

## _Research and Publications_

- Beegle, K. 1995. "The Quality and Availability of Family Planning Services and Contraceptive Use in Tanzania." The Living Standards Measurement Study Working Paper No. 114 Washington: World Bank.

- Feyisetan, B. J. and M. Ainsworth. 1996. "Contraceptive Use and the Quality, Price, and Availability of Family Planning in Nigeria." _World Bank Economic Review,_ 10: 1, pp. 159-87.

- Mroz, T. A., K. A. Bollen, I. S. Speizer, and D. J. Mancini. 1999. "Quality, accessibility, and contraceptive use in rural Tanzania." _Demography,_ 36:1, pp. 23-40.

- Steele, F., S. L. Curtis, and M. Choe. 1999. "The impact of family planning service provision on contraceptive-use dynamics in Morocco." _Stud Fam Plann,_ 30:1, pp. 28-42.

# _The Living Standards Measurement Study (LSMS)_

The Living Standards Measurement Study (LSMS) was established by the World Bank in 1980 to explore ways of improving the type and quality of household data collected by government statistical offices in developing countries. LSMS surveys are multitopic surveys, designed to permit four types of analysis: (i) simple descriptive statistics on living standards; (ii) monitoring of poverty and living standards over time; (iii) describing the incidence and coverage of government programs; and (iv) measuring the impact of policies and programs on household behavior and welfare (Grosh and others 2000). The first surveys were implemented in Cote d'Ivoire and Peru. Other early surveys followed a similar format, although considerable variation has been introduced over time.

The household questionnaire forms the heart of LSMS survey. It typically includes a health module that provides information on (i) health related behavior; (ii) the utilization of health services; (ii) health expenditures; (iv) insurance status; (v) access to health services. The level of detail of the health section has however varied across surveys. Complementary data are typically collected through community and price questionnaires. In fact, over half of the LSMS surveys conducted before 1997 included community and price questionnaires.

31
Community questionnaires are administered separately to community "informants", and collect information on infrastructure, employment opportunities, availability of credit, and public services, including schools and health facilities.<sup>65</sup> In some LSMS surveys, detailed service provider (health facility or school) data have been collected. In the case of the health sector, facility surveys were implemented in C6te d'Ivoire (1987), Jamaica (1989), and Viet Nam (1998).66 The facility surveys have been included to provide complementary data primarily on prices of health care and medicines, and health care quality. Health facilities have rarely been the object of analysis in research based on LSMS surveys. Quality data may include both structure (staffing, equipment, drugs, etc.) and process (diagnosis, treatment, attentiveness and staff attitude) dimensions of health care quality. The facility surveys have in some cases included private health care providers (Jamaica).

# Experiences with LSMS Health IFacuty Surveys

_Jamaica._ A survey of health care facilities was carried out in September 1990. The data were meant to complement the expanded health module of the 1989 LSMS survey. All public health facilities and a sample of private providers were surveyed. Data collection was based on four separate health services questionnaires: public primary, private primary, public secondary/tertiary and private secondary/tertiary. At primary level, questions related to: catchment area, facility characteristics, patient services, immunization offered, personnel, beds, transportation, drug supply and equipment, family planning services, and maternal health services.<sup>6 7</sup> There were slight differences in the questionnaires administered to public and private facilities.<sup>68</sup> The survey instruments for secondary/tertiary level included more detailed questions on facility characteristics, personnel, and equipment. In the household survey, detailed data on illness episodes and care seeking patterns, including the name of facilities visited, were collected (The World Bank 2001).

_Cote d'Ivoire._ The survey instruments for C6te d'Ivoire preceded the Jamaica survey, and are much more limited. The facility questionnaire includes many of the same sections as the Jamaica survey, but each section is less comprehensive. In addition, no specific data on family planning and maternity care services were collected, and no attempt was made to capture process dimensions of health care quality.

> **65** In the area of health, surveys have covered question conceming the type of health care providers that are available in the community, costs of treatments and medicines, travel times to the respective providers, means and costs of transport, and public health services and programme, including immunization programmes and information campaigns. In some countries, community level data have been sufficiently detailed to permit analysis of the relationship between health care infrastructure and health seeking behaviour and outcomes (e.g., LSMS surveys in Ghana (1987/88, 1991/92, 1998/99) Guyana (1992/93), Pakistan (1991), and Tanzania (1991, 1993)).

> **66** School surveys were included in Jamaica (1990) and Viet Nam (1997/98).

> **67** In respect of maternity care, the questionnaire sought to assess the process of care. This was done by reviewing a range of services/activities with the appropriate staff member, and asking him/her whether the respective service was included m a standard pre-natal visit and for what group of women.

> 6 For private facilities, questions conceming laboratory services were more detailed. Also, questions concerning staff wages, patient flows, revenues, and expenditures were included.

32
_Viet Nam._ The 1998 Viet Nam LSMS survey included school and health facility surveys. The health facility survey was limited in scope and detail relative to both the Jamaica and Cote d'Ivoire surveys. It collected information on distance to different enumeration areas, staffing and areas of operation, number of beds, service range, equipment and drug availability, and cost of services and drugs.

## **_Research and Publications_**

- Lavy, V., J. Strauss, D. Thomas, and P. d. Vreyer. 1996. "Quality of Health Care, Survival and Health Outcomes in Ghana." _Journal of Health Economics,_ 15:3, pp. 333-57.

- Peabody, J., P. Gertler, and A. Liebowitz. 1998. "The policy implications of better structure and process on birth outcomes." _Health Policy,_ 43, pp. 1-13.

- Thomas, D., V. Lavy, and J. Strauss. 1996. "Public policy and anthropometric outcomes in C6te d'Ivoire." _Journal of Public Economics,_ 61, pp. 139-54.

- Gertler, P. and J. Litvack. Forthcoming. "Healthcare During Transition," in _Household Welfare and Viet Nam 's Transition to a Market Economy._ Dollar, Glewwe and Litvack eds. Washington, D. C.: The World Bank.

# _Situation Analysis_

Situation analysis (SA) was introduced as a tool for program evaluation by the Population Council in 1989. In general, the SA can be described as "a comprehensive and standardized approach for systematically assessing both the readiness of family planning/reproductive health programs to delivery services and the quality of care received by clients." (Miller and others 1998). The development of the tool was stimulated by indications from DHS surveys that service delivery weaknesses were important in explaining low contraceptive prevalence rates in many countries.

The first SA was carried out in Kenya. Subsequently, SAs have been carried out extensively, including in 11 African countries. The SAs are based on a representative sample of service delivery units within a geographic area of interest. The approach includes structured interviews with managers and facility staff; inventory review and direct observation of clinic facilities and the availability of equipment and consumables; review of service statistics for 12 months; nonparticipant direct observation of family planning client-provider interaction; and interviews with clients of both family planning and MCH services. The general approach has been modified in some cases to address a broader set of concerns.

Situation analyses have provided clear evidence of the poor state of service delivery in many countries. Documented problems include poor contraceptive stocks, lack of basic . infrastructure and equipment, poor adherence to diagnostic and treatment protocols. Through the implementation of follow-up studies, it has been possible to measure changes over time, and to assess the impact of policies aimed at improving service delivery. Data have been used, _inter alia,_ in the design of family planning programs, training initiatives, and in the formulation of sector strategies.

33
## _Background publications_

- Miller, K., R. Miller, A. Askew, M. C. Horn, and L. Ndhlovu. 1998. "Clinic-based family planning and reproductive health services in Africa: findings from situation analysis studies." Population Council.

- Miller, R., K. Fisher, K. Miller, L. Ndhlovu, I. Maggwa, A. Askew, D. Sanogo, and P. Tapsoba. 1997. The Situation Analysis Approach to Assessing Family Planning and Reproductive Health Services: A Handbook. New York: Population Council.

# _RAND_ **_Surveys_**

RAND has supported the design and implementation of Family Life Surveys (FLS) in developing countries since the 1970s. Currently available country surveys include Malaysia (1976-77, 1988-89), Indonesia (1993, 1997, 1998, 2000), Guatemala (1995), and Bangladesh (1996). Here, the Indonesia and Bangladesh surveys are discussed in further detail.

## **The Indonesia Family Life** Survey

The Indonesia Family Life Survey (IFLS) is an ongoing, multi-topic longitudinal survey. It aims to provide data for the measurement and analysis of a range of individual- and household-level behaviors and outcomes. The survey has collected data at individual and household level, including indicators of economic well-being, education, migration, labor market outcomes, fertility and contraceptive use, health status, use of health care and health insurance, intra-household relationships, and participation in community activities. In addition, community level data are collected. These include detailed surveys of service providers (schools and health care providers) in the selected communities. The first wave of the survey (IFSL1) was conducted in 1993/94, covering approximately 7000 households. The IFLS2 and IFLS2+ were conducted in 1997 and 1998, and a further wave (IFLS3) was planned for 2000.69 Re-interview rates of over 90 percent were achieved in the second and later waves of the survey.

In respect of the health facility survey, visits to local health care providers were carried out, and staff representatives were interviewed about the staffing, operation, and usage of their facilities. The surveys covered (i) government health centers and sub-centers; (ii) private clinics and doctors, midwives, nurses, and paramedics; and, (iv) community health posts. For each community, up to 3 government health centers, 6 private clinics, doctors, etc., and 2 community health posts were surveyed. Health care providers were selected on the basis of information provided by household respondents about where they normally seek care.

Different questionnaires were used for the respective providers to reflect differences in organization and scope of services. In general, the questionnaires collected data on availability and prices of services, lab tests, and drugs; availability of equipment and supplies; direct observations about the facility's cleanliness and other features that might influence its attractiveness to patients. In addition, five hypothetical patient scenarios or vignettes were

> 69 The IFLS2+ covered a sub-sample (25 percent) of households and was designed to assess the effects of Indonesia's economic crisis.

34
presented to the relevant health worker to assess the respondents' knowledge of process in patient care.<sup>70</sup>

# **Matlab Health and Socio-Economic Survey (MHSS)**

The MHSS was implemented in 1996, in Matlab, a rural region in Bangladesh. The general focus of the survey was on issues relating to health and well-being for rural adults and the elderly, including the effects on health status and health care utilization of socio-economic characteristics; health status, social and kin network characteristics and resource flows; community services and infrastructure. The study included a survey of individuals and households, a specialized outmigrant survey (sample of individuals who had left the households of the primary sample since 1982), and a community provider survey.

The provider survey covered seven types of health care providers, from government health complexes, family welfare centers, traditional practitioners, and community health workers. Separate instruments were used for each type of provider, although there were similarities in content. Clinical vignettes were administered to all providers.

# _Research and Publications_

- Frankenberg, E. 1995. "The effects of access to health care on infant mortality in Indonesia." _Health Transit Rev,_ 5:2, pp. 143-63.

- Frankenberg, E. and D. Thomas. 2001. "Women's Health and Pregnancy Outcomes: Do Services Make a Difference?" _Demography,_ 38:2, pp. 253-65.

- Gertler, P. J. and J. W. Molyneaux. 1994. "How Economic Development and Family Planning Programs Combined to Reduce Indonesian Fertility." _Demography,_ 3 1: 1, pp. 33-63.

- Molyneaux, J. W. and P. J. Gertler. 2000. "The Impact of Targeted Family Planning Programs in Indonesia." _Population and Development Review,_ 26:0, pp. 61-85.

- Frankenberg, E., B. Sikoki, W. Suriastini, and D. Thomas. 2001. "Changes in the Use of Health Services During Indonesia's Economic Crisis." Processed.

# **_MEASURE Evaluation_**

MEASURE Evaluation is a project under the MEASURE Program.71 The main purpose of the project is to develop and apply methods for monitoring and evaluation in the areas of family planning; maternal health; sexually transmitted diseases, especially HIV/AIDS; nutrition; and infectious disease. MEASURE _Evaluation_ works in collaboration with programs in developing countries, USAID and other international donor agencies to (i) improve performance monitoring systems for tracking results; (ii) identify appropriate indicators, test their measurement, and establish needed data systems<sup>72</sup> ; and, (iii) evaluate

> 70 The vignettes covered: provision of IUDs, provision of oral contraceptives, prenatal care, treating a child with vomiting and diarrhea, and treating and adult with a respiratory illness.

> **71** The MEASURE (Monitoring and Evaluation to Assess and Use Results) Program is funded by USAID since 1997. MEASURE mcludes five projects, which offer technical services in data collection, analysis, dissemination, and use Details can be found on http://www.measureprogram.org/.

> **72** This has included the establishment of a Facility Survey Working Group to determine the use and procedures for cntical aspects of data collection, monitoring, evaluation and dissemination.

35
interventions and their cost-effectiveness. As a component of this general work program, MEASURE Evaluation project has provided technical support in the design, implementation and analysis of a range of health facility surveys. Examples include Uganda and Paraguay.

# The Uganda D$S1BI suirvey

The DISH project, funded by USAID, is one of the largest reproductive health programs in Uganda. It operates in 12 of the countries 45 districts, and covers approximately 30 percent of the population. Activities under the DISP project include training, support of community reproductive health workers, capacity building in health management information systems, capacity building in financial management, and IEC activities. As a component of ongoing monitoring and evaluation work, a series of surveys have been implemented to measure changes in reproductive, maternal and child health knowledge and behavior. The DISH surveys comprise interim surveys between the 1995 and 2000 Uganda DHSs. Two survey rounds have been carried out, in 1997 and 1999. Each survey collects both household and facility data. The household component is based on reduced DHS-style questionnaires. The 1999 survey collected information from 1766 women, 1057 men and 478 facilities (including health units, drug stores and pharmacies), and information is reported on IEC, family planning, maternal health, child health and nutrition, STDs and HIV/AIDS, and health facilities (Kantende and others 1999). At facility level, survey instruments included facility interview questionnaire and a facility inventory questionnaire. In addition, separate inventory questionnaires were designed for pharmacies and drug shops. Through these instruments, information is collected on general facility characteristics (including services offered), DISH training, availability of supplies and IEC material.

# _Source_

Kantende, C., R. Bessinger, N. Gupta, R. Knight, and C. Laettenmaier. 1999. "Uganda Delivery of Improved Services for Health (DISH) Evaluation Surveys." MEASURE Evaluation Technical Report Series, No. 6.

# HIeaRlth Care Decentrafization _fi_ lParaguay

> The objectives of the study was to identify and quantify the effect of transferring management control for basic health services provision from central to municipal govermment in Paraguay on costs, efficiency, basic health service quality, and patterns of health service use and equity. The main units of analysis were public health facilities, their clients,

> municipality populations. The study uses pre- and post-decentralization design, with a control group, and focused on family planning, maternal health, and infant and child health. The first round in the survey was carried out in 1998. The survey covered 124 public and 19 private facilities and 1261 exit interviews. In addition, with a view to obtain information on socio-

> economic characteristics, health outcomes, and health care seeking behavior, a general household survey was also implemented. This was carried out as a component of the 1998 maternal and infant health survey.

At facility level, three types of questionnaires were administered: (i) short inventory questionnaire; (ii) interview with facility director or other staff member, and (iii) direct observation of availability of supplies, equipment, infrastructure, etc. In addition, a self36
administered time sheet was completed by staff members. The exit poll collected information on socio-economic characteristics, reason for visit, services received, travel time, payments, perceptions, etc. More detailed data were collected from randomly selected households, covering both socioeconomic and demographic characteristics, health, and health care seeking behavior. A second round of data collection is planned, but has been delayed due to slower than expected implementation of decentralization.

## _Source_

Angeles, G., J. F. Stewart, R. Gaete, D. Mancini, A. Trujillo, and C. I. Fowler. 1999. "Health Care Decentralization in Paraguay: Evaluation of Impact on Cost, Efficiency, Basic Quality, and Equity. Baseline Report." MEASURE Evaluation Technical Report Series No. 4. Carolina Population Center, University of North Carolina at Chapel Hill.

# _WHO Multi-Country Evaluation of IMCI_

There is a considerable history of facility surveys in the WHO. Originally, health facility instruments focused on issues relating to child mortality and morbidity, and survey instruments were often disease specific (i.e., designed to evaluate specific programs or projects), e.g., ARI (acute respiratory infection), diarrhea. The principal interest lay in assessing quality (typically process focused assessment, though also measurement of input/structure - e.g., drug availability, staffing), and effect of quality on household behavior and health outcomes. Hence facility surveys were typically accompanied by exit polls and/or household surveys (coverage surveys).

Recently, integrated surveys have been designed and implemented. An important element of this work is carried out in the context of multi-country evaluations (MCE) of the Integrated Management of Childhood Illnesses (IMCI), and is implemented by different institutions (John Hopkins, CDC, etc.), under coordination by WHO. 73 Integrated instruments for costs and quality have been developed and implemented (or are being implemented) in Bangladesh, Tanzania, and Uganda. The purpose of the MCEs is to (i) document the effects of IMCI interventions on health workers performance, health systems and family behaviors; (ii) determine whether and to what extent, the IMCI strategy as a whole has a measurable impact on health outcomes (reducing under five morbidity and mortality); (iii) describe the cost of IMCI implementation at national, district and health facility levels; (iv) increase the sustainability of IMCI and other child health strategies by providing a basis for the improvement of implementation; and (v) support planning and advocacy for childhood interventions by ministries of health in developing countries and national and international partners in development. Worldwide there are 30 countries at different stages of implementation of IMCI among which Uganda, Peru, Bangladesh and Tanzania will participate in the MCE.

## _Source_

> 73 The Integrated Management of Childhood Illnesses (IMCI) Strategy was developed by WHO and UNICEF to address five leading causes of childhood mortality, namely: malaria, pneumonia, diarrhea, measles and malnutrition. The three main components addressed by the strategy are: improved case management, improved health systems and improved family and comrnunity practices.

37
- WHO. 2001. "The Multi-Country Evaluation of IMCI Effectiveness, Cost and Impact (MCE): Progress Report May 2000-April 2001." Geneva.

_JPub5lic Expenditre 2"rackiirg Smrveys_ In recent years, the public expenditure tracking surveys (PETS) have been implemented in a range of countries in response to perceived public expenditure management and service delivery problems. These surveys have varied considerably both in terms of scope and focus, to the extent that the use of a single term in reference to the variegated survey activities may be misleading. In general, it is possible to distinguish two types of surveys under the umbrella of public expenditure tracking: tracing studies and quantitative service delivery surveys (QSDSs). Tracing studies have been used to identify the location and extent of impediments in financial flows to sub-national levels of administration or service delivery units. Government resources, which are typically earmarked for particular uses in the budget (votes, line items), flow, upon release, within a pre-defined legal, regulatory, and institutional framework, passing through the layers of the government and via banking system down to districts or facilities. Tracing studies track the flow of resources through these institutional strata in order to determine how much of the original resource reaches each new level, and how long the resource takes to get there. In contrast, QSDSs are more akin to facility surveys, although they have often tried to capture different aspects of the relationship between service delivery units and the administrative and logistical system "upstream". The focus on efficiency and quality in service delivery is in recognition of the fact that even if budget outturns reflect allocations, ultimate impact in terms of development objectives also requires that financial resources are efficiently transformed into appropriate public services.

As noted, PETS can in part be seen as a response to the lack of information about the systems through which budget resources are transformed into services. Indeed, the first PETS, implemented in Uganda in 1996, asked why it was that service delivery outputs in the health and education sectors had failed to respond to considerable increases in public spending. The survey was carried out in 19 of Uganda's 39 districts. It focused on a sample of 250 government schools and 100 public health clinics, and spending data for the period 1991-95 were collected (Ablo and Reinikka 1998; Reinikka 1999). For the education sector, the survey compared central government outlays for non-wage spending with the corresponding school income. Specifically, it focused on the capitation grant that is paid out per student enrolled as a matching government contribution against the mandated tuition fees paid by parents. The results demonstrate how weak governance and lack of accountability can hamper the process of budget implementation, and that _defacto_ public spending may be determined by the bargaining power of individual facilities (Reinikka and Svensson 2001). The survey also demonstrated the inaccuracy of administrative data on school outputs. In the health sector records were poor or non-existing, and little quantitative data were collected. On the basis of interviews, it was however estimated that on average, almost 70 percent of medical supplies and drugs were expropriated by staff.

A follow up study was carried out in Uganda in 1999 (Ministry of Education and Sports (Republic of Uganda) 2000). This was a tracing study in the education sector, where, for a sample of 427 schools and 11 District Headquarters, four monthly releases in 1998 and 1999 were traced through the payment system. The objective of the survey was to track flow 38
of funds, verify compliance regulations, and assess the scope and sources of delays in budget execution. The survey found considerable improvements in the amount of funding that schools received compared to the 1996 study. In part, these improvements can be attributed to measures to increase transparency and accountability - e.g., through publication of school and health facility in local papers and on local notice boards (Reinikka 1999). However, the study also documents considerable delays in the transfer of funds. The study found that these delays do not occur uniformly throughout the system. Central ministries, the Central Bank, and the head quarters of the government owned Uganda Commercial Bank (UCB) all transferred resources in full and without delays. At lower levels, however, transfers were held up by local branches of UCB and by districts administrators.

The problems highlighted in the Uganda studies are far from new. However, these studies provided important quantitative evidence on the scope and seriousness of problems in budget execution, and demonstrated how a new tool can be applied to analyze these issues. Other surveys have been carried out in Ghana, Honduras, Tanzania, and Uganda. These surveys have focused on leakage, "job capture" and other means of extracting private benefits from public positions may be systematically, the implicit allocation of resources at different levels of administration, and the costs and efficiency of health care delivery. For example, the study of public employment in the Honduran health sector showed that the best qualified health sector staff engaged in more shirking, 'ghost working', and job capture than lowerranked employees (World Bank 200 lb). In addition, a number of surveys are currently being planned or implemented. Many of these surveys are variants of school of facility surveys, and in some cases efforts are made to link the surveys with household data collection exercises.

## _Source_

- Dehn, J., R. Reinikka, and J. Svensson. 2001. "Basic Service Delivery: A Quantitative Survey Approach.". Development Research Group, The World Bank.: Washington, D.C. Processed.

# **References**

The word "processed" describes informally reproduced works that may not be commonly available through library systems.

- Ablo, Emmanuel, and Ritva Reinikka. 1998. "Do budgets really matter? Evidence from public spending on education and health in Uganda." Policy Research Working Paper 1926. Development Research Group, World Bank, Washington, D.C.

- Ackerberg, D. A., M. P. Machado, and M. H. Riordan. 2001. "Measuring the Relative Performance of Providers of Health Service." NBER Working Paper No. W8385. Cambridge, Mass.

- Akin, J., C. Griffin, D. K. Guilkey, and B. M. Popkin. 1986. "The Demand for Primary Health Care Services in the Bicol Region of the Philippines." _Economic Development and Cultural Change_ 34(4): 755-82.

39
- Akin, J., D. Guilkey, and H. Denton. 1995. "Quality of Services and Demand for Health Care in Nigeria: A Multinomial Probit Estimation." _Social Sciences and Medicine_ 40(11): 1527-37.

- Akin, J. and P. Hutchinson. 1999. "Health-Care Facility Choice and the Phenomenon of Bypassing." _Health Policy and Planning_ 14(2): 135-51.

- Akin, J., S, C. Griffin, D. K. Guilkey, and B. M. Popkin. 1984. _The Demandfor Primary Health Care in Developing Countries._ Totowa, N.J.: Littlefield, Adams.

- Alderman, H., and V. Lavy. 1996. "Household Responses to Public Health Services: Cost and Quality Tradeoffs." _World Bank Research Observer_ 11(1): 3-22.

- Anderson, D. L. 1980. "A Statistical Cost Function Study of Public General Hospitals in Kenya." _Journal of Developing Areas_ 14(2): 223-35.

- Angeles, G., J. F. Stewart, R. Gaete, D. Mancini, A. Trujillo, and C. I. Fowler. 1999. "Health Care Decentralization in Paraguay: Evaluation of Impact on Cost, Efficiency, Basic Quality, and Equity. Baseline Report." MEASURE Evaluation Technical Report Series No. 4. Carolina Population Center, University of North Carolina at Chapel Hill.

- Arrow, K. J. 1963. "Uncertainty and the Welfare Economics of Medical Care." _American Economic Review._

- Atkinson, S. J. 1993. "Anthropology in Research on the Quality of Health Services." _Caderno de Sauide Publica, Rio de Janeiro_ 9(3): 283-99.

- Barnum, H. N. and J. Kutzin. 1993. _Public Hospitals in Developing Countries._ Baltimore: Johns Hopkins University Press.

- Barnum, H. N., J. Kutzin, and H. Saxenian. 1995. "Incentives and Provider Payment _Methods." International Journal of Health Planning and Management 10:_ 23-45.

- Baumol, W. J., J. C. Panzar, and R. D. Willig. 1982. _Contestable markets and the theory of industry structure._ New York: Harcourt Brace

- Beegle, K. 1995. "The Quality and Availability of Family Planning Services and Contraceptive Use in Tanzania." World Bank, Washington, D.C.

- Benefo, K. and P. T. Schulz. 1994. "Determinants of Fertility and Child Mortality in Cote d'Ivoire and Ghana." Living Standards Measurement Study Working Paper 103. World Bank.

- Bennet, S. and L. Franco. 1999. "Public Sector Health Worker Motivation and Health Sector Reform: A Conceptual Framework." Major Applier Research 5, Technical Paper 1. Partnerships for Health Reform.

- Bennet, S., D. Gzirishvili, and R. Kanfer. 2000. "An In-depth Analysis of the Determinants and Consequences of Worker Motivation in Two Hospitals in Tbilisi, Georgia." Major Applier Research 5, Working Paper 9. Partnerships for Health Reform.

- Bertrand, J. T., T. Sullivan, and J. Rice. 2001. "Quick investigation of quality: measuring quality of care in family planning." Caroline Population Centre, Chapel Hill, NC.

- Bessinger, R. and Bertrand. 2001. "Monitoring Quality of Care in Family Planning Programs: A Comparison of Observations and Client Exit Interviews." _International Family Planning Perspectives_ 27(2): 63-70.

40
- Bitran, R. 1995. "Efficiency and Quality in the Public and Private Sectors in Senegal." _Health Policy and Planning_ (Health Policy Plan.) 10(3): 271-83.

- Bitran-Dicowsky, R. and D. W. Dunlop. 1993. "The Determinants of Hospital Costs: An Analysis of Ethiopia." In Mills and Lee, eds., _Health Economics Research in Developing Countries._ Oxford: Oxford University Press.

- Brown, L. D., L. M. Franco, N. Rafeh, and T. Hatzell. 1992. "Quality Assurance of Health Care In Developing Countries." Quality Assurance Project. Bethesda, Md.

- Chaix-Couturier, C., I. Durand-Zaleski, D. Jolly, and P. Durieux. 2000. "Effects of financial incentives on medical practice: results from a systematic review of the literature and methodological issues." _Int J Qual Health Care_ 12: 13 3-42.

- CIET international. 1996. "Uganda Baseline Service Delivery Survey: Final Report." CIET.

- Cockcroft, A., L. Monasta, J. Onishi, and E. Karim. 1999. "Bangladesh Baseline Service Delivery Survey: Final Report." CIET International.

- Cowing, T. G. and A. G. Holtmann. 1983. "Multiproduct short-run hospital cost functions: Empirical evidence and policy implications from cross-section data." _Southern Economic Journal_ (Southern Economic Journal ) 49: 637-5 3.

- De Geyndt, W. 1995. "Managing the quality of health care in developing countries." Technical Paper No. 258. World Bank, Washington, D.C.

- de Silva, A. 1999. "A Framework for Measuring Responsiveness." GPE Discussion Paper Series: No. 32. WHO.

- Deci, E. L. 1975. _Intrinsic Motivation._ New York: Plenum Press.

- Dehn, J., R. Reinikka, and J. Svensson. 2000. "The Supply Side of Service Delivery: A Quantitative Survey Approach." Development Research Group. World Bank, Washington, D.C. Processed.

- Dixit, A. 1997. "Power of incentives in private vs public organisations." _American economic Review Papers and Proceedings_ 87(2).

- Dixit, A. 2001. "Incentives and Organizations in the Public Sector: An Interpretative Review." Processed.

- Donabedian, A. 1980. _Explorations in Quality Assessment and Monitoring, Volume 1: The Definition of Quality and Approaches to its Assessment._ Washington D.C.: Health Administration Press.

- Dor, A. 1987. "Estimates of cost functions for the Peruvian hospital sector: Some policy implications." Unpublished report, The Urban Institute.

- Dor, A. 1994. "Non-minimum Cost Functions and the Stochastic Frontier: On Applications to IHealth Care Providers." _Journal of Health Economics_ 13(3): 329-34.

- Drummond, M. F., B. O'Brien, G. L. Stoddart, and G. W. Torrance. 1997. _Methods for the economic evaluation of health care programmes._ Oxford: Oxford University Press.

- Ensor, T. and S. Witter. 2001. "Health economics in low income countries: adapting to the reality of the unofficial economy." _Health Policy_ 57(1): 1-13.

- Farrel, M. J. 1957. "The Measurement of Productive Efficiency." _Journal of the Royal Statistical Society,_ A CXX(3): 253-90.

41
- Feldstein, M. S. 1967. _Economic analysisfor health service efficiency : Econometric studies of the British National Health Service._ Amsterdam: North-Holland.

- Ferrinho, P., W. Van Lerberghe, M. R. Julien, E. Fresta, A. Gomes, F. Dias, A. Goncalves, and B. Backstrom. 1998. "How and why public sector doctors engage in private practice in Portuguese-speaking African countries." _Health Policy Plan_ 13(3): 332-28.

- Feyisetan, B. J. and M. Ainsworth. 1996. "Contraceptive Use and the Quality, Price, and Availability of Family Planning in Nigeria." _World Bank Economic Review_ 10(1): 159-87.

- Filmer, D., J. Hammer, and L. Pritchett. 2000. "Weak Links in the Chain: A Diagnosis of Health Policy in Poor Countries." _The World Bank Research Observer 15_ (2, August): 199-224.

- F0rsund, F. R., C. A. K. Lovell, and P. Schmidt. 1980. "A survey of frontier production functions and their relationship to efficiency measurement." _Journal of Econometrics_ (Journal of Econometrics ) 13: 5-25.

- Franco, L. M., S. Bennett, and R. Kanfer. 2002. "Health sector reform and public sector health worker motivation: a conceptual framework." _Social Science and Medicine 54:_ 1255-66.

- Frankenberg, E. 1995. "The Effects of Access to Health Care on Infant Mortality in Indonesia." _Health Transitions Review 5(2)._

- Frankenberg, E. and D. Thomas. 2001. "Women's Health and Pregnancy Outcomes: Do Services Make a Difference?" _Demography_ 38(2): 253-65.

- Fried, H. O., C. A. K. Lovell, and S. S. Schmidt. 1993. _The measurement ofproductive efficiency._ New York, Oxford: Oxford University Press.

- Garner, P., J. Thompson, and D. Donaldson. 1990. "Quality Assessment of Health Facilities in Rural Papua New Guinea." _Health Policy and Planning 5(1):_ 49-59.

- Gertler, P., L. Locay, and W. Sanderson. 1987. "Are user fees regressive? The welfare implications of health care financing proposals in Peru." _Journal of Econometrics_ 36(suppl.): 67-88.

- Gertler, P., and J. van der Gaag. 1990. _The willingness to payfor medical care: Evidencefrom two developing countries._ Baltimore and London: Johns Hopkins University Press for the World Bank.

- Gertler, P. J., and J. Molyneaux. 1997. "Experimental Evidence on the Effect of Raising User Fees for Publicly Delivered Health Care Services: Utilization, Health Outcomes, and Private Provider Response." Processed.

- Gilson, L., H. Kitange, and T. Teuscher. 1993. "Assessment of process quality in Tanzanian primary care." _Health Policy_ 26: 119-39.

- Gilson, L., M. Magomi, and E. Mkangaa. 1995. "The structural quality of Tanzanian primary health facilities." _Bulletin of the World Health Organization_ 73(1): 105-14.

- Goddard, M., R. Mannion, and P. Smith. 2000. "Enhancing Performance in Health Care: Theoretical Perspectives on Agency and the Role of Information." _Health Economics_ 9: 95-107.

42
- Goldman, F., and M. Grossman. 1983. "The production and cost of ambulatory care in community health centers." In Scheffler and Rossiter, eds., _Advances in health economics and health services research Vol. 4._ Greenwich CT: JAI Press.

- Gosden, T., L. Pederson, and D. Torgerson. 1999. "How Should We Pay Doctors? A Systematic Review of Salary Payments and their Effect on Doctor Behaviour." _Quarterly Journal of Medicine_ 92: 47-55.

- Grosh, M. E., P. Glewwe, and World Bank. 2000. _Designing household survey questionnaires for developing countries: lessons from 15 years of the living standards measurement study._ Washington, D.C.: World Bank.

- Hanson, K. and L. Gilson. 1996. "Cost, Resource Use and Financing Methodology for District Health Services: A Practical Manual." Bamako Initiative Technical Report Series Number 34. UNICEF: New York.

- Hossain, S. 1989. "Effects of public programs on family size, child education and health." _Journal of Development Economics_ 30(1): 145-58.

- Kanfer, R. 1999. "Measuring Health Worker Motivation in Developing Countries." Major Applied Research 5, Working Paper 1. Partnerships for Health Reform.

- Kantende, C., R. Bessinger, N. Gupta, R. Knight, and C. Laettenmaier. 1999. "Uganda Delivery of Improved Services for Health (DISH) Evaluation Surveys." MEASURE Evaluation Technical Report Seris, No. 6.

- Killingsworth, J. R., N. Hossain, Y. Hedrick-Wong, S. D. Thomas, A. Rahman, and T. Begum. 1999. "Unofficial fees in Bangladesh: price, equity and institutional issues." _Health Policy Plan_ 14(2): pp. 152-63.

- Kreps, D. M. 1997. "Intrinsic Motivation and Extrinsic Incentives." _American Economic Review_ 87(2): 359-64.

- Lavy, V. and J.-M. Germain. 1994. "Quality and Cost in Health Care Choice in Developing Countries." Living Standards Measurement Study Working Paper 105. Development Research Group, World Bank, Washington, D.C.

- Lavy, V., J. Strauss, D. Thomas, and P. d. Vreyer. 1996. "Quality of Health Care, Survival and Health Outcomes in Ghana." _Journal of Health Economics_ 15(3): 333-57.

- Leonard, K. L., G. Mliga, and D. H. Mariam. 2000. "Bypassing Health Centers in Tanzania: Revealed Preference for Quality." Processed.

- Lewis, M. A., G. M. La Forgia, and M. B. Sulvetta. 1996. "Measuring public hospital costs: empirical evidence from the Dominican Republic." _Soc Sci Med_ 43(2): 221-34.

- Li, T. and R. Rosenman. 2001. "Cost inefficiency in Washington hospitals: a stochastic frontier approach using panel data." _Health Care Management Science_ 4(2): 73-81.

- Lindelow, M., R. Reinikka, and J. Svensson. 2001. "Health Care On The Frontline: Survey Evidence From Uganda." Processed.

- Litvack, J. and C. Bodart. 1993. "User fees plus quality equals improved access to health care: results of A field experiment in Cameroon." _Social Science and Medicine_ 37(3): 36983.

- L6pez Casanovas, G. and A. Wagstaff. 1996. "Hospital costs in Catalonia: A stochastic frontier analysis." _Applied Economics Letters_ (Applied Economics Letters) 3: 471-74.

43
Macro International. 2000. _DHS+ Dimensions:_ Vol. 2 No. 2.

- Manning, N., R. Mukherjee, and 0. Gokcekus. 2002. "Public officials and their institutional environment: An analytical model for assessing the impact of institutional change on public sector performance." Policy Research Working Paper 2427. World Bank, Washington, D.C.

- Martinez, J. and T. Martineau. 1998. "Rethinking human resources: an agenda for the millennium." _Health Policy Plan_ 13(4): 345-58.

- McPake, B., A. Asiimwe, F. Mwesigye, M. Ofumbi, L. Orthenblad, P. Streefland, and A. Turinde. 1999. "Informal economic activities of public health workers in Uganda: implications for quality and accessibility of care." _Social Science and Medicine_ 49(4): 849-65.

- Miller, K., R. Miller, A. Askew, M. C. Horn, and L. Ndhlovu. 1998. "Clinic-based family planning and reproductive health services in Africa: findings from situation analysis studies." Population Council.

- Miller, R., K. Fisher, K. Miller, L. Ndhlovu, I. Maggwa, A. Askew, D. Sanogo, and P. Tapsoba. 1997. _The Situation Analysis Approach to Assessing Family Planning and Reproductive Health Services: A Handbook._ New York: Population Council.

- Mills, A. 1997. "Improving the Efficiency of Public Sector Health Services in Developing Countries: Bureaucratic versus Market Approaches." In Colclough, ed., _Marketizing education and health in developing countries: Miracle or mirage?_ Oxford and New York: Oxford University Press.

- Mills, A. J., J. Kapalamula, and S. Chisimbi. 1993. "The cost of the district hospital: a case study in Malawi." _Bulletin of the World Health Organization_ 71(3-4): 329-39.

- Ministry of Education and Sports (Republic of Uganda). 2000. "Tracking the Flow of and Accountability for UPE Funds." International Development Consultants Ltd: Kampala, Uganda.

- Mowday, R. and R. M. Steers. 1979. "The measurement of organisational commitment." _Journal of Vocational Behaviour_ 14: 224-47.

- Mroz, T. A., K. A. Bollen, I. S. Speizer, and D. J. Mancini. 1999. "Quality, accessibility, and contraceptive use in rural Tanzania." _Demography_ 36(1): 23-40.

- Mwabu, G., M. Ainsworth, and A. Nyamete. 1993. "Quality of Medical Care and Choice of Medical Treatment in Kenya: An Empirical Analysis." _Journal of Human Resources_ 28(4): 838-62.

- National Bureau of Statistics (Tanzania) and MEASURE Evaluation. 2000. "Tanzania Reproductive and Child Health Facility Survey." MEASURE Evaluation Technical Report Series, No. 7.

- Newman, J., M. Pradhan, L. Rawlings, G. Ridder, R. Coa, and J. L. Evia. 2000. "An Impact Evaluation of Education, Health and Water Supply Investments of the Bolivian Social Investment Fund." World Bank. Processed.

- Over, M. and N. Watanabe. 2000. "Evaluating the Impact of Organizational Reforms in Hospitals." In Preker and Harding, eds., _Hospital Organizational Reform in Developing Countries._

44
- Paredes, P., M. Pena, E. Flores-Guerra, J. Diaz, and J. Trostle. 1996. "Factors influencing physician's prescribing behaviour in the treatment of childhood diarrhoea: knowledge may not be the clue." _Social Science and Medicine_ 42: 1141-53.

- Paul, S. 1992. "Accountability in Public Services: Exit, Voice and Control." _World Development_ (July).

- Paul, S. 2000. "Making Voice Work: The Report Card on Bangalore's Public Services." Public Affairs Centre. Processed.

- Peabody, J., P. Gertler, and A. Liebowitz. 1998. "The policy implications of better structure and process on birth outcomes." _Health Policy_ 43: 1-13.

- Peabody, J., J. Luck, P. Glassman, T. R. Dresselhaus, and M. Lee. 2000. "Comparisons of Vignettes, Standardized Patients, and Chart Abstractions." _Journal ofAmerican Medical Association_ 283(13): 1715-22.

- Pestieau, P. and H. Tulkens. 1993. "Assessing and explaining the performance of public sector activities." _Finanz Archiv_ 50: 293-323.

- Prennushi, G., G. Rubio, and K. Subbarao. 2001. "Monitoring and Evaluation." In _PRSP Sourcebook._ Bank ed.

- Price, J. L. and C. W. Mueller. 1986. _Handbook of Organizational Measurement._ Marshfield: Pitman Publishing Inc.

- Price Waterhouse Coopers. 1999. "Tanzania Public Expenditure Review: Health and education financial tracking study. Final report, Vol. I-IH." Dar es Salaam. Processed.

- Pritchett, L. 1996. "Mind your P's and Q's: The cost of public investment is not the value of public capital." Policy Research Working Paper 1660. World Bank, Washington, D.C.

- Propper, C. 1996. "Market Structure and Prices: The Responses of Hospitals in the UK National Health Service to Competition." _Journal of Public Economics_ 61: 307-35.

- Propper, C., D. Wilson, and N. Soderlund. 1998. "The effects of regulation and competition in the NHS internal market: The case of general practice fundholder prices." _Journal of Health Economics_ 17(6): 645-73.

- Reinikka, R. 1999. "Using Surveys for Public Sector Reform." _PREMNotes._ World Bank: Washington, DC.

- Reinikka, R. 2001. "Recovery in service delivery: Evidence from schools and health centers." In Reinikka and Colliers, eds., _Uganda 's Recovery: The Role of Farms, Firms, and Government._ Washington, D.C.: The World Bank, Regional and Sectoral Studies.

- Reinikka, R. and J. Svensson. 2000a. "Cost Efficiency and Incentives in Health Care: A Study Proposal." Development Research Group, World Bank, Washington, D.C. Processed.

- Reinikka, R. and J. Svensson. 2001. "Explaining Leakage of Public Funds." Policy Research Working Paper 2709. World Bank, Development Research Group, Washington, D.C.

- Research on Poverty Alleviation (REPOA) and Economic and Social Research Foundation (ESRF). 2001. "Pro-poor expenditure tracking." Dar es Salaam. Processed.

- Rosenzweig, M. R. and T. P. Schultz. 1982. "Child mortality and fertility in Colombia: individual and community effects." _Health Policy and Education_ 2: 305-48.

45
- Rosenzweig, M. R. and K. I. Wolpin. 1982. "Government interventions and household behaviour in A developing country: Anticipating the unanticipated consequences of social programs." _Journal of Development Economics_ 10: 209-25.

- Rosko, M. D. 1999. "Impact of internal and external environmental pressures on hospital inefficiency." _Health Care Management Science_ 2(2): 63-74.

- Rosko, M. D. 2001. "Cost efficiency of US hospitals: a stochastic frontier approach." _Health Economics_ 10(6): 539-5 1.

- Rowe, A. K., F. Onikpo, M. Lama, F. Cokou, and M. S. Deming. 2001. "Management of childhood illness at health facilities in Benin: problems and their causes." _American Journal of Public Health_ 91(10): 1625-35.

- Segall, M. 2000. "Human Development Challenges for Health Care Reform." In Ferrinho and Lerberghe, eds., _Providing Health Care Under Adverse Conditions: Health Personnel Performance and Individual Coping Strategies._

- Soderlund, N., I. Csaba, A. Gray, R. Milne, and J. Raferty. 1997. "Impact of the NGS reforms on English Hospital Productivity: an analysis of the first three years." _British Medical Journal_ l(November): 1126-29.

- Somanathan, A., K. Hanson, T. Dorabawila, and B. Perera. 2000. "Operating Efficiency in Public Sector Health Facilities in Sri Lanka: Measurement and Institutional Determinants of Performance." PHR and Abt Assoc.

- Steele, F., S. L. Curtis, and M. Choe. 1999. "The impact of family planning service provision on contraceptive-use dynamics in Morocco." _Studies in Family Planning_ 30(1): 28-42.

- Tatchell, M. 1983. "Measuring hospital output: a review of the service mix and case mix approaches." _Social Science and Medicine_ 17(13): 871-83.

- Thomas, D., V. Lavy, and J. Strauss. 1996. "Public policy and anthropometric outcomes in Cote d'Ivoire." _Journal of Public Economics_ 61: 139-54.

- Turner, A. G., G. Angeles, A. 0. Tsui, M. Wilkinson, and R. J. Magnani. 2001. "Sampling Manual for Facility Surveys." MEASURE Evaluation and USAID.

- Vitaliano, D. F. and M. Toren. 1994. "Cost and efficiency in nursing homes: a stochastic frontier approach." _Journal of Health Economics_ 13(3): 281-300; discussion 41-3.

- Wagstaff, A. 1989a. "Econometric Studies in Health Economics: A Survey of the British Literature." _Journal of Health Economics_ 8(1): 1-51.

- Wagstaff, A. 1989b. "Estimating Efficiency in the Hospital Sector: A Comparison of Three Statistical Cost Frontier Models." _Applied Economics_ 21(5): 659-72.

- Wagstaff, A. and H. Barnum. 1992. "Hospital cost functions for developing countries." Policy Research Working Paper 1044. World Bank, Washington, D.C.

- WHO. 1988. "Estimating Drug Requirements: A Practical Guide." WHO Action Programme on Essential Drugs and Vaccines, WHO/DAP/88.2: Geneva.

- WHO. 2000. "Health Systems: Improving Performance." 2000 World Health Report. Geneva.

- WHO. 2001. "The Multi-Country Evaluation of IMCI Effectiveness, Cost and Impact (MCE): Progress Report May 2000-April 2001." Geneva.

46
- Williams, A. 2001. "Science or marketing at WHO? A commentary on 'World Health 2000'." _Health Economics_ 10(2).

- Wilson, J. Q. 1989. _Bureaucracy: what government agencies do and why they do it._ [New York]: Basic Books.

- World Bank. 1998. _Assessing aid: What works, what doesn 't, and why._ Oxford and New York: Oxford University Press for the World Bank.

- World Bank. 200 la. "Filipino Report Card on Pro-Poor Services: Draft Concept Note." http://www.worldbank.org/participation/filipino.htm. 28 Sept., 2001.

- World Bank. 200 lb. "Honduras: public expenditure management for poverty reduction and fiscal sustainability." World Bank: Washington, D. C. Report No. 22070. Poverty Reduction and Economic Sector Management Unit, Latin America and the Caribbean Region. Washington, D.C.

- World Bank. 2001. "Jamaica Survey of Living Conditions (JSLC) 1988-98: Basic Information." DECRG, The World Bank. Processed.

- Wouters, A. 1993. "The cost and efficiency of public and private health care facilities in Ogun State, Nigeria." _Health Economics_ 2(1): 31-42.

- Wouters, A. M. 1991. "Essential National Health Research in Developing Countries: Healthcare Financing and the Quality of Care." _International Journal of Health Planning and Management 6:_ 253-71.

- Ye, X. and S. Canagarajah. 2001. "Tracking public resource flows in Ghana's schools and clinics." Africa Region Working Paper Series. World Bank, Washington, D.C. Forthcoming.

- Zuckerman, S., J. Hadley, and L. Iezzoni. 1994. "Measuring hospital efficiency with frontier cost functions." _Journal of Health Economics_ 13(3): 255-80; discussion 335-40.

47
# **Policy Research Working Paper Series**

||**Title**|**Author**|**Date**|**Contact**<br>**forpaper**|
|---|---|---|---|---|
|WPS2938|RecurrentExpenditure Requirements<br>ofCapitalProjects.Estimationfor<br>Budget Purposes|RonHood<br>DavidHusband<br>FeiYu|December2002|MGalatis<br>31177|
|WPS2939|SchoolAttendanceand ChildLabor<br>inEcuador|GladysL6pez-Acevedo|December2002|MGeller<br>85155|
|WPS2940|The PotentialDemandforanHIV/<br>AIDSVaccineinBrazil|HillegondaMariaDutilh<br>Novaes<br>ExpeditoJALuna<br>Mois6s Goldbaum<br>Samuel Kilsztajn<br>AnaclaudiaRossbach<br>JosedelaRochaCarvalhe|December2002<br>iro|HSladovich<br>37698|
|WPS2941|IncomeConvergenceduringthe<br>DisintegrationoftheWorld<br>Economy,1919-39|BrankoMilanovic|January2003|PSader<br>33902|
|WPS2942|WhyisSonPreferencesoPersistent<br>inEastand SouthAsia<sup>9 </sup>ACross-<br>CountryStudyofChina, India,andthe<br>Republicof Korea|MonicaDasGupta<br>Jiang Zhenghua<br> LiBohua<br>XieZhenming<br>WoojinChung<br>BaeHwa-Ok|January2003|MDasGupta<br>31983|
|WPS2943|CapitalFlows,CountryRisk,<br>andContagion|Norbert Fiess|January2003|Rlzquierdo<br>84161|
|WPS2944|Regulation, Productivity,and<br>GrowthOECDEvidence|GiuseppeNicoletti<br>StefanoScarpetta|January2003|Social Protection<br>AdvisoryService<br>85267|
|WPS2945|Micro-FinanceandPoverty Evidence<br>Using PanelDatafromBangladesh|ShahidurRKhandker|January2003|DAfzal<br>36335|
|WPS2946|RapidLabor Reallocationwitha<br>StagnantUnemploymentPool.The<br>PuzzleoftheLabor MarketinLithuania|JanRutkowski<br>|January2003|JRutkowski<br>84569|
|WPS2947|TaxSystemsinTransition|PradeepMitra<br>NicholasStern|January2003|STassew<br>88212|
|WPS2948|TheImpactofContractualSavings<br>InstitutionsonSecuritiesMarkets|GregorioImpavido<br>AlbertoRMusalem<br>ThierryTressel|January2003|PBraxton<br>32720|
|WPS2949|IntersectoralMigrationinSoutheast<br>Asia.EvidencefromIndonesia,<br>Thailand,andthePhilippines|RitaButzer<br>YairMundlak<br>DonaldFLarson|January2003|PKokila<br>33716|
|WPS2950|IstheEmerging NonfarmMarket<br>Economythe RouteOutofPoverty<br>inVietnam<sup>9</sup>|DominiquevandeWalle<br>Dorothyjean Cratty|January 2003|HSladovich<br>37698|
|WPS2951|LandAllocationinVietnam's<br>AgrarianTransition|MartinRavallion<br>DominiquevandeWalle|January 2003|HSladovich<br>37698|
# **Policy Research Working Paper Series**

- **Contact**

- **Title Author Date for paper**

- WPS2952 The Effects of a Fee-Waiver Program Nazmul Chaudhury January 2003 N. Chaudhury on Health Care Utilization among the Jeffrey Hammer 84230 Poor Evidence from Armenia Edmundo Murrugarra