Measuring the Impact 

of Health Systems Strengthening 

A Review of the Literature
1

Measuring the Impact 

of Health Systems Strengthening

A Review of the Literature

Written by (in alphabetical order)

Soumya Alva 

Eckhard Kleinau

Amanda Pomeroy 

Kathy Rowan

November 2009 

U.S. Agency for International Development 
2

Acknowledgments

This report was prepared for the United States Agency for International Development (USAID) by the 

Analysis, Information Management & Communications (AIM) Activity. Sincere thanks to the Bureaus 

for Europe and Eurasia (E&E), Latin America and the Caribbean (LAC), and Global Health of USAID for 

creating the opportunity and entrusting the AIM Activity to conduct this literature review. The authors 

wish to thank Susanna Baker, Forest Duncan, Bob Emrey, Roshni Ghosh, Ligia Paina, Kelly Saldana and 

others who reviewed and provided insightful comments to the drafts of this report. Credit is also due to 

the AIM Activity communications team for its support in editing and producing this report. The AIM

Activity is funded by USAID and managed by MasiMax Resources, Inc., with ICF Macro; John Snow,

Inc.; and Insight Systems (contract no. GS-10F-0311M, order no: GPO-M-00-05-0043-00). The AIM 

Activity provides the Bureau for Global Health and others with essential information, products, and 

services about program needs, technologies, costs, and impacts to support accurate priority setting, 

program design, management, and evaluation. The authors’ views expressed in this publication do not 

necessarily reflect the views of USAID or the U.S. Government.
3

Contents

Acknowledgments........................................................................................................................... 2

Acronyms and Abbreviations ......................................................................................................... 4

Introduction .................................................................................................................................... 6

Part I: Summary of Findings ........................................................................................................ 7

Defining Health Systems ........................................................................................................................ 8

Why Do Health Systems and Their Performance Matter? ................................................................. 9

Past and Current Efforts in Measuring Health Systems Performance ........................................... 10

Health Systems Indicators and Data ................................................................................................... 15

Data and Indicators: Limitations, Uses, and Criteria ....................................................................... 21

Part II: Detailed Findings by Health System Building Block .................................................... 23

Service Delivery .................................................................................................................................... 23

Health Workforce ................................................................................................................................. 29

Information ........................................................................................................................................... 34

Medical Products, Vaccines, and Technologies ................................................................................. 42

Financing ............................................................................................................................................... 50

Leadership and Governance ................................................................................................................ 56

References .................................................................................................................................... 66

Appendix 1: Indicators by Organization ..................................................................................... 73

Guide to Appendix 1: ........................................................................................................................... 94

Appendix 2: Health Output Indicators ........................................................................................ 98

Tables

Table 1: Past and Current Efforts in Measuring Health Systems Performance ...................... 12

Table 2: Most Recommended Health Systems Indicators by Building Block (BB) – WHO & 

Other Sources ............................................................................................................................... 19
4

Acronyms and Abbreviations

CGD Center for Global Development

CPI Corruption perceptions index

CPIA Country Policy and Institutional Assessment 

CSO Civil society organization

DfID U.K. Department for International Development

DHS Demographic and Health Surveys 

DOS Department of State

DPT Diphtheria-pertussis-tetanus

E&E Europe and Eurasia

ECHI European Community Health Indicators 

ECHIM European Community Health Indicators Monitoring 

EO European Observatory 

EPHF Essential public health functions

EU European Union 

GGE General government expenditure 

GGHE General government health expenditure 

HAI Health Action International 

HFC Health Facility Census

HIS Health information system

HISPIX Health information system performance index

HMIS Health management information systems

HMN Health Metrics Network

HSAA Health Systems Assessment Approach

HSS Health systems strengthening

JSI John Snow, Inc.

LAPM Long-acting and permanent method 

LSMS Living Standards Measurement Study 

M&E Monitoring and evaluation 

MCA Millennium Challenge Account 

MCC Millennium Challenge Corporation 

MCH Maternal and child health

MDG Millennium Development Goal 

MICS Multiple Indicator Cluster Surveys 

MOH Ministry of health 

MSH Management Sciences for Health

NGO Nongovernmental organization 

NHA National Health Account 

OECD Organization for Economic Cooperation and Development

OPD Outpatient department 

PAHO Pan American Health Organization

PEPFAR U.S. President’s Emergency Plan for AIDS Relief

PHRplus Partners for Health Reform Plus

QAP Quality Assurance Project

RPM Rational Pharmaceutical Management

SAM Service Availability Mapping

SBA Skilled birth attendance

SFA Strategic Framework for Foreign Assistance
5

SPA Service Provision Assessment

SWEF System-Wide Effects of the Fund 

TB Tuberculosis

THE Total health expenditure

TI Transparency International

UNICEF United Nations Children’s Fund 

URC University Research Corporation

USAID United States Agency for International Development

WHO World Health Organization 

WHO/EMRO World Health Organization Regional Office for the Eastern Mediterranean 

WHOSIS World Health Organization’s Statistical Information System

WHO/WPRO World Health Organization Regional Office for the Western Pacific

WHS World Health Surveys 
6

Introduction

Funding for the health sector in developing countries has grown significantly in the last decade. While 

health sector investments have increasingly focused on alleviating the impact of HIV/AIDS, tuberculosis, 

and malaria, both development actors and the governments of developing countries are shifting their 

priorities toward linking disease-specific interventions with longer-term investments in health systems 

strengthening (HSS). With increased funding, there comes a greater need for evidence-based 

decisionmaking and rigorous monitoring and evaluation of HSS programs.

The purpose of this literature review is to summarize current efforts in measuring health system 

performance and to highlight the indicators and performance benchmarks most frequently used by the 

global community. The review also aims to serve as a resource for health system experts working on 

building consensus around a core set of indicators for monitoring and evaluating health system 

performance. The literature review is particularly useful in the context of the Millennium Development 

Goals and the “Countdown to 2015” initiative, as well as in the context of current U.S. Government 

presidential initiatives, bilateral programs, and host-country planning processes. 

The review builds on the latest available published work on health system performance indicators and 

existing data sources. Part I summarizes the review findings, and part II provides a detailed overview of 

each of the six health system building blocks identified by the World Health Organization. 

The current version of this document has benefited from discussions with the United States Agency for 

International Development technical experts and implementing partners. Because of the growing global 

emphasis on monitoring and evaluating health system performance, the document will be periodically 

updated to reflect the most current thinking in the field. Future versions will build on the current one and 

incorporate feedback received through the online portal, as well as from published updates from academia, 

multilateral organizations, nongovernmental organizations, and special initiatives.
7

Part I: Summary of Findings

Over the years, the World Health Organization (WHO) has placed increased importance on health 

systems, treating them as the means to deliver effective and affordable interventions to those in need. This 

is particularly relevant to meeting the Millennium Development Goals (MDGs) and to achieving better 

health equity among all population sectors, especially the poor. The principal aim of health systems 

strengthening (HSS) interventions is to improve three aspects of priority health services: access, quality, 

and utilization. However, another crucial element of HSS is to strengthen host-country health systems in 

order to ensure sustainability over time and to eventually phase out donor assistance to the health sector. 

The HSS at the country level is based on high-impact and cost-effective interventions in maternal and 

child health (MCH), family planning and reproductive health, HIV/AIDS, tuberculosis (TB), malaria, and 

other infectious diseases. 

The United States Agency for International Development (USAID) plays a key role in strengthening 

health systems in developing countries by working with individual governments, nongovernmental 

organizations (NGOs), and donor agencies. USAID utilizes various contract mechanisms to support 

health systems through bilateral projects in individual countries as well as through centrally funded 

activities that support multiple countries. The Agency partners with international organizations and other 

bilateral donors to strengthen health systems to maximize the effectiveness of its efforts. USAID’s current 

central project activities on HSS include Health Systems 20/20, Strengthening Pharmaceutical Systems, 

the Health Care Improvement Project, Drug Quality and Information, MEASURE DHS, and MEASURE 

Evaluation. The main contributions of these projects have been assessment methodologies and 

instruments such as the Demographic and Health Surveys, Service Provision Assessment, and National 

Health Accounts. They also have produced practical guidelines, for example, about health facility 

assessment methods (MEASURE Evaluation, 2006) and performance indicators (MEASURE Evaluation, 

2007). A more comprehensive list can be found in Table 1. The guidelines and tools are used in several of 

USAID’s bilateral projects that support health systems, for example, in Armenia, Bolivia, Ethiopia, 

Guatemala, Indonesia, Kyrgyzstan, Nepal, Nicaragua, Tanzania, Zambia, and many other countries. 
8

Defining Health Systems

WHO describes a health system as consisting of all the organizations, institutions, resources, and people 

whose primary purpose is to improve health. It needs staff, funds, information, supplies, transport, 

communications, and overall guidance and direction. And it needs to provide services that are responsive 

and financially fair, while treating people decently (WHO, 2007). Strengthening health systems involves

addressing key constraints related to health worker staffing, infrastructure, health commodities (such as 

equipment and medicines), logistics, tracking progress, and effective financing. 

The 2000 WHO World Health Report examines and compares aspects of health systems around the world. 

It provides conceptual insights into the complex factors that explain how health systems perform and 

offered practical advice on how to assess performance and achieve improvements with available 

resources. The report focuses on the performance (ultimate outcomes) of health systems, described health 

system functions (stewardship, resource creation, service provision, and financing), emphasizing the 

stewardship role of the government. However, the key failing elements of the report are that it provides 

little information on why a particular system setup yields a certain outcome, what features of that system 

contribute the most to the outcome, and how one could restructure the system to achieve a better outcome. 

Later WHO reports have focused on health systems performance – the 2006 World Health Report, for 

example, covers the need for human resources for health (WHO, 2006a), and the 2007 report, 

Everybody’s Business: Strengthening Health Systems to Improve Health Outcomes: WHO’s Framework 

for Action, develops a conceptual framework for understanding how health systems operate, which also 

provides the basis for a toolkit of HSS monitoring indicators (WHO, 2007). 

Source: WHO, 2007.
9

The framework developed in the 2007 report depicts a health system in terms of six core building blocks, 

shown in the diagram on page 8. 

The main purpose of the framework is to portray what a health system is and what constitutes HSS. Using 

a single framework is important for understanding where investment is necessary to provide better access, 

coverage, quality, and safety, resulting in improved health. 

These building blocks represent the basic functions of health systems. They cannot function 

independently of one another; systems strengthening activities in one building block have repercussions 

on the functions of another. This interconnectedness is particularly evident in the case of governance and 

information systems. Information provides the evidence base for overall sector policies, although not 

always observed in practice, and leadership and governance impacts the performance of all the other 

health system blocks. Financing and human resources for health are core inputs that affect all other 

building blocks. Medical products, vaccines, and technologies impact service delivery and reflect the 

availability and distribution of care, which are immediate outputs of the health system. Based on this 

framework, WHO developed a toolkit for monitoring HSS that contains six sections with draft indicators, 

each section corresponding with a building block (WHO, 2008). 

Why Do Health Systems and Their Performance Matter?

A health system is a means to an end. The main objective of a good health system is to improve people’s 

lives tangibly every day. The health system is the means to achieve better health outcomes, such as better 

child survival through immunization, improved maternal health through emergency obstetric care and 

birth spacing, and lower levels of incidence of HIV, malaria, and other infectious diseases through 

prevention. 

The characteristics of a well-performing health system are greater equitability, efficiency, and 

sustainability of health service outputs by delivering accessible, high-quality, and affordable curative and 

preventive services (Rockefeller Foundation, 2008). As a recent review commissioned by the U.K. 

Department for International Development (DfID) explains, the purpose of improved measurements of 

health systems performance is threefold: 

 

• To provide better accountability for expenditures on health 
10

• To increase aid effectiveness through more efficient allocation of resources and better 

performance management

• To increase interest in performance- and results-based aid, especially when aid disbursements are 

related to results (Walford, 2007) 

As a result, health systems performance rather than just health funding is all the more important for 

enabling improvements in health outputs. Following this line of thought, strengthening health systems and 

making them more equitable have been recognized as key strategies for fighting poverty and fostering 

development. A review by Marchal et al. (2009) finds that global health actors implement very different 

interventions in their field projects. These can be categorized as: 

(1) providing inputs or resources; 

(2) reinforcing capacities of health services that are directly related to implementation of diseasecontrol programs; and 

(3) integrating program activities into general health services. 

This process, whether implemented by individual governments, NGOs, or donor agencies, is under way in 

many countries to better respond to their population’s needs. A growing number of WHO member states 

and the world’s political and international health leaders also recognize the urgent need to make a major, 

sustained commitment to strengthening health systems. 

Past and Current Efforts in Measuring Health Systems Performance

With the increased emphasis on health systems performance, WHO has taken a lead role in identifying 

appropriate measurement indicators that can be used for multiple purposes, such as: 

• In-country planning and monitoring 

• Assessment of country performance 

• Results-based funding 

• Making intercountry comparisons to aid funding allocations 

Organizations such as the GAVI Alliance and The Global Fund to Fight AIDS, Tuberculosis and Malaria 

already use indicators for these purposes. GAVI encourages proposals on HSS where the link to 

immunization coverage can be established. The Global Fund uses health information systems (HIS) for 

performance-based disbursements (HMN, 2006). 
11

Other organizations and working groups, such as the WHO Health Metrics Network (HMN), The World 

Bank, the Global Health Indicators Working Group of the Center for Global Development (CGD), the 

University of Washington Institute of Health Metrics and Evaluation, and USAID, also have developed

country indicators of health system performance, with plans to test them in selected countries (Walford, 

2007). The Interagency Group on Health System Metrics, which comprises WHO, The World Bank, 

HMN, The Global Fund, and GAVI, are working together to compile summary indicators of health 

systems performance as well as a longer list of indicators for countries to choose from, with the objective 

of monitoring support to countries, provided by The Global Fund and GAVI. Beside these collaborative 

efforts, organizations have devised indicators and tools to measure health systems performance to meet 

their program needs. For example, The World Bank has developed indicators as part of its results-based 

financing for health; USAID projects have developed health system assessment tools covering MCH

services; and the Organization for Economic Cooperation and Development (OECD) tracks health care 

quality as part of its Health Care Quality Indicators project through internationally comparable indicators.

Table 1 summarizes past and current efforts in measuring health systems performance. The tools and 

indicators developed by various organizations that examine the indicators proposed for each of the health 

systems building blocks according to the WHO framework also will be cited in the remaining sections of 

this paper.
12

Table 1: Past and Current Efforts in Measuring Health Systems Performance

Organization Publications, Tools, and Assessment Methods (in italics)

1. WHO • Everybody’s Business: Strengthening Health Systems to Improve Health Outcomes: 

WHO’s Framework for Action (WHO, 2007)

• Toolkit on Monitoring Health Systems Strengthening (WHO, 2008)

• Service Availability Mapping (SAM)

(http://www.who.int/healthinfo/systems/serviceavailabilitymapping/en/index.html) 

• Prevention SAM (http://www.who.int/healthinfo/systems/sampsam/en/index.html)

• Action Program on Essential Drugs, Indicators for Monitoring National Drug 

Policies: A Practical Manual (WHO, 1999)

• WHO Medicines Strategy (WHO, 2008a)

• Level I and Level II Indicators to Assess Country Pharmaceutical Situations 

(WHO, 2006)

• Public Education in Rational Drug Use: A Global Survey (Fresle & Wolfheim, 1997)

• Developing Health Management Information Systems: A Practical Guide for 

Developing Countries (WHO, 2004)

• Alliance for Health Policy and Systems Research (http://www.who.int/alliancehpsr/en/) 

• Framework for Assessing Health System Governance (WHO/EMRO, 2007) 

• Essential Public Health Function Measurement (WHO/PAHO, 2008) 

• World Health Surveys (WHS) (http://www.who.int/healthinfo/survey/en/index.html) 

2. WHO/HMN • Health Metrics Network (HMN) Framework and Standards For Country Health 

Information Systems (HMN, 2008)

• HMN Health Information System Assessment Tool (HMN, 2008a)

3. UNICEF • Multiple Indicator Cluster Surveys (MICS)

(http://www.unicef.org/statistics/index_24302.html)

4. USAID MEASURE DHS: (http://www.measuredhs.com/)

• Service Provision Assessment (SPA)

• Demographic and Health Surveys (DHS)

5. USAID MEASURE Evaluation/International Health Facility Assessment Network (IHFAN)

(http://www.cpc.unc.edu/measure/networks/ihfan) 

• Profiles of Health Facility Assessment Methods

• Guidance for Selecting and Using Core Indicators for Cross-Country Comparisons of 

Health Facility Readiness to Provide Services

• Facility Audit of Service Quality (FASQ)

• Signature Domain and Geographic Coordinates: A Standardized Approach for 

Uniquely Identifying a Health Facility

• Quick Investigation of Quality (QIQ): A User’s Guide for Monitoring Quality of Care 

in Family Planning

• Performance of Routine Information System Management (PRISM)

6. USAID • Child Survival and Technical Support Plus: Rapid Health Facility Assessment 

(R-HFA) (http://www.childsurvival.com/rhfa_1.cfm)

7. USAID • Logistics System Assessment Tool (LSAT) (DELIVER, 2009)

8. USAID • Strategic Pathway to Reproductive Health Commodity Security (SPARHCS) (Hare, 

Hart, Scribner, & Sheperd, 2004)

9. USAID • Contraceptive Security Index (DELIVER, 2006)

• Family Planning Effort Index (Ross & Stover, 2001)

10. USAID ACQUIRE Project: (http://www.engenderhealth.org/our-work/majorprojects/acquire.php) 

• ACQUIRE Evaluation of LAPM Services (ELMS)

11. USAID • Capacity Project: Human Resource Information System (HRIS) Strengthening

(http://www.capacityproject.org/)
13

Table 1: Past and Current Efforts in Measuring Health Systems Performance

Organization Publications, Tools, and Assessment Methods (in italics)

12. USAID Population Council: Population Council Health Facility Assessment (PCHFA) 

(http://www.populationcouncil.org)

13. USAID Health Systems 20/20 (http://www.healthsystems2020.org/):

• Health Systems Assessment Approach (with Rational Pharmaceutical Management 

Plus /QAP) 

• National Health Accounts (NHAs) 

• Health systems database

• The System-Wide Effects of the Fund (SWEF) research protocol with the SWEF

Research Network (Bennett & Fairbank, 2003)

14. USAID Rational Pharmaceutical Management Plus/QAP

(http://www.msh.org/projects/rpmplus/):

• Rapid Pharmaceutical Management Assessment: An Indicator-based Approach

15. USAID • Strategic Framework for Foreign Assistance (SFA) Governance Indicators

(http://www.state.gov/f/c23053.htm) 

16. USAID • Graduation Report on Increased Health Promotion and Access to Quality Health 

Care in the Europe and Eurasia (E&E) Region (internal document, developed by 

E&E Bureau’s Program Objective Team for Strategic Objective 3.2) (USAID 

Program Objective Team 3.2, 2001)

17. U.S. President’s 

Emergency Plan for 

AIDS Relief

(PEPFAR)

•Health Facility-based Survey of Human Resource for Health in HIV/AIDS, TB, 

Malaria and MCH Services

18. Japan International 

Cooperation Agency 

(JICA)

• Health Facility Census 

19. The World Bank • Statistical Capacity Indicator

• Results-based Financing in Health

• Country Policy and Institutional Assessment (CPIA) Index

• Living Standards Measurement Study (LSMS)

• Public Expenditure Review (PER)

• Quantitative Service Delivery Surveys (QSDS)

• Public Expenditure Tracking Survey (PETS)

• Performance Monitoring Indicators: A Handbook for Task Managers (Mosse & 

Sontheimer, 1996)

• Benefit Incidence Analysis (BIA): BIA tools and evaluations have been made publicly 

available as part of the Research Department of the World Bank. 

20. The Global Fund to 

Fight HIV/AIDS, 

Tuberculosis and 

Malaria 

• Global Fund Monitoring and Evaluation Toolkit (The Global Fund, 2006 and 2008)

21. The Global 

HIV/AIDS Initiatives 

Network (GHIN)

• Country studies examine the national and subnational effects of global HIV/AIDS 

initiatives (by The World Bank Global HIV/AIDS Program, Global Fund, and 

PEPFAR) at the country level. (Gottret & Schieber, 2006)

22. GAVI Alliance • Guidelines for GAVI Alliance Health System Strengthening (HSS) Applications (GAVI 

Alliance, 2007)

23. DfID • Future Health Systems consortium, with the aim of publishing health systems

evaluations (http://www.futurehealthsystems.org)

Health Systems Resource Centre (http://www.dfidhealthrc.org/)
14

Table 1: Past and Current Efforts in Measuring Health Systems Performance

Organization Publications, Tools, and Assessment Methods (in italics)

24. RAND • Quality of Care Assessment Tools (QA Tools)

(http://www.rand.org/health/surveys_tools/qatools/index.html) 

25. United Nations

MDGs

• MDG indicator list (http://www.un.org/millenniumgoals/) 

26. University of 

Washington Institute 

of Health Metrics and 

Evaluation

• Evaluation criteria for health systems currently in development

(http://www.healthmetricsandevaluation.org/) 

27. Harvard University 

School of Public 

Health

• International Health Systems Program: Carried out small-scale health systems 

evaluations (http://www.hsph.harvard.edu/ihsg/research.html)

28. Center for Global 

Development

• Measuring Commitment to Health. Global Health Indicators Working Group Report 

(Becker, Pickett & Levine, 2006). Recommended Millennium Challenge Account 

indicators for country selection.

29. Millennium 

Challenge 

Corporation (MCC)

• MCC Selection Criteria (http://www.mcc.gov/mcc/selection/index.shtml)

30. European 

Community Health 

Indicators Monitoring 

(ECHIM) Project 

• ECHIM Shortlist of Indicators

(http://www.healthindicators.org/healthindicators/root/o13.html) 

• ECHIM Comprehensive List of Indicators

(http://www.healthindicators.org/healthindicators/object_document/o5415n28314.ht

ml)

31. International Health 

Partnership Plus (IHP 

+)

• International Health Partnership Plus was launched in 2007 to respond to the MDG 

challenges that called for action to scale up coverage and use of health services and 

deliver improved outcomes against the health-related MDGs and universal access 

commitments. The focus is on health-related MDG outcomes. The Partnership 

produces monitoring and evaluation frameworks and country-level reports. A focus of 

the partnership is on health systems performance. 

(http://www.internationalhealthpartnership.net/en/home)

32. European 

Observatory on 

Health Systems and 

Policies

• The Observatory is a partnership of the WHO Regional Office for Europe; the

Governments of Belgium, Finland, Norway, Slovenia, Spain, and Sweden; the Veneto 

Region of Italy; the European Investment Bank; The World Bank; the London School 

of Economics and Political Science; and the London School of Hygiene and Tropical 

Medicine. Its work includes the “Health Systems in Transition (HiTs)” country 

profiles and other analyses of the dynamics of health care systems in Europe.

(http://www.euro.who.int/observatory)

33. OECD • OECD’s Health Care Quality Indicators Project’s aim is to collect internationally 

comparable data reflecting health outcomes and health improvements attributable to 

medical care delivered in OECD countries. It produced five technical papers on health 

care quality indicators, covering cardiac care, diabetes, primary care and prevention, 

mental health, and patient safety. It also participated in the ECHIM effort.

(http://www.oecd.org/document/31/0,2340,en_2649_33929_2484127_1_1_1_1,00.ht

ml)

Note: Household surveys such as USAID’s Demographic and Health Surveys (DHS), the Multiple Indicator 

Cluster Surveys (MICS) of the United Nations Children’s Fund (UNICEF), and WHO’s World Health Surveys 

(WHS) are included because they contain information related to health systems performance and overall goals and 

outcomes shown in the WHO framework.

 
15

The past and current efforts in measuring health systems performance listed in table 1 serve different 

needs. Some are broad assessments of health systems performance, such as the WHO Toolkit on 

Monitoring Health Systems Strengthening, while others target specific systems components, such as the 

logistics of distributing essential products or the quality of care provided for a specific disease category. 

The tools addressing a specific systems component usually cover performance measures in much greater 

detail than the comprehensive systems assessment tools. As can be expected from such a multitude of 

efforts, considerable overlap exists between various tools. While some tools complement each other (for 

example, rapid assessment tools and more comprehensive health facility audits), there exists little 

agreement on a standard set of indicators and assessment methods that can be used for a cross-country 

comparison. The WHO Toolkit and several publications from USAID’s MEASURE Evaluation and DHS 

projects are the most recent efforts toward standardization. 

Indicators also have been developed to evaluate specific health outcomes, particularly in the areas of 

HIV/AIDS and reproductive health. Some examples are:

Reproductive Health: 

• Bertrand, J., & Escudero, G. (2002). Compendium of Indicators for Evaluating Reproductive 

Health Programs. Vol. I

HIV/AIDS: 

• Management Sciences for Health/World Health Organization. (2006). Tools for Planning and 

Developing Human Resources for HIV/AIDS and Other Health Services

• U.S. President’s Emergency Plan for AIDS Relief (PEPFAR). (2009). Next Generation Indicators 

Reference Guide. Version 1.0

Furthermore, specific indicators listed in table 1, such as the Family Planning Effort Index, the 

Contraceptive Security Index, and SPARHCS, are relevant only to specific health outcomes, such as those 

related to reproductive health. 

Health Systems Indicators and Data

The international health community has come to a consensus regarding the key indicators representing 

health outputs and outcomes (see appendix 2). This paper summarizes past and current efforts on HSS 

indicators and aims to capture broad recommendations across development organizations. The 

WHO Toolkit on Monitoring Health Systems Strengthening serves as the organizing framework for the 
16

HSS indicators included in this review because the Toolkit is based on a simple framework, and there 

appears to be some consensus among development actors around several of its indicators. Where such a 

consensus has not been reached yet, the following sections provide options for further discussion. These 

options include indicators recommended by other agencies, such as the USAID Health Systems 

Assessment Approach (HSAA), SWEF Research Network, MCC, MDG indicators, the ECHIM Project, 

etc., which are captured to some degree in the indicators presented by the WHO Toolkit. The indicators 

recommended by the Toolkit, as well as by other sources, are described in the respective sections on each 

of the building blocks in this report. A summary of indicators per building block that were most often 

recommended by these organizations follows in table 2. These indicators are recommended by key 

organizations engaged in HSS and usually by multiple sources. Appendix 1 presents a detailed list of all 

indicators by donor and other organization, and includes information on thresholds, benchmarks, and 

targets. 

The challenge in selecting a few indicators measuring health systems performance from a large number of 

potential candidates lies in the fact that no single indicator can capture the performance of a systems 

building block adequately. Instead, the selection compromises between a parsimonious set of indicators 

and covering health systems core functions comprehensively. One approach to keeping the number of 

indicators small is the creation of composite indices. While this effectively leads to fewer indicators, 

indices pose several challenges that need to be carefully addressed. Firstly, composite indices are 

constructed of several – sometimes many – indicators and therefore do not reduce the burden of data 

collection. Secondly, indicators contributing to an index may not all be of equal importance and may 

require some form of weighted averages; weighing the components of an index could be perceived as 

arbitrary. Thirdly, interpreting indices might be difficult without breaking them into their individual 

components, especially when data are intended to inform decisionmaking and actions. Lastly, different 

indicators may be included in an index to reflect a higher or lower level of health system development, 

which could limit comparability across countries even when indices are normalized. Table 2 shows which 

indicators are composite indices (or where no consensus exists yet) and which indicators could be used to 

create a new index. While these indicators are generally applicable to all countries and are useful for 

cross-country comparisons, there may be other indicators that are more specific to conflict or post-conflict 

settings that are beyond the scope of this review. Furthermore, they are primarily limited to indicators that 

can be aggregated to the national level for easier application and analysis, although theoretically, 

measures of facility- or community-level care and services may be relevant.

Some summary observations from the information presented in table 2 for each building block of the 

WHO framework follow:
17

• The quality of health service delivery or service capacity is of concern to many development 

organizations, including USAID, WHO, and The World Bank. Projects often develop their own 

assessment instruments that measure a specific health systems component supported by them. For 

example, several tools focus on HIS or family planning services. USAID and other organizations 

have used numerous indicators measuring these and other health systems functions, often 

combining specific health systems areas with logistics functions; the latter falls under a separate 

building block in the WHO Toolkit. Consensus about quality-of-service indicators or an 

appropriate composite index has not been reached yet. One reason for this might be that 

measurements of quality of care are likely to differ by region and by the level of a country’s 

development. Some consensus seems to exist about indicators measuring availability of services, 

although, as table 2 shows, these indicators do not capture the unequal distribution of services in 

many developing countries well. 

• The issue of unequal distribution is addressed by indicators under health workforce, including 

geographic availability by type of health worker. While important, a uniform measure of country 

actions to strengthen its health workforce has not yet been agreed upon. 

• The WHO Toolkit, as well as HMN publications, emphasizes health information systems

performance. A proposed HIS performance index consists of 29 individual measurements. Other 

indicators require simple qualitative assessments and yes or no answers, such as the existence of 

reports or certain institutional responsibilities. The indicators shown in table 2 are discussed by 

WHO and other organizations. They are a subset of a larger number of indicators recommended by 

WHO. The large number of indicators related to information gathering and processing in the health 

sector seems somewhat disproportionate to the relatively small number of indicators proposed 

under service delivery.

• While some consensus seems to exist about indicators measuring medical products, vaccines, 

and technologies, these face challenges similar to those of the quality-of-service indicators under 

the service delivery building block. Which commodities are considered “essential” or “tracer 

products” will vary by region and a country’s level of development; this means that different items 

may contribute to an index in a different context, and this may limit cross-country comparability 

even when indices are normalized. The proposed index for measuring the availability of tracer 

medicines and commodities can be composed of as many as 61 individual items. Some potential 
18

supplemental indicators are much less demanding and require simple qualitative assessments and 

yes or no answers, such as the existence of guidelines or policies.

• Health financing is covered by a relatively small number of indicators; WHO proposes three. Two 

of the three, total health expenditure per capita and general government health expenditure as a 

proportion of total government expenditure, are well established and reported annually by 

international organizations. USAID routinely uses these indicators and variations of them. The 

main challenges relate to the need to better measure inequities in health service financing and 

whether resources reach the poorest and most vulnerable population groups. 

• Leadership/governance (stewardship) is another building block characterized by a large number 

of proposed measurements. A proposed policy index consists of 10 components that are qualitative 

and relatively easy to assess and report as yes or no answers (although the latter could be refined 

by assessing whether individual policy components meet minimum standards of effectiveness). In 

addition to the policy index, WHO proposes six marker indicators for governance that overlap to 

some extent with other health systems building blocks, such as financing, medical products, and 

health workforce. The World Bank is using a country policy and institutional assessment index

composed of 16 subcomponents. As for health service financing, the challenge remains of 

measuring whether the needs of the poorest and most vulnerable population groups are identified 

and met.
19

Table 2: Most Recommended Health Systems Indicators by Building Block (BB) – WHO & Other Sources

BB Health Systems Indicators (indicator in italics) Sources Service delivery

• Number and distribution of inpatient beds per 10,000 population (service availability)

• Number and distribution of health facilities per 10,000 (or 1,000) population (service availability)

• Proportion of health facilities that meet basic service capacity standards – a composite index of 

five components with 20 items in total (service capacity)

• Proportion of health facilities that meet basic service capacity standards/number and distribution of 

health facilities with basic service capacity per 10,000 population – a composite index of a 

maximum of nine program components with 15 items in total (service capacity)

• Number of outpatient department (OPD) visits per 10,000 population per year (service utilization)

• Service quality: 14 sample indicators suggested, plus 13 additional data sources

• WHO, multiple

• WHO, multiple

• WHO, USAID, 

multiple

• WHO

• WHO, multiple

• WHO

Health workforce

• Number of health workers/health professionals per 10,000 population/per capita/per 1,000 

population

• Distribution of health workers – by profession/specialization, region, place of work, and sex

• Annual number of graduates of health professions’ educational institutions per 100,000 population 

– by cadre

• Selected indicators for monitoring country actions for strengthening the health workforce – WHO 

suggests an additional 10 indicators for country monitoring.

• WHO, multiple

• WHO, multiple

• WHO

• WHO

Information

• Health information system performance index (HISPIX) – a summary measure based on 29 

standardized indicators for assessing data quality and the overall performance of the health 

information systems (HIS). The indicators below are part of the 29 standardized indicators.

• Existence of demographic or household surveys – an indicator proposed, in addition to five 

indicators measuring health surveys

• Percent of births and deaths registered in the country – two of three indicators measuring birth and 

death registration

• Percentage of districts that submit timely, complete, accurate reports to national level – one of 

seven indicators measuring health facility reporting

• Completion of at least one national health account in last five years – one of four indicators 

measuring health systems resource tracking

• Existence of designated mechanisms charged with analysis of health statistics – one of eight 

indicators measuring the capacity for analysis, synthesis, and validation of health data

• Availability of a national summary report that contains HIS information – one of eight indicators 

measuring the capacity for analysis, synthesis, and validation of health data

• WHO/HMN

• Central Statistics 

Office

• UNICEF/MICS,

 DHS, WHO/ 

WHS, Census 

Bureau

• MOH

• NHA program, 

Abt Associates

• MOH

• MOH

Medical products, vaccines, and technologies

• Proportion of population with access to affordable essential drugs on a sustainable basis – The 

MDG indicator on health service delivery has not been monitored regularly, but WHO recommends 

nine indicators to measure the structure and process components of access to essential drugs. 

The availability and price of essential medicines is measured through the following three indicators:

• Percent of facilities that have all tracer medicines and commodities in stock on the day of visit, 

and in the last three months – a composite index based on 61 essential medicines, commodities, 

and vaccines (availability of essential medicines) 

• Supplemented by median proportion of tracer drugs that are in stock on the day of visit, and in 

the last three months (availability of essential medicines)

• Ratio of median local medicine price to international reference price (median price ratio) for 

core list of drugs (price of essential medicines)

Supplemental indicators recommended by various sources:

• The existence and year of last update of a published national medicines policy

• Existence and year of last update of a published national list of essential medicines

• Existence of standard treatment guidelines

• Percent of drugs purchased through competitive bidding of total pharmaceutical expenditures 

• Appropriate prescription practices and rational drug use

• MDG

• HFS

• SPA

• EMPP

• HFS

• WHO/HAI

• IDPG/MSH

• MOH

• WHO/HAI

• Research studies

• WHO/WHS
20

Table 2: Most Recommended Health Systems Indicators by Building Block (BB) – WHO & Other Sources

BB Health Systems Indicators (indicator in italics) Sources Financing

• Total health expenditures (THE) per capita in international and US$

• THE as a percent of gross domestic product (GDP)

• General government health expenditure as a proportion of total general government expenditure 

(GGHE/GGE)

• The ratio of household out-of-pocket payments for health to total health expenditures

Supplemental indicators [Walford (2007), Kruk and Freedman (2008)]:

• Amount of total donor spending on health as a percent of THE

• Percent of government health spending that reaches the poorest income quintile

• Selected indicators for monitoring country actions for strengthening health financing – WHO 

suggests an additional six indicators for country monitoring.

• WHO, multiple

• USAID, multiple

• WHO, multiple

• WHO

• USAID, multiple 

• MCC/MCA, 

USAID 

• WHO

Leadership/governance (stewardship)

• WHO Policy index – consists of 10 items, each of which would be rated as zero (adequate policy 

does not exist or cannot be assessed) or one (adequate policy is available). With a maximum score 

of 10, the index consists of the following components: 

1. National health strategy 

2. Essential medicines list 

3. Policies on drug procurement

4. National strategic plan for TB

5. National malaria strategy/policy

6. United Nations General Assembly Special Session (UNGASS) national composite policy index 

questionnaire for HIV/AIDS

7. Comprehensive reproductive health policy

8. Comprehensive multiyear plan for childhood immunization

9. Key health sector documents

10. Surveys for obtaining timely client input

• Marker indicators of governance (six core indicators and one supplemental):

- Human Resources for Health: Health worker absenteeism in public health facilities

- Health Financing: Proportion of government funds that reach district-level facilities

- Health Service Delivery: Stock-out rates (absence) of essential drugs in health facilities

- Health Service Delivery: Proportion of informal payments within the public health care system

- Pharmaceutical Regulation: Proportion of pharmaceutical sales that consist of counterfeit drugs

- Voice & Accountability: Existence of effective civil society organizations in countries with 

mechanisms in place for citizens to express views to government bodies (social responsiveness 

and accountability)

- Supplemental: Disparity in coverage between lowest- and highest-income 

groups/regions/rural/urban areas

• Country Policy and Institutional Assessment (CPIA) Index – measured by The World Bank and 

based on a set of criteria that are captured in 16 subcomponents

• WHO

• MOH

• WHO

• HFS

• MOH

• NHA

• HFS

• Research studies

• MOH

• Transparency 

International

• WB, USAID

• WB

Key to Sources: DHS = Demographic and Health Surveys; EMPP = WHO Essential Medicines and Pharmaceutical Policies; HAI = Health Action International; 

HFS = Health Facility Survey; IDPG = International Drug Price Guide; MCC/MCA = Millennium Challenge Corporation/Millennium Challenge Account; MICS 

= UNICEF Multiple Indicator Cluster Surveys; MOH = Ministry of health; MSH = Management Sciences for Health; NHA = National Health Account; SPA = 

Service Provision Assessment; WB = The World Bank; WHS = WHO World Health Surveys 
21

Data and Indicators: Limitations, Uses, and Criteria 

While identifying key health systems indicators is an important step, equally essential is the availability 

of cross-national data, as well as country-specific data, for these indicators. Unfortunately, reliable data 

are often not available for many indicators. Several reasons account for this paucity of data. While a 

multitude of indicators and tools exist to measure various aspects of health systems performance (see 

tables 1 and 2), there is hardly any consensus on a core set of indicators that are measured across 

programs and countries. Furthermore, investments in monitoring health systems indicators have been 

inadequate, leading to a sketchy information base. 

This limited availability of data led to a multipartner effort to develop, fund, and support the future

implementation of a global survey on health systems, such as the Country Health Systems Survey 

(CHeSS), that would contribute to strengthening HIS and countries’ capacities to monitor their progress.

This recent effort by WHO and HMN seeks to close the information gap by focusing on data collection 

methods that will be useful at the country level. Their objective is to identify methods that provide 

reliable data to assess program performance, including district-level health systems surveys and other 

health facility surveys. As another part of its survey program, WHO has developed and implemented the 

World Health Surveys (WHS) to compile comprehensive baseline information on the health of 

populations and on the outcomes associated with the investment in health systems, including data about 

the way health systems are currently functioning and the ability to monitor health systems inputs, 

functions, and outcomes. 

USAID also has made efforts to standardize the measurement of health systems performance, often in 

collaboration with WHO and HMN. Considerable effort has already gone into the standardization of 

Service Provision Assessments (SPAs), which are implemented by MEASURE DHS. The MEASURE 

Evaluation project has published several guidelines and tools that include systematic reviews of health 

facility assessments and commonly used performance indicators. In recent years, WHO has expanded its 

efforts to encourage the use of data from facility surveys to fill gaps in information on health resources, 

including infrastructure, workforce, and service delivery, in many developing countries, thus providing a 

comprehensive picture of health systems in the areas of service status and availability of services. Other 

health systems strengthening (HSS)-related activities examine best practices and indicators of 

performance-based incentives as a means to address the problem of poor quality of services, low 

productivity of health workers, dysfunctional management behaviors, and low utilization of essential 
22

services by the poor; all contribute to poor health outcomes. The Center for Global Development Working 

Group on Performance-Based Incentives has addressed these issues.

When reviewing these indicators, it may be helpful to keep in mind the main criteria for selection. A 

review by DfID suggests that an indicator should perform well on dimensions such as data availability, 

comparability, ability to collect with accuracy and independence (reliability), relevance to health systems 

performance, association with final outcome indicators, possibility of aggregation with other indicators, 

and the likelihood that it will distort behavior negatively (Walford, 2007). Another literature review 

(including work from WHO) of the top criteria for assessing indicators identifies validity, relevance, 

precision (accuracy), reliability, sensitivity, timeliness, and cost, as well as the ability to quantify the 

measure, interpret it without modifiers, and collect it often enough for it to have statistical power (Hutton, 

2000). Equally important is the need to balance the need for information with the time and cost of data 

collection, keeping in mind the consequences on data quality when sufficient resources are not available 

(Martinez, 2008). As information on all the above-mentioned criteria are not always available, 

recommended indicators also are based on consensus by reviews performed by the major sources of the 

indicators. While data constraints for each set of indicators are addressed in the discussion of each 

building block, they also will be examined in greater details in the next step in this analysis when specific 

indicators will be identified.

The following sections describe in detail health systems indicators for each of the six building blocks of 

the WHO framework. Subsections for each building block identify the sources from which the indicators 

were taken and summarize the key indicators recommended by these sources. The last section of this 

paper presents health output indicators and relevant thresholds that can be linked to the HSS indicators. 

These are taken from the MDGs, the CGD’s Global Health Indicators Working Group, and USAID. 
23

Part II: Detailed Findings by Health System Building Block 

Service Delivery 

WHO defines service delivery as the way inputs are combined to allow the delivery of a series of 

interventions or health actions (WHO, 2001a). Service delivery is the main function the health system has 

to perform, and it is often thought of as the only function of a health system. Service delivery is an 

immediate output of the inputs of the other building blocks, such as health workforce, medical products, 

and finances (Islam, 2007). 

The measurement of service delivery has a more ambiguous scope, in part due to the less quantifiable 

nature of service delivery. There is also no one model of good service delivery, which allows for many 

variations in different settings. However, WHO defines a list of requirements that a service delivery 

system must meet (WHO, 2007). There must be efforts to increase the demand for care through public 

health outreach; a package of integrated health services that is offered based on need and availability; an 

organized provider network; effective management; and the infrastructure and logistical control to support 

providers in supplying health care. Indicators of health service delivery need to measure these various 

characteristics, as well as the adequacy of the structure of the service delivery system, while also

collecting indicators of service delivery outputs (Islam, 2007). 

Measures of service delivery outputs include access, utilization, and coverage, which indicate whether 

people are receiving the services they need (WHO, 2008). Access includes a wide array of measures, 

including physical, financial, and sociopsychological access to services. 

Physical access to services is often called the availability of services. Availability of services can be 

measured based on whether services are available within a certain maximum distance or by availability 

per capita. Data on the population distribution of health service resources are required to estimate physical 

access, and estimates of types of services rendered need to be reported by facilities (WHO, 2008). Data 

about service delivery infrastructure are easier to collect than data about other aspects of service delivery 

because they are mostly durable, tangible, and less mobile. Financial affordability of services is covered 

in the health finance section. The last dimension of access, sociopsychological access, is considered the 

acceptability of the service. Acceptability is an area that is often included as a component of the “quality 

of care” measures in service delivery indicator lists.
24

WHO, as well as many other sources cited below, focuses primarily on the physical availability of 

services. Ideally, other aspects of service delivery should be covered as well, such as acceptability and 

quality of care (safety, efficiency, and effectiveness of selected interventions) (WHO, 2008). Because of 

the difficulty in collecting these data, however, fewer indicators are available on these dimensions. 

Several sources also include coverage indicators. These types of “output” indicators are not considered by 

WHO to be part of monitoring service delivery. Therefore, they are not included here under the health 

systems building blocks but rather discussed in the last section, where health systems performance 

indicators are linked to these health systems outputs (WHO, 2008). 

Recommended Indicator Sources

The WHO Toolkit on Monitoring Health Systems Strengthening includes a list of draft indicators for 

service delivery (WHO, 2008). The Toolkit focuses largely on availability and utilization among the 

overall population of a country. In addition to the indicators on the number of hospital beds and inpatient 

facilities described later, this source also suggests a measure of the number and distribution of health 

facilities that meet basic service capacity standards. WHO notes that this measure depends on the 

availability of detailed data about basic amenities (access to water, etc.), basic equipment, infection 

control, human resources, and tracer drugs and diagnostics. This may be a measure better used within a 

country at the facility level as a checklist, rather than aggregated for cross-country comparison because 

many countries have moved toward a higher level of service delivery that requires a more sophisticated 

tool for measuring differences in service capacity. 

The USAID-funded Health Systems Assessment Approach (HSAA) effort of Rational Pharmaceutical 

Management (RPM) Plus, Health Systems 20/20, and the Quality Assurance Project (QAP) matches the 

WHO framework on several indicators but also includes a greater number of indicators intended for incountry purposes (Islam, 2007). This tool was designed so that USAID Missions could choose a smaller 

set of indicators from a larger list to better match their program activities closely. In addition to health 

systems performance indicators, the approach includes many coverage and health outcome indicators, 

such as mortality rates, immunization coverage, and contraceptive prevalence that WHO does not 

consider part of health systems indicators. 

USAID also put together a health systems indicator list for its 2001 graduation report on increased 

health promotion and access to quality health care in the Europe and Eurasia (E&E) Region (USAID 

Program Objective Team 3.2, 2001). This report used the indicator list to rate the health systems of 

countries in the E&E region. Even for this group of relatively more developed countries, there were 

significant limitations to health systems data and information. This lack of reliable data resulted in the 
25

inclusion of more qualitative indicators in the report. Like the HSAA, the report uses a number of output 

indicators for service delivery related to graduation criteria, but it does have one input measure related to 

the quality assurance process. 

Finally, the ECHIM Project is part of an effort to develop and implement health indicators in Europe and 

to develop the European Union (EU) HIS as a whole. Participants in this project include EU member 

states, Eurostat, OECD, and WHO. The list of indicators is geared toward evaluating health systems in 

more developed countries, but many of the indicators are still relevant and overlap with the health service 

delivery indicators designed for the developing world. In its focus on high-level measures of care, 

ECHIM largely matches the WHO approach to service delivery indicators. It covers the availability and 

utilization indicators in the WHO Toolkit but also go beyond with more precise measures of care, such as 

equity of access and disease-specific outcomes (ECHIM, 2008). Many of these specific indicators may 

require more data collection than is possible at present in developing countries and thus are excluded from 

this review. 

Regarding sources of data for the indicators suggested by these assessment tools, the USAID-supported 

Service Provision Assessment (SPA), conducted by ICF Macro, is a detailed and reliable source of

service delivery data. This is a respected source for this type of data because of the relatively large 

number of facilities that are included in its sample, which covers all facility types in a country. The 

downside is that samples for the client exit interview are small. Moreover, SPAs are not conducted in 

nearly as many countries as DHS or National Health Accounts (NHAs). In addition, client exit interviews 

will only capture the portion of the population who chose to receive care and therefore miss vital 

information about the reasons why people choose not to seek care when needed. 

In addition to SPAs, there are a few other sources of service delivery data. WHO’s Statistical Information 

System (WHOSIS) collects data on hospital beds in each region through its core indicators. Some of the 

WHO regions also collect outpatient visit data, but that information is not reported in WHOSIS. Data 

about health care facilities might be obtained from the ministry of health (MOH) in each country. Some 

service delivery data also can be found in The World Bank’s Quantitative Service Delivery Surveys and 

Public Expenditure Tracking Surveys, although these surveys have only been conducted in a few 

countries. 
26

Most Recommended Service Delivery Indicators

Besides a general agreement on the need for quality-of-care indicators, which are discussed below, 

several specific service delivery indicators were recommended by many of the aforementioned sources. 

• The number and distribution of inpatient beds per 10,000 population is the single mostmentioned indicator by major sources. This indicator can serve as a proxy for the availability of

health services, where more direct measures are not available. It also provides information on 

health care institutions’ capacities for care and their resource use (ECHIM, 2008b). It can be used 

to assess the adequacy of beds in relation to the population and as a measure of whether inpatient 

services are even available in the most resource-poor settings. This indicator measured relative to 

population size better represents the adequacy of services available as compared to a measure of 

the total number of beds in the country. There is no specific threshold for this indicator; generally, 

a greater number of hospital beds suggest greater availability of inpatient health services. 

• Number and distribution of health facilities per 10,000 (or 1,000) population is mentioned by 

several sources, but each has its own variation on the measure. USAID’s HSAA chose this 

indicator but suggested measuring only primary care facilities, not all health facilities. A 2008 

literature review cites this measure in terms of 1,000 population, not 10,000 population (Kruk & 

Freedman, 2008). Regardless of the permutation, the rationale behind this measure is that it 

collects data on the ratio of health resources to the total population. It is more rudimentary than 

the first indicator and more easily measurable. There are few benchmarks available for this 

indicator; a comparison with regional or peer-country averages may be the most useful way to 

gauge it.

• Measuring basic service capacity is an area highlighted by WHO in the 2008 Toolkit, suggesting 

a measure based on an index of availability of basic amenities, basic equipment, infection control, 

health workers, and tracer drugs and diagnostics in a facility. This idea has been echoed by other 

sources, although how they define the index and what they include vary by source. USAID’s 

HSAA looks only at primary care facilities and states that country standards should dictate the 

minimum equipment that facilities at each level of care should have available. This tool 

recommends that “the standard should be obtained directly from the MOH and may include 

standards or conditions other than presence of certain equipment (e.g., materials, electricity, 

running water, and laboratory services), in which case this situation should be explained.” 

Walford also notes that WHO supports the use of management capacity indicators but comments 

that there is limited availability and comparability across countries, due to the wide variations in 
27

what can be considered basic service capacity, and suggests that it would be best to choose these 

indicators in country (Walford, 2007). The SWEF Research Network does not suggest an index 

but gives a specific indicator relating to the number of new services offered by type of facility. It 

also asks about tracer drugs, but does so in the pharmaceutical section. Collecting data on service 

capacity is important to ensure that the full range of services is available to clients. The absence 

of capacity standards at the MOH would indicate lack of management capacity for the health 

system.

• WHO also suggests another basic service capacity indicator, this one more specific to a disease, 

such as malaria control, Integrated Management of Childhood Illness, safe motherhood, family 

planning, HIV/AIDS, control of sexually transmitted diseases, TB control, or control of 

noncommunicable diseases. These can be measured on their own or in any combination, given the 

type of facility that is being evaluated. It suggests that a short set of indicators should be devised 

to measure all the areas mentioned above and suggests indicators for the areas of services offered, 

staff and training, equipment, diagnostics, and treatment/prevention for each disease. USAID’s

HSAA does not include such an indicator but does ask about the availability of updated clinical 

standards for MOH-priority areas, high-burden disease areas, and/or areas responsible for high 

morbidity and mortality. The modularity of this indicator would allow for other high-risk diseases 

specific to a particular country, such as neglected tropical diseases, to be added. Both of the basic 

service capacity indices require detailed facility-level information, which increases the difficulty 

of collecting these data. A facility audit such as the SPA would be the best source for such 

information. 

• The only utilization measure to be cited across multiple sources is the number of outpatient 

department (OPD) visits per 10,000 population per year. In several countries, OPD visit rates 

significantly increase when constraints to using health services are removed, suggesting this is a 

good proxy indicator for improved access to care, although only for countries that start with very 

low rates (WHO, 2007). This measure also can suggest a basic level of functioning in the health 

system, if poverty is not an overriding concern, because use of OPD visits would stay low despite 

increased per capita income if services are poor or staff are not present. It is important to note the 

interaction of this indicator with inpatient beds/visits; in some cases, OPD visits can increase 

because of constricted supply of inpatient services, reflecting the substitution of ambulatory 

procedures for inpatient admission. This is not necessarily desirable. However, in most 

developing countries, especially those where OPD visit rates are strikingly low (such as Uganda, 
28

Burundi, and others), increases in OPD visits would mean increases in primary health care use, so 

rises in OPD visits would suggest positive movement (Sjönell, 1984).

• Quality of care is an area characterized by lack of clarity on indicator definitions. Many sources, 

including WHO, suggest there are a wide range of indicators to choose from, and each situation 

may call for different combinations. The WHO-recommended list includes a variety of indicators 

to choose from, such as providers questioning patients about medications, providers’ knowledge 

of hand hygiene/safety procedure, providers adhering to treatment protocols, patient success at 

seeing a provider before having to leave a facility, etc. (WHO, 2008). USAID’s HSAA also 

includes a secondary group of indicators that delve more deeply into quality-of-care measures for 

health service delivery (Islam, 2007). Walford’s review for DfID suggests an indicator to cover 

patient satisfaction and/or wait times (Walford, 2007), but further work is needed to create a 

usable indicator from this guidance. Kruk and Freedman cite quality-of-care indicators most often 

used in the literature: providers treating patients with respect, the quality of physician-patient 

communication, length of wait for care, use of evidence-based diagnostics and therapies, and the 

rate of avoidable hospitalization (Kruk & Freedman, 2008). RAND Health’s Quality of Care 

Assessment Tools is the most well-documented and verified listing of indicators for this area

(McGlynn, et al., 2000), but, like ECHIM’s more specific indicators, many of these measures are 

more suitable for the more advanced health and information systems of Eastern Europe and Latin 

America than for those in the more resource-poor settings of sub-Saharan Africa and South Asia. 

The context of each evaluation will have to be considered when choosing which quality-of-care 

indicators to include.

Issues and Concerns

A major consideration with all quality-of-care and service capacity indicators is again data availability. 

Particularly in the case of measuring quality of care, there is a need to consider these indicators in the 

context of evidence-based guidelines. Not only are these indicators not widely collected at present, some 

are also very subjective, and poor collection could result in noncomparable or, at worst, useless data. 

Furthermore, little data on newly emerging diseases may be available or, when available, are limited to

endemic areas. For all of these indicators, a major stumbling block will be to get data for not only a 

country’s public facilities but also its private facilities. In addition, significantly more value would be 

gained by gathering these data by facility type, so that these indicators could be disaggregated. For 

example, it would be very useful to know whether the majority of OPD visits are occurring in rural 

primary care clinics or in hospital OPDs. Currently, few sources call for this level of detail to be reported. 
29

Health Workforce

The ability of a country to meet its goals in health depends on the workforce responsible for organizing 

and delivering health services (WHO, 2008). They are the gatekeepers to health. There is ample evidence 

that the size and quality of the health workforce are positively associated with immunization coverage, 

outreach of primary care, and infant, child, and maternal survival, among other outcomes (WHO, 2006a). 

Unfortunately, there is wide variation in the type, skill, and gender mix in the health workforce across and 

even within countries (WHO, 2007). This can create chasms in care where the workforce is not sufficient 

to produce positive gains in health outcomes. Reasons for workforce shortages include geographic biases, 

migration of health workers within and across countries, poor mix of skills, lack of domestic training 

capacity, HIV/AIDS and other pandemic-related deaths, and demographic imbalances. Better knowledge 

of the composition of the health workforce will help countries optimize their use of what is currently 

available and also plan for changes to be made in the future. 

According to WHO, the health workforce includes not only clinically trained health professionals but also 

nonclinical health management and support workers. They serve in both the public and private sectors. 

Each country has different workforce needs, which will have to be estimated while taking into 

consideration the limitations of range, skill mix, and demographics of the currently available workforce. 

Among nonclinical health workers, there has been increased interest in those working in health systems 

research. Consensus among developing countries at the 2008 World Health Assembly was that health 

systems financing, policy, and management experts were “extremely needed, and that there was a strong 

demand in the local job markets for such workers” (Rockefeller Foundation, 2008). In order to measure 

this section of the workforce, there are several hurdles to overcome. First, there is no concrete definition 

of what constitutes a health system professional. Second, compared to clinical fields, there are fewer 

professional associations, universities, and international groups who track these types of professions, 

specifically in health. There are efforts under way to improve this, with WHO and the World Federation 

of Public Health Associations working on a global mapping of public health schools, institutes, and 

associations. WHO also is partnering with the University of Copenhagen on the Avicenna database, a 

global directory of education institutions for health professionals. Until these tools are fully available, and 

a definition of health systems workers is agreed upon, any indicators related to measuring this section of 

the workforce should be interpreted with caution. 

Compared with the other building blocks, fewer indicator sources deal with health workforce issues as a 

separate area of health systems. Some address them under service delivery, while others are concerned 
30

primarily with the financing and planning for human resources for health. Still other sources focus on 

health workforce management, which is more accurately measured at the facility level. WHO expects 

workforce indicators to cover whether human resources are allocated fairly and efficiently; whether 

allocation is responsive to change; and whether there are sufficient resources to achieve the best health 

outcomes. They also mention that the health workforce should be competent, responsive, and productive 

(WHO, 2007). 

Because of workforce mobility, it is sometimes difficult to collect accurate information on specific cadres 

of health workers. Where there are domestic institutions for training health professionals, it may be more 

feasible to collect exit data on the number of graduates, although this will not take into account 

individuals who leave the country or who do not go on to practice. It also will not count health workers 

who are trained in other countries but migrate after school. Data on the competency and responsiveness of 

workforces in developing countries are even harder to find, in part because of the ambiguous definition of 

these measures but also because it may be difficult to separate provider shortcomings from shortcomings 

of facilities, the supply pipeline, etc. On this subject, there is significant overlap with the service delivery

building block. 

Recommended Indicator Sources

Most sources deal with indicators pertaining to the number and distribution (density) of health 

professionals rather than the quality of the workforce. The WHO Toolkit on Monitoring Health Systems 

Strengthening devotes the majority of its indicators to this type of measure. Its listing includes both 

general and cadre-specific worker counts per 10,000 population, along with the density of workers by 

various subcategories (region, profession, sex, place of work). The suggested indicator list in a review for 

DfID largely follows this approach (Walford, 2007). This is likely because density indicators are already 

widely available, whereas quality indicators are harder to collect.

ECHIM’s workforce indicators are included under its health services category (ECHIM, 2008). It also 

focuses on the number and distribution of the workforce. It includes subdivisions within professional 

groups (for example, number of physicians practicing versus not practicing). For countries where it is 

feasible to collect these data, it would be useful to do so. 

USAID’s HSAA also includes density measures but devotes more time to the planning and financing of 

the human resources system (Islam, 2007). Its indicators deal with topics that other sources may define as 

part of information systems or finance. Its secondary component contains some indicators that are more 
31

appropriate for collection at the facility or training institution level rather than the national level. For 

instance, it asks about the incentives and benefits available to staff, in-service trainings, and the review 

process; these may vary widely from facility to facility and between private and public employers. To 

report one national-level result would be uninformative. 

The SWEF Research Network is the only major source that includes indicators that could be considered 

quality indicators. These indicators deal with periodic training of health workers and supervision. These 

indicators tend to rely more on data collected by The Global Fund and/or by human resource information 

systems, both of which have serious problems as sources of publicly available national data.

More promising sources of health workforce data include WHOSIS and the World Development 

Indicators Database, both of which have many measures of density for various health professions, 

although in developing countries, some may not be reported as frequently as others. USAID’s HSAA

suggests using MOH data, provider surveys, private provider groups, and United Nations agencies 

in country to get rural and urban density data, but any other within-country disaggregation of data on 

workforce will be more difficult to collect, with data quality suspect in more remote areas. Countries with 

SPAs will have more complete information. 

Most Recommended Health Workforce Indicators

There is only one indicator that is mentioned in more than two major sources: a measure of density of the 

health workforce. There are, however, two other thematic groups that are often mentioned, although a 

specific indicator will have to be worked out to make them measurable. 

• Number of health workers/health professionals per 10,000 population/per capita/per 1,000 

population is the most often cited indicator for health workforce. This indicator is a necessary, 

but not sufficient, measure of coverage (HMN, 2006). There are many variations on this indicator, 

so further research may be needed to decide which combination of options best fits the indicator 

selection criteria. Some sources allow for all health workers, clinical and nonclinical, to be 

included (WHO, 2008). More of these sources call for aggregate totals of clinical staff only, while 

others require this measure by profession, i.e., physicians, nurses, midwives, etc. More data are

generally available on the supply of physicians, so they may be used as a “tracer” population, if 

needed. Because very few developing countries suffer from oversupply of health professionals, 

any upward movement of this indicator would be considered a positive movement. WHO has 

found that a minimum of 2.3 physicians, nurses, and midwives per 1,000 population are needed to 

meet 80 percent coverage of skilled birth attendance (SBA), and cites another study that found a 
32

similar threshold of 2.5 per 1,000 to reach 80 percent coverage for SBA or measles vaccination

(Chen, et al., 2004; WHO, 2006a). These numbers have been cited by other sources as a general 

workforce threshold, but perhaps more work is needed to test for additional basic output 

indicators other than SBA and measles. Individually, countries can develop rough estimates of 

optimum and minimum workforce needs using WHO’s staffing requirements model (Hall, 2001). 

• Distribution of health workforce is an area that many sources touched on with various

indicators. Distribution is an important modifier to the density indicator, which can be deceiving 

if distribution is not taken into account. Some suggest measuring rural versus urban densities of 

health professionals. However, this may not measure geographic barriers or other impediments to 

care. Another group of sources suggests measuring densities by region and place of work, which 

give more detailed information but may be harder to collect. The goal would be to decrease 

disparities in density per capita/per X population across measurement units. 

• Domestic education of health professionals is another thematic area for which many sources 

included indicators, although there is no agreement on how to measure them. The WHO Toolkit

suggests measuring the annual number of graduates of health professions’ educational institutions 

per 100,000 population, by cadre. Several other sources suggest an indicator to capture the 

number of graduates from health professional schools, but this type of data is sometimes hard to 

capture even in developed countries and nearly impossible to collect in more resource-poor 

settings (WHO, 2006a). It is also important to note that the number of graduates does not directly 

translate into the number of new health workers in a country. For instance, WHO found that a 10 

percent increase in the entering class of medical students will result in only a 2 percent increase in 

the supply of credentialed doctors in 10 years (WHO 2007a). Another source suggests an 

indicator relating to the number of health professional schools in a country. This indicator is more 

easily collected but gives little useful information. 

• Indicators to monitor national workforce management. The WHO Toolkit gives additional 

“output” indicators related to the above input indicators. These measure the government outputs 

that would produce measures of density. They note that the list is neither exhaustive nor absolute, 

but they do point out the value of having indicators that can be disaggregated to monitor health 

workforce management at the country level. USAID’s HSAA includes some additional national 

workforce management indicators, but further research will be needed to merge the two different 

groups of measures. 
33

Issues and Concerns

One concern with measurement of indicators related to the health workforce is whether workers who are 

not part of the formal health system should be included in the calculation of the indicator. For instance, 

community health workers play an important role in influencing health outcomes but are not often 

included in the measurement of the health workforce. Similarly, there may be others who are not formally 

trained and hence excluded from measurement, even though they play a key role.

 
34

Information

Reliable and timely health information is an essential foundation of public health action and health 

systems strengthening, both nationally and internationally. The main objective of health information is to 

provide data to improve health service delivery and provide evidence for policy decisions that will lead to 

improved health status of the population. A well-developed information system will include all 

information relevant for health decisionmaking, including financial, programmatic, and geographic 

information about health services. The need for sound information is especially urgent in the case of 

emergent diseases and other acute health threats, where rapid awareness, investigation, and response can 

save lives and prevent broader national outbreaks and even global pandemics (HMN, 2008). The 

generation and strategic use of information, intelligence, and research on health and health systems are 

also integral parts of the leadership and governance function (WHO, 2007). Monitoring of the information 

“building block” therefore needs to capture the generation of data, the quality of the data produced, the 

creation of information, and the application and synthesis of information into knowledge for 

decisionmaking that improves health systems operations (Alliance for Health Policy and Systems 

Research, 2008). Each of these elements can be assessed as part of information performance monitoring. 

Much emphasis has been placed on information systems that produce data, while, more recently, 

monitoring and evaluation (M&E) groups and academics have examined the application of information in 

decisionmaking, as well as the financial and human resources allocated to information systems and 

application (Hanney, et al., 2003; Islam, 2007; Stansfield, et al., 2006). 

A good information system has four main functions – namely, to generate, compile, analyze and 

synthesize, and communicate and use health data – to help in the decisionmaking process (WHO, 2008). 

Publicly funded health programs are increasingly being asked to account for their performance, and 

information systems provide a means of operationalizing this quest for accountability (Perrin, et al., 1999) 

Their main objective is to produce information for improved health services and evidence-based policy 

decisions that will lead to improvements in the health status of the population. Although they do not 

directly improve or reduce health status, reliable and timely health information is an essential foundation 

of public health action and health systems strengthening, both nationally and internationally. Information 

is therefore used as a management and oversight tool to improve outcomes through the analysis of 

changes in outcomes, as well as the processes and capacity being applied to achieve the outcomes (Perrin, 

et al., 1999). 
35

According to WHO, the key components of the information building block include the generation and use 

of information; development of health information and surveillance systems; development of standardized 

tools and instruments; and collation and publication of international health statistics (WHO, 2007). These 

components should deliver timely and accurate reporting of population- and facility-based data that 

support decisionmaking and, where relevant, be utilized at the point of collection. The information system 

should also be able to detect events that threaten public health security so that authorities can investigate 

and contain the threats. The data provided by information systems also can draw attention to systems that 

are performing well or populations that are achieving better health as well as to areas in need.

Information systems include health information systems, which provide data on health status and 

integrate data collection processing, reporting, and use of the information necessary for improving health 

service delivery through better management, and health management information systems (HMIS), 

which are specially designed to assist in the management and planning of health programs, as opposed to 

the delivery of care. According to the WHO Toolkit, indicators of country HIS performance can be 

grouped into two broad types, namely: 

• Indicators related to data generation using core sources and methods (health surveys, civil 

registration, census, facility reporting, health system resource tracking). These indicators reflect 

country capacity to collect relevant data at appropriate intervals. 

• Indicators related to country capacities for synthesis, analysis, and validation of data. These 

indicators measure key dimensions of the institutional frameworks needed to ensure data quality, 

including independence, transparency, and access.

Core sources of information include: 

• Population-based data relating to the whole population, not only to groups using health facilities. 

Such data can be gathered continuously from administrative records, such as census or birth 

registries, or, periodically, through cross-sectional household surveys (HMN, 2008). 

• Institution-based data that produce data from administrative and operational activities. These

include patient records, which contain clinical information obtained in the course of providing 

health care, and administrative data collected as part of the operation of a program, such as billing 

information or number of patients served. 

Different data are needed at different levels of the health system. At a lower level, data regarding a patient, 

often presented in patient charts, are needed for patient management. At the facility and district levels, 
36

summary indicators are needed for management, planning, and procurement purposes. Indicators also are 

needed at the district level for planning and reporting to the national level. The national summary 

indicators are then used for the governance of the health system and regional or global reporting (Islam, 

2007). The guiding principles of information systems are that no health data should be requested from a 

service level to be reported to higher levels that do not have an actionable use at the recording level; 

health data should be used to analyze and solve important health and service problems; and priority 

attention should be given to improving data generation and use at the local level, to support the 

enhancement of service performance at that level (WHO, 2000). 

Recommended Indicator Sources

The WHO Toolkit identifies 29 indicators across six categories: country health surveys, birth and death 

registration, census, health facility reporting, health system resource tracking, and capacity for analysis, 

synthesis, and validation of health data. These core indicators assess the strength of the system based on 

the presence and quality of the data generated using core sources and methods (health surveys, civil 

registration, census, facility reporting, and databases), the reporting of data, and the country’s capacity for 

analysis, use, and dissemination of health data. The Toolkit also proposes an overall health information 

system performance index (HISPIX), a summary measure based on the 29 indicators that captures overall 

data quality and overall health information system performance. The sources of the data for the indicators 

in the Toolkit and for HISPIX are, for the most part, available within WHO databases and from other 

international agencies. However, the draft Toolkit has not yet identified sources for all proposed indicators.

WHO/HMN and its partners have developed the Health Information System Assessment Tool, a selfassessment tool that has been completed by 68 countries to date, with several more awaiting validation

(HMN, 2008a). HMN states that the self-assessment approach appears to have worked well, although it 

will lead to comparability issues. The tool contains more than 125 indicators, also grouped in six areas 

similar to the WHO draft Toolkit: health information system resources (policies, human resources, 

financing, and infrastructure); indicators; data sources; data management; information products; and 

dissemination and use. Scenarios are provided that allow for objective rating of the system. From these 

ratings, an overall measure is calculated.

In its Framework and Standards for Country Health Information Systems, HMN describes a minimum set 

of core health indicators that all national health information systems should be able to report (HMN, 

2008). The Framework points to the indicators used for World Health Statistics 2005, which were 

developed in part to use to monitor WHO’s efforts to strengthen health information systems. WHO aims 

for all countries to have systems in place to be able to report on these core indicators, and the extent to 
37

which a national health information system can do so is a measure of the system’s quality. The 39 core 

indicators reflect determinants of health, health status, and health systems, and were selected based on the

availability and quality of data. They include a number of indicators to measure progress toward the MDGs.

The Working Group on Health System Metrics, which involves HMN, WHO, The World Bank, The 

Global Fund, and the GAVI Alliance, has chosen one indicator for information systems: the percent of 

births and deaths registered. 

WHO also has a manual titled Developing Health Management Information Systems: A Practical Guide 

for Developing Countries to help program managers monitor the establishment and functioning of HMIS. 

It contains questions and indicators designed to assess data collection processes and identify issues with 

the current information system, including issues with the use of information. It also demonstrates how to 

identify indicators that the HIS should collect based on the national health strategy for a given area, such 

as child health or HIV/AIDS (WHO/WPRO, 2004). 

The Global Fund’s Monitoring and Evaluation Toolkit (second edition) provides four examples of 

information system indicators. Two indicators relate directly to The Global Fund goals: 1) staff and civil 

society training in M&E and 2) the presence of a nationally coordinated multiyear plan, with a schedule 

for survey implementation and data analysis prepared (The Global Fund, 2006). 

Health Systems Assessment Approach: A How-To Manual, developed by the USAID-funded Health 

Systems 20/20, Partners for Health Reform Plus (PHRplus), RPM Plus, and QAP projects, classifies 26 

HIS indicators in seven topical areas: health status indicators for both mortality and morbidity; health 

system indicators; resources, policies, and regulations; data collection and quality; data analysis; and use 

of information for management policymaking, governance, and accountability. It provides the definition 

and rationale for each indicator, as well as the sources and caveats. The first seven indicators cover health 

status and systems (different from the WHO Toolkit). These represent data collection for commonly 

agreed upon indicators of health status and systems. For example, unlike other HIS assessments, it does

not measure the existence of survey or vital registry to measure mortality or HIV prevalence but rather 

uses the indicator itself (under-5 mortality). Whether a given country’s HIS has collected and reported 

these agreed-upon indicators of health indicators is a basic level of function and capacity, and a lack of 

current data for these critical indicators would also imply serious weaknesses in the information system. 

The next 19 indicators are concerned with resources and policies (5); regulation of information systems 

(6); data analysis (6); and use of information for management, policymaking, governance, and 

accountability (2) (Islam, 2007). 
38

USAID indicators for its Strategic Framework for Foreign Assistance Operational Plan (Department of 

State [DOS], 2006) include several indicators that are designed to measure progress in capacity building 

of information systems within each element. However, they capture only activities conducted by the U.S. 

Government and do not measure the strength of national information systems. They measure such things 

as individuals trained in strategic information activities, including M&E; surveillance; the number of 

host-country institutions with improved management information systems; the number of monitoring 

plans or number of sector assessments conducted by the U.S. Government.

Other relevant indicators include The World Bank’s Statistical Capacity Indicator, which is a summary 

measure that provides an overview of the statistical capacity of developing countries. It was developed to 

assess the capacity of statistical systems using metadata generally available for most countries. The 

quality of the statistical system is also important (de Vries, 1998). The Performance of Routine 

Information System Management (PRISM) framework developed by MEASURE/Evaluation and John 

Snow, Inc. (JSI) helps managers assess the performance of HIS in terms of data quality and continuous 

information use. 

No information indicators are explicitly mentioned by the SWEF Research Network, The World Bank’s

results-based financing in health project, which plans to link funding to performance on five standard 

indicators, or the CGD Global Health Indicators Working Group. Countries also use a number of 

measures to monitor performance-based funding schemes, such as the percent of districts with disease 

surveillance reports, the proportion of districts submitting monthly reports to MOH on time, or the 

proportion of research findings translated into policy and practice. However, these are extensive, vary 

widely, and are country specific.

Most Recommended Information Indicators 

Based on this review, several indicators are recommended by multiple organizations. It is important, 

however, that indicators be chosen according to their purpose. 

• HISPIX. The WHO Toolkit proposes a health HISPIX, a summary measure based on a set of 29 

standardized indicators for assessing data quality and the overall performance of the health 

information system. The score is calculated from information available from WHO databases and 

other international agencies. Information on inputs and resources is available from country health 
39

statistics reports and from the self-assessments conducted through HMN. The indicators below are 

part of the 29 standard indicators. 

• Existence of demographic or household surveys – two or more data points available for child 

mortality in the past five years. This indicator measures the existence of demographic or 

household surveys and is relevant only to countries without complete civil registration systems (less 

than 90 percent coverage of births and deaths), based on whether the country has had more than one 

demographic or health survey. Data quality and comparability may be an issue. Because the 

indicator captures the existence of the survey and not the survey quality, it cannot be used to 

compare issues across countries. The indicator has a threshold of two or more surveys within the 

past five years. 

• Percent of births and deaths registered in the country. This indicator measures the functioning 

of the vital registration system and the availability of key indicators at the national level. The 

numerator is civil registration records and the denominator is the census.

• Percentage of districts that submit timely, complete, accurate reports to national level. A 

variation of this is “health facilities or districts reporting all indicators according to national 

guidelines.” This is a measurement of the comprehensiveness of health information management 

and capacity at the lower levels to provide the data. It also demonstrates the extent to which 

information is available on a specific disease or service statistic. Sources are administrative records 

aggregated at the national level.

• Completion of at least one NHA in last five years. The existence of an NHA demonstrates (to 

varying degrees) health systems resource tracking, health systems governance, and decisionmaking. 

NHAs provide managers with reliable national information on the sources and uses of funds for 

health. Governments in countries with NHAs can use them to understand how resources are 

mobilized and managed for the health system; who pays and how much is paid for health care; who 

provides goods and services; and how health care funds are distributed across the different services. 

• The existence of designated mechanisms charged with analysis of health statistics, synthesis of 

data from different sources, and validation of data from population and facility sources. The 

definition of this concept varies. The HSAA describes it as “the presence of mechanisms to review 

the utility of current information system indicators for the planning, management, and evaluation 

process, and to adapt and modify accordingly” (Islam, 2007). The Global Fund describes it as “a 
40

nationally coordinated multiyear plan with a schedule for survey implementation and data analysis 

prepared” (The Global Fund, 2008). The underlying rationales are similar: to provide some 

indication of the use of data in program planning. 

• Availability of a national summary report that contains HIS information, analysis, and 

interpretation (most recent year). This is an indicator of the capacity for analysis, synthesis, and 

validation of health data. It also provides a proxy for the national commitment to transparency and 

data dissemination, and provides stakeholders the opportunity to engage and take action together. 

The main difference between the HISPIX and the HMN self-assessment score is that the HISPIX 

indicators can be assessed externally on the basis of information largely available in the public domain, 

while the HMN is a self-assessment. However, indicators of health systems resources are not widely 

available and used by most agencies. The HMN assessments are one of the few sources available that 

capture the details of health information system resources and data management and quality. The WHO 

Toolkit, HSAA, and HMN all recommend indicators for data use and analysis. The extent to which “use” 

is measured varies widely. The Toolkit includes the indicator mentioned above (“the existence of 

designated mechanisms charged with analysis of health statistics, synthesis of data from different sources

...”), as well an indicator on immunization data validation that does not actually measure data use but

rather the establishment of mechanisms to do so. The HMN assessment includes 10 indicators to measure 

use of data for planning and resource allocation, but countries will vary in their assessment of 

demonstrated use of data.

Other relevant indicators not widely included for monitoring information system performance are 

indicators of health systems resource tracking. These indicators, where available, indicate the level of 

support the government provides to the HIS. Another area that varies includes indicators concerning data 

quality assessments. WHO is one of the few sources to establish a target for this, specifying that data 

quality assessments should be carried out and published within the last three years and that assessment 

should cover all routine data sources. 

One indicator that may overlap with the medicines building block of the framework is the existence of 

information on the availability of tracer medicines and commodities in facilities. Similarly, the indicator 

of the existence of a national database for human resources may overlap with the human resources 

building block. 
41

Issues and Concerns

The lack of data availability or an information system itself is an indication of a country’s HIS situation 

and may be useful in determining system needs and where capacity building should occur. Furthermore, 

several HIS indicators, such as the existence of two or more data points on child mortality, are not a 

continuous measurement, as in health status, but rather categorically defined as yes or no. 

Information systems also are difficult to compare across countries, as rarely has the same assessment tool 

been applied to all countries, and even when assessed on the same indicator, data quality may be an issue. 

Often HIS indicators are suited to the local context; data needs will vary according to the epidemiological 

profile and development (HMN, 2008a). HIS indicators also are difficult to include in progress 

monitoring, as they lack a proximity to better health outcomes, and progress in information systems is 

difficult to associate with progress toward better health or reaching MDGs (Walford, 2007). 

Differences in health systems, the level of decentralization, and the various economic and social and 

political contexts also affect the nature and effectiveness of the information system. The World Bank 

attempts to compare the quality of national information systems through its Statistical Capacity Index. 

However, the Index does not include health facility reporting and data utilization. 
42

Medical Products, Vaccines, and Technologies 

A well-functioning health system ensures equitable access to medical products, vaccines, and 

technologies of assured quality, safety, efficacy and cost, and their sound and cost-effective use (WHO, 

2007). The lack of affordable access to lifesaving medicines contributes to enormous inequities in health 

between developed and developing countries and leads to preventable mortality and morbidity, episodes 

of catastrophic illness that cause further impoverishment, low quality of life, and large-scale economic 

and health system losses. Economic factors are frequently the most important barriers to access. 

Measuring and understanding the reasons for prices of medicines and how the price affects consumption 

of health care are important in helping countries develop and establish policies that ensure the 

affordability of medicines. Yet cost is only part of measuring access to medicines.

According to WHO, access to and appropriate use of effective medicines require a complex and 

coordinated system that has five components. It must encompass: 

• National policies, standards, guidelines, and regulations that support policy and evidence-based 

selection of medicines, vaccines, and technologies according to international standards 

• Information on prices, international trade agreements, and capacity to set and negotiate prices

• Reliable manufacturing practices and quality assessment of priority products 

• Procurement, supply, and storage and distribution systems that minimize leakage and other waste

• Support for rational use of essential medicines, commodities, and equipment through guidelines and 

strategies that ensure adherence, reduce resistance, and maximize patient safety and training

Monitoring of this building block is closely intertwined with at least two other building blocks –

leadership and governance, and information – and, where rational use is concerned, the service delivery

building block. 

Recommended Indicator Sources

The indicators that measure access to medicines in the WHO indicator Toolkit are constructed around the 

MDG 8 indicator “Proportion of population with access to affordable essential drugs on a sustainable 

basis.” This was created to monitor the goal of providing access to affordable essential drugs in 

developing countries, in cooperation with pharmaceutical companies (United Nations, 2008). The target is 

formulated to capture the extent to which the pharmaceutical sector is contributing to providing access. 

The contribution of the pharmaceutical sector to access, affordability, and sustainability components of 
43

the indicator depends on both domestic and international factors. However, the data needed to form the 

MDG indicator has not been collected regularly, and the lack of comparable data is a significant problem.

Given the limitations of the MDG indicator, WHO has broken it down to two components, namely the 

structure component (policy and legal provisions) and the process component (cost, supply, and use), 

which contain nine indicators. The nine indicators include measures of national legislation and policies, 

insurance coverage, cost, and availability of medicines. The Toolkit does not include indicators on 

rational use of medicines. The Toolkit itself specifies two indicators. The first is the percent of facilities 

with tracer medicines and commodities in stock on the day of visit and in the last three months. A 

supplemental indicator is the median proportion of tracer medicines and commodities in stock on the day 

of visit and in the last three months. Data for this indicator come primarily from facility visits. However, 

comparisons across countries may vary based on the epidemiology. The second indicator compares the 

median local medicine price to the international reference price for a core list of drugs, which also 

requires the availability of facility data. 

 

The only indicator included in the 2007 draft monitoring plan of the Working Group on Health System 

Metrics is the availability of lifesaving medicines, including those for TB, antiretroviral HIV/AIDS 

treatment, and malaria (Walford, 2007). 

The SWEF Research Network included in its list two frequently cited indicators – facilities without 

stockouts and price paid compared with the international price – but also includes the percent of facilities 

with expired items, an indicator of quality (SWEF Research Network, 2003). The set included an 

indicator on affordability (average drug cost per encounter), which would require including a comparison 

to wages or purchasing power.

USAID’s Health Systems Assessment Approach: A How-To Manual contains 39 indicators related to 

pharmaceutical management, including national policies regarding medicines, indicators of availability, 

and procurement and financing (cost) arrangements (Islam, 2007). However, it provides more detailed 

indicators of procurement, pharmaceutical registration, mechanisms for licensing providers, appropriate 

use, and quality than the WHO Toolkit. It also includes indicators of private and public expenditures per 

capita on pharmaceuticals and total expenditures on medicines as a percentage of total health expenditures 

(THE), the latter being a measure of the significance of pharmaceutical spending relative to other

spending on health and of financial and institutional sustainability. 
44

In 1999, the WHO Action Program on Essential Drugs, now the Essential Medicines and Pharmaceutical 

Policies Department, issued Indicators for Monitoring National Drug Policies: A Practical Manual

which proposes a set of 31 background indicators, 50 structural indicators, 38 process indicators, and 10 

outcome indicators (WHO, 1999). These indicators are intended for self-use by developing countries to 

monitor their pharmaceutical systems. The manual allows a country to evaluate the performance of the 

pharmaceutical sector, monitor progress in the implementation of national drug policies, and assess the 

effects of changes on drug policy objectives. 

The draft WHO Medicines Strategy (2008–2013) aims to improve medicine-related progress in three 

areas: policy and access, quality safety and efficacy, and cost-effective use (WHO, 2008a). It measures 

country progress on four indicators that are assessed from eight specific expected results. The four 

country progress indicators are access to essential medical products and technologies; availability of and 

median consumer price ratio for 30 selected generic essential medicines in the public, private, and 

nongovernmental sectors; national regulatory capacity; and percentage of prescriptions in accordance 

with current national or institutional clinical guidelines.

To monitor the progress of efforts to improve the global medicines situation and measure important 

aspects of country pharmaceutical situations, WHO has developed Using Indicators to Measure Country 

Pharmaceutical Situations: Fact Book on WHO Level I and Level II Monitoring Indicators (WHO, 

2006). Level I indicators measure the existence and performance of key national pharmaceutical 

structures and processes. Level II indicators measure key outcomes of these structures and processes in 

the areas of access, product quality, and rational use. These indicators can be used to assess progress over 

time, compare situations across countries, and reassess and prioritize efforts based on the results. 

The Level I and Level II data are gathered through surveys. Level I is a questionnaire completed by health 

officials, which leads to several data limitations. The interpretation of questions and validation of data are

issues, and respondents may not have the resources to provide accurate responses (Carandang & PierreJacques, 2005). Level II indicators are completed through facility and household surveys and have been 

completed on a smaller sample of countries. It contains 13 questions on pharmaceutical-seeking behavior 

and affordability. WHO conducted a survey of Level I indicators in 2003 and Level II indicator surveys 

between 2002 and 2004. A key finding was that most countries have a national medicines policy, but few 

monitor the policy with indicators. It also was found that many basic policies fundamental to promoting 

rational use of medicines are not being implemented. 
45

WHO’s Action Program on Essential Drugs conducted a global survey (Public Education in Rational 

Drug Use: A Global Survey), on public education interventions in rational drug use (Fresle & Wolfheim, 

1997). The rationale for these indicators is the need to evaluate such interventions by measuring 

consumer’s information and education on medicines and appropriate treatment-seeking strategies.

Rapid Pharmaceutical Management Assessment: An Indicator-based Approach is a manual that 

presents an indicator-based approach for rapidly assessing pharmaceutical management systems and 

programs, and contains indicators similar to the WHO Level I and II indicators, although it covers a 

broader range of topics (MSH/URC, 1995). It presents a set of 46 indicators of performance, grouped 

under eight topics of pharmaceutical management (policy, legislation, and regulation; essential drug lists; 

health budget and finance; pharmaceutical procurement; pharmaceutical logistics; patient access and drug 

utilization; product quality assurance; and private sector pharmaceutical activity).

The ECHIM Project comprehensive indicator list (ECHIM, 2008a) and the European Observatory 

(EO) Health Systems in Transition: Template for Analysis (Mossialos, et al., 2007) also contain

indicators that intend to provide a detailed description of a health system. Both of these sources are geared 

toward evaluating more developed-country health systems and are mainly concerned with expenditures 

for pharmaceutical products and the appropriateness of prescribing practices in developed countries. The 

EO asks about the government’s ability to regulate providers. It includes regulatory indicators that could 

be applied for more developed systems, including regulations on alternative complementary medicines

and malpractice. The EO template is one of the only sources to include indicators of patient education. 

Other sources containing information on medicines include JSI’s Logistics System Assessment Tool

(LSAT) (DELIVER, 2009), which is a qualitative data collection instrument that provides a 

comprehensive system-level assessment of logistics system performance for any program that manages a 

health commodity. The indicators can be used to diagnose problems in drug supply systems, management, 

or procurement. USAID’s Strategic Framework for Foreign Assistance Operational Plan (DOS, 

2006) contains several indicators regarding medicines, particularly on the availability of drugs at U.S. 

Government-supported locations and the value of drugs purchased by the U.S. Government. 

Data on indicators of access to and use of medicines can be obtained from a range of surveys, including 

WHS (health expenditures and insurance, conducted in 2001 and 2007); NHAs (cost and expenditure 

data); WHO/Health Action International (HAI) national medicine pricing surveys (prices, availability, 

affordability, and components of medicine prices in low- and middle-income countries, conducted in 45 

countries in 1999, 2003, and 2007); The World Bank Living Standards Measurement Study (LSMS) 
46

surveys (level and distribution of out-of-pocket payments for health care and extent to which such 

payments act as barriers to health care access); and USAID-funded SPA facility surveys (availability of 

essential medicines/first-line medicines, indicators for monitoring vaccine storage conditions, adherence 

to standard treatment guidelines, and quality assurance activities). 

Most Recommended Medical Products, Vaccines, and Technologies Indicators 

From the above sources, the indicators most frequently cited are components of the MDG indicator that

collectively try to measure the proportions of national populations that have access to affordable essential 

drugs on a sustainable basis. Four dimensions are largely captured by the indicators: drug prices, drug 

availability, affordability (financial burden of drug costs felt by consumers and governments), and quality. 

• Proportion of population with access to affordable essential drugs on a sustainable basis

(MDG indicator on health service delivery). This has not been monitored regularly, and WHO 

now recommends nine indicators that are components of access to essential drugs.

• Availability and price of essential medicines. The average availability of 30 selected essential 

medicines in public and private health facilities indicates the frequency of stockouts in facilities and 

the degree of system function in financing, procurement, logistics, and distribution. This is a 

common measure but detailed definitions vary by country as medicines vary by epidemiological 

profiles and health goals. Although the WHO global target is 80 percent availability, countryspecific targets are required. Furthermore, data may not include all private sector facilities. The 

median consumer price ratio of essential medicines also has a target set by WHO of not more than 

four times the world market price. This is an indicator of cost, as well as of procurement efficiency, 

governance, and corruption. The core list of medicines tends to vary among countries, and therefore, 

results are often not comparable. The number of facilities (including pharmacies) is sometimes 

small, leading to large sampling error. Data need to be presented by type of facility and by public 

and private sectors. To evaluate affordability, several studies have compared medicine treatment 

costs to a day’s wages of the lowest-paid unskilled government worker (WHO, 2008; 

WHO/AusAID, 2006). This is a useful complement to the actual cost of the drug compared with

international prices. Drug availability and price make a composite indicator that includes the 

following three subindicators:

- Percent of facilities that have all tracer medicines and commodities in stock. The availability 

of critical supplies and frequency of stockouts in facilities are common measures of how well a 
47

system is functioning in financing, procurement, logistics, and distribution. Alternatively, some 

surveys may collect data on the percent of facilities that have all tracer medicines and 

commodities in stock on the day of the visit. Data on medicine availability can be used with data 

on other components of service capacity to assess the ability of facilities to provide specific 

services. 

- Median proportion of tracer drugs that are in stock on the day of visit and in the last three 

months. 

- Ratio of median local medicine price to international reference price (median price ratio) 

for core list of drugs. 

• The existence and year of last update of a published national medicines policy. The existence of 

such a policy indicates a commitment to improving pharmaceutical management in the public and 

private sectors. Data can be found at WHO and MOH Web sites, and in country studies. The WHO 

target is the existence of a policy updated within the last 10 years.

• The existence and year of last update of a published national list of essential medicines. This 

indicator measures a country’s commitment to rational resource allocation and containing 

pharmaceutical costs. Information can be found in National Essential Medicines Program and WHO 

reports. The WHO target is the existence of a list updated within the last two years.

• Standard treatment guidelines. These indicate the capacity to provide consistent treatment for 

common health problems. If guidelines exist, evidence-based best practices for treatments of 

common conditions are reviewed and codified. Data comparability is an issue with this indicator, as 

guidelines may vary by country. The WHO target is the existence of a policy updated within the last 

five years.

• Percent of drugs purchased through competitive bidding of total pharmaceutical expenditures.

A well-organized procurement unit should have this information readily available. This indicator 

has a WHO target of 80 percent. An estimate of the value would be acceptable in most cases if the 

question is also asked about the percentage of suppliers that are international versus national or local. 

However, not all items are best procured through competitive tenders, and country or donor policies 

may affect purchasing arrangements. In some cases, such as vaccines, there are few reliable 

suppliers, so these products are usually procured through direct purchase. 
48

• Appropriate prescription practices and rational drug use. Several publications provide 

indicators that assess the quality of drug prescribing and use, which is important for monitoring the 

impact of essential drugs programs on hospitals, providers, and consumers. Monitoring 

pharmaceutical use is a timely issue in developed countries, but resources for this are lacking in 

developing countries. As Laing and his co-authors noted in 2001, policies that affect the health 

system structure and financing may have negative impacts on individuals’ use of pharmaceuticals, 

with outcomes such as reductions in access or overconsumption. Policymakers should monitor the 

effects of implementing heath system reforms through key patient use indicators that have already 

been developed and tested (Laing, et al., 2001). 

The WHO Level I and II Fact Book includes indicators on policies for rational use, prescribing practices, 

public education campaigns, and the provision of independent information for providers and consumers 

(WHO, 2006). The WHO Toolkit does not include indicators of use. In general, indicators of use, the 

presence of a national strategy to contain antimicrobial resistance, and public education on medicines are 

less frequently mentioned by agencies or countries. Other less frequently mentioned indicators are

standards and regulation of traditional medicines, and legal provisions for generic substitutions. 

There are several international targets for access to medicines, such as the WHO and Joint United Nations 

Program on HIV/AIDS (UNAIDS) ‘3 by 5’ indicators for access to HIV/AIDS treatment, the Amsterdam 

targets for TB treatment, and the Abuja targets for malaria. These are targets for access and end goals; 

they are less useful for monitoring progress toward improved access and cost.

Another issue is government expenditures on medicines as a percent of the health budget. In 1999, WHO 

established a target of 20 percent to be sustained over a three-year period. This indicator is not included in 

the WHO draft Toolkit, nor is it widely used by other agencies. The HSAA includes a measurement of 

government expenditures on drugs but does not set a target. Also, less frequently mentioned is the 

existence of quality assessment systems for products prior to procurement, such as a drug prequalification 

process. WHO does not include prequalification as an indicator in the Toolkit, although it encourages the 

use of the WHO/United Nations prequalification program. Regarding traditional medicines, while WHO 

has put forth guidelines for the registration of herbal medicines and technical guidelines related to safety, 

efficacy, and quality, most agencies do not assess progress in the use and regulation of traditional 

medicines as part of HSS activities. However, as of 2007, about 50 countries had a national policy on 

traditional medicines, and more than 110 had various mechanisms in place to regulate these medicines. 
49

Researchers recently were able to use data from the WHO/HAI surveys to determine drug prices, 

availability, and affordability in 36 low- and middle-income countries (Cameron, et al., 2008), and other 

researchers have used 2002 WHS to examine health expenditures and medicine access among 

respondents in eight countries (Wagner, et al., 2008). The WHS indicators were used to assess out-ofpocket expenditures on medicines and barriers to access to medicines. The second phase of this research 

is now under way and will involve analytic work to assess the key determinants of medicine access and 

affordability, and the extent to which health insurance protects households from catastrophic health 

expenditures, as well as the impact of insurance on cost-related barriers to medicine access (WHO/

PAHO Collaborating Center in Pharmaceutical Policy). 

Issues and Concerns

Despite the existence of several global and regional price information services, including the WHO/HAI

database, data availability on access to and use of medicines is a critical issue. SPAs capture many of the 

facility-based aspects of medicine availability and prescribing but only have been conducted in a few 

countries. WHS capture expenditure and insurance information, but the same questions may not be asked 

in all countries. Product quality is also difficult to assess (other than by expiration date), as are

appropriate prescription practices, rational drug use, and user compliance. WHO is working to improve 

the accessibility of unique drug information at the country level and create one central Web site with links 

to all medicine-related country information relevant for planning and measuring progress (WHO, 2008). 
50

Financing 

According to WHO, the purpose of health financing is to “make funding available, as well as to set the 

right financial incentives to providers, to ensure that all individuals have access to effective public health 

and personal health care” (WHO, 2000). This building block is essential for the operation of a national 

health system – if financing were to fail, no health promotion or disease prevention would be able to 

take place. 

Health financing refers not only to funds coming from the government but also to funds spent by 

individuals on their health care (out-of-pocket expenditures) and funds coming from and managed by the 

private sector, such as employers and insurers. In developing countries, additional funds come from 

donors and are either administered by donors themselves or given to the government or a private 

institution to be administered as they do their own funds. Donor funding is not a sustainable source of 

funding in the long term, so health system plans should include increasing domestic spending on health to 

eventually bridge the gap currently covered by donor funds. In the short term, increasing the diversity of 

international donors is important to protect against unexpected discontinuation of funds (WHO, 2007). 

As mentioned above, the health financing system also will set financial incentives for providers. These 

incentives can have positive and negative effects on how health service workers provide care

(Christianson, et al., 2007). Ideally, health systems should reward providers who give better care and 

serve more patients with greater payment, but in practice these “pay for performance” schemes can result 

in the creation of unintended (“perverse”) incentives. Health system planners need to take into account 

what incentives they are creating when they decide on provider payment methods. These payment design 

questions have been covered extensively in the health economics literature for more developed countries,

as well as for India, China, Russia, and some Latin American countries, but for less-developed countries 

there has been less study of incentives and payment methods because of the general simplicity of their 

health finance systems (Schneider & Hanson, 2007). 

Ensuring all individuals have access to care requires that financing be available universally, whether 

through public or private funding. This funding needs to be available in sufficient amounts to cover basic 

care and prevent financial disaster due to a catastrophic illness (WHO, 2007). The definition of 

catastrophic health costs is still under debate, but a 2003 WHO-supported study found that three key 

preconditions for incurring catastrophic health costs were the availability of health services requiring 

payment, low capacity to pay, and the lack of prepayment or health insurance (Xu, et al., 2003). The 
51

importance of pooling financial risk through prepayment is reiterated in WHO’s 2007 Framework for 

Action report (WHO, 2007) and USAID’s HSAA manual (Islam, 2007). For poor and vulnerable 

populations with a low capacity to pay, prepayment may not provide full protection from financial 

hardship, and WHO encourages taking additional steps to cover their health care costs. 

Finally, the supply and demand of health care is dynamic, and thus health financing needs to be able to 

flow efficiently, effectively, and equitably to the facilities and individuals who need them. 

In order to measure whether these aspects of health financing are being met, indicators should cover 

whether adequate funds are available in ways that ensure access to health care, if the system protects from 

financial catastrophe, and if there are incentives for providers and users to be efficient with funds (WHO, 

2007). In general, the indicators in the sources below fall neatly into these categories. Financing 

indicators are generally quantifiable, making measurement less difficult. The indicators are also largely at 

the national level. Other building blocks such as service delivery and information systems will have more 

district- and facility-level indicators.

Recommended Indicator Sources

For finance, the WHO indicator Toolkit has just two recommended core indicators and one subindicator, 

reflecting the dearth of reliable financial information (WHO, 2008). It does, however, cover both national 

expenditures and personal expenditures, and there are also three optional indicators being considered to 

gather greater information on resource utilization and household impoverishment due to health 

expenditures. 

There is significant overlap in indicators between WHO’s Toolkit and USAID’s HSAA, but the latter has 

more indicators and information on the rationale behind the choice of each. It includes nearly all of the 

indicators given in the Toolkit. 

USAID’s graduation report on increased health promotion and access to quality health care 

(USAID Program Objective Team 3.2, 2001) includes many of the recommended indicators described 

below and an indicator for effective payment systems for primary and secondary health care providers. 

This type of indicator relates to provider incentives and is one of the few sources that touch on this topic. 

In order to operationalize such a measure, greater detail needs to be given on what counts as an effective 

payment system. 
52

The SWEF Research Network takes a slightly different approach to categorizing indicators but still uses

several of the same measures as the WHO Toolkit. It also includes measures relating to financial system 

sustainability, the pace of disbursement of donor and MOH funds, and the verticality of the system. The 

majority of these additional indicators seem to be of more relevance to funders rather than to the broader 

health systems community. This is in part because The Global Fund is a disease-specific funding 

organization and also because the stated purpose of the indicator list is to monitor how The Global Fund’s 

financial support is being used (SWEF Research Network, 2003). 

A CGD review of indicators for MCC resulted in only one finance-related input indicator on the final 

MCC list – the portion of total general government expenditure (GGE) that is used for general 

government health expenditure (GGHE) – but the earlier list included many other financial indicators that 

were disqualified due to data availability concerns (Becker, Pickett & Levine, 2006). These may become 

usable later, if data become available. It is notable that The World Bank Country Policy and 

Institutional Assessment (CPIA) Index also relies solely on the GGHE/GGE ration for measuring 

health financial systems (World Bank, 2007 and 2008). 

MDG indicators are largely involved in measuring health outputs and outcomes, but two would collect 

information on how much donor funding is going to health. The ECHIM Project includes one indicator 

on health financing (ECHIM, 2008). 

Data availability is possibly the most important criterion for judging health systems indicators. The main 

data sources for the indicators mentioned in this section are NHAs, household surveys such as DHS and 

LSMS, and SPA exit interviews. As pointed out earlier by the 2000 World Health Report, NHAs are 

subject to the same quality, validity, and reliability issues as other data sources from less-developed 

countries but are nonetheless an important part of estimating the efficiency of national health systems 

(WHO, 2002). WHO worked from the data available in full NHAs from 67 countries to choose its

financial indicators. In some cases, additional disaggregated data can be taken from The World Bank’s 

public expenditure reviews.

Most Recommended Financial Indicators

From these sources, we found a collection of indicators recommended by multiple organizations. 

• THE per capita is one of the financial indictors most often cited, as it is a commonly collected 

indicator that measures the availability of resources for health. While there is not a direct linear 
53

relationship between expenditures and health at all expenditure levels, a 2001 study found that at 

low expenditure levels, health system efficiency is positively related to health expenditure per 

capita (Evans, et al., 2001). Performance sharply increases with expenditure up to about $80 per 

capita per year. According to the WHO’s Commission on Macroeconomics and Health, basic 

essential health care services would have required expenditures in 2007 of at least US$34 

[US$30–40] per capita per year in low-income countries (WHO, 2001). Countries spending 

below this threshold are likely to have poor access, a low quality of health care, or both (Kruk & 

Freedman, 2008). 

• THE as a percent of gross domestic product (GDP) is cited as often as THE per capita. Some 

would argue that this measure is not as easily interpretable, and HMN has noted that a target 

specifying a minimum threshold for this indicator is not very useful on scientific grounds (HMN, 

2006). For the purposes of determining graduation countries in its E&E region, USAID also has 

set a minimum threshold for this indicator (USAID, 2001). 

• As mentioned earlier, another often-cited indicator is GGHE/GGE. This indicator illustrates the 

level of government financial support to the health sector relative to other financial commitments. 

This is one way to quantify the political support for health within a country. According to the 

Abuja Declaration, 15 percent or greater GGHE/GGE would reflect that health is a high priority 

in that country. However, as with THE per capita, when GGHE/GGE is already high, increasing 

it will not necessarily be a positive movement. This indicator will have to be gauged using 

country context. 

Two other notable indicators are the amount of total donor spending on health as a percent of THE

and the percent of government health spending that reaches the poorest income quintile:

 

• The first is an indicator that measures how sustainable health funding is in a country. If donor 

contributions to a country’s total health spending are above 10 percent, a significant burden 

would be placed on private spending if donor contributions are withdrawn (Islam, 2007). 

• The second of these indicators deals with equity in resource allocation and risk protection, which 

is very important, but there is no easily applicable threshold that would be appropriate in every 

situation, and data collection for this would be difficult (Walford, 2007). Whether or not this 

particular indicator is used, there is a need for some type of measure dealing with the equity of 

health care finance in order to describe a population’s exposure to catastrophic health 
54

expenditures. Several similar indicators deal with the financial risk households are exposed to by 

health expenditure, but there is a lack of consensus on how best to measure this. The WHO 

Toolkit suggests using a ratio of out-of-pocket expenditures to total health expenditures, but it 

also considers the percent of the population incurring catastrophic health expenditures, which is a 

measure also suggested by DfID. Other sources approach this question by measuring not the 

exposure to risk but the extent of protection against such risk. ECHIM includes a measure related 

to the percent of the population covered by prepaid care/health insurance, as does an academic 

literature review (Kruk & Freedman, 2008). Risk pooling through insurance and prepayment are 

acknowledged to be the best ways to increase financial protection against health expenses and 

equity of health financing (Gottret & Schieber, 2006). While insurance and prepayment schemes 

may not be widely used currently, including an indicator to measure insurance/prepayment could 

encourage the creation of such programs. Clearly, there is still considerable discussion on how to 

capture these data and what types of thresholds would be appropriate. The World Bank’s work on 

the idea of affordability, notably by Wagstaff and van Doorslaer, has contributed to a better 

description of the difficulties of measuring this indicator (Wagstaff, 2002; Wagstaff & van 

Doorslaer, 2004). 

• Indicators to monitor country actions for strengthening health financing. The WHO Toolkit

gives additional “output” indicators related to the above input indicators. These measure the 

government outputs that would produce measures of resources. Some of these indicators are 

collected under information systems in other sources, while others, such as government 

expenditures on salaries and number of people covered by health insurance, are mentioned as 

stand-alone indicators in other sources under finance. 

In an extensive literature review, THE per capita and donor spending on health as a percent of THE were 

found to be among the most often used in evaluations of health systems. The authors discuss the data 

availability for these indicators, as well as perhaps a stronger link to health outputs and outcomes (Kruk & 

Freedman, 2008). 

Issues and Concerns

To have cross-country comparability, it is important that the classification of health expenditures be 

standardized in NHAs. Particularly in the first round of NHAs, it was not possible to disaggregate total 

expenditures by function due to country-by-country variations in classifying funds (Nandakumar, et al., 

2004). Until this standardization occurs, any indicator based on this data will have to be used with caution 
55

across countries. The OECD has published a very complete guide to data collection for its own work on 

its System of Health Accounts, which may be a good basis for standardization (OECD, 2000).

Expenditure data in household and facility surveys also will be of concern due to the lack of monetization 

of markets in many countries, so dollar amounts may not always capture the total cost of a transaction

(Kahn & Hotchkiss, 2006). It is important, however, to collect whatever information is available on the 

personal cost of health care to measure household hardship due to health care. 
56

Leadership and Governance

Governance in health systems is about establishing effective rules in the institutional arenas for policies, 

programs, and activities related to public health functions in order to achieve health sector objectives

(USAID, 2008). These rules determine the roles and responsibilities for three sets of actors: state actors 

(who include policymakers and other government officials), health service providers, and service users. 

Determining the roles and responsibilities among these three sets of actors can reduce the hierarchical or 

authoritarian nature of public sector bureaucracy, often a core obstacle to the practice of good governance 

within health systems (Doherty & Gilson, 2006). According to WHO, this building block (also referred to 

as stewardship) is the most complex but critical building block of any health system (WHO, 2007). There 

is no clear blueprint for effective health leadership and governance. Although governance is ultimately 

the responsibility of the government, leadership and governance functions are carried out at lower levels, 

as well. Furthermore, as the concept of governance/stewardship in the health sector is relatively new,

there is little guidance for the collection or standardization of information on this aspect of the health 

system (Islam, 2007). 

The rationale for governance and accountability is that health systems are responsible for the 

improvement of population health in an equitable manner, and those affected by decisions and policies 

that affect well-being must have an understanding of and ultimate control over that system. Such control 

requires reasonable accountability from the government (Daniels, et al., 2000). Good governance has 

been shown to correlate with property rights, civil liberties, greater foreign investment, and increased aid 

effectiveness. These clearly have relevance to health care (Lewis, 2006). Good governance also includes 

evidence-informed policymaking and knowledge generation and dissemination. 

Governance has been the subject of multiple definitions and interpretations that derive from different 

purposes (USAID, 2008). Some definitions concentrate on technical government functions and how they 

are administered. The World Bank views governance as economic policymaking and implementation, 

service delivery, and accountable use of public resources and regulatory power. It divides governance into

six dimensions: voice and accountability, political stability, government effectiveness, regulatory quality, 

rule of law, and control of corruption. Other definitions address how government connects with other 

sectors and with citizens. For example, USAID considers health governance as the process of 

“competently directing health system resources, performance, and stakeholder participation toward the 

goal of saving lives and doing so in ways that are open, transparent, accountable, equitable, and 

responsive to the needs of the people” (Brinkerhoff, 2007). Different United Nations agencies have 
57

various descriptions of governance, while WHO prefers the concept of governance as stewardship, “the 

careful and responsible management of the well-being of the population” (WHO, 2000). 

Better governance influences health in various ways. For example, it can: 

• Improve the policy process in the health sector by greater use of data, evidence, and policy 

research, and increase the extent to which policy processes are informed by independent, valid 

research and analysis, and are open and accessible to citizens

• Improve accountability and transparency, and reduce corruption, informal payments, fraudulent 

billing, and employee theft by installing fee collection systems or facility-level accounting and 

reporting systems and by monitoring the sale of publicly funded drugs in the public and private 

sectors 

• Enhance participation at local levels by providing information to the public on citizens’ rights and 

duties in health sector activities and increase participation by civil society and community groups

To achieve better governance, WHO helps governments develop health sector policies and frameworks, 

design regulatory frameworks, support greater accountability, generate and interpret intelligence and 

research on policy options, build coalitions, and work with other partners (WHO, 2007).

According to USAID’s Health Systems 20/20 project, governance indicators are of two types:

1. Rules-based indicators, which measure whether the government has established key rules or 

policies in the health sector, such as a national essential medicines list or a national policy on 

malaria control. These indicators cannot be measured along a continuum and are discrete 

achievements. 

2. Outcome-based indicators, which measure whether rules and procedures are being effectively 

implemented or enforced based on the experience of relevant stakeholders. These include 

integration and responsiveness indicators that measure aspects of hospital or facility 

organizational structure and functioning, such as information use and technology, coordination 

among staff, and use of standardized protocols. 

Recommended Indicator Sources

The WHO draft Toolkit on Monitoring Health Systems Strengthening classifies indicators into the 

rules-based and outcome-based categories. There are 10 rules-based indicators that result in a Policy 
58

Index for assessing overall policies, regulations, and strategies in relation to the health sector. The Toolkit

also has six select markers of governance: health worker absenteeism, government funding reaching the 

district level, stockouts, informal payments, pharmaceutical regulation, and the existence of civil society 

organizations. The Toolkit also uses The World Bank’s CPIA Index to provide a composite measure of 

governance. Again, the indicators overlap with the medicines and HIS sectors.

The WHO Regional Office for the Eastern Mediterranean (EMRO) has developed an analytical 

framework that intends to measure the governance function of the health system at the national and 

subnational levels (WHO/EMRO, 2007; Siddiqi, 2008). It assesses nine principles of governance: 

strategic vision; participation and consensus orientation; rule of law; transparency; responsiveness; equity 

and inclusiveness; effectiveness and efficiency; accountability; and intelligence, information, and ethics. 

These nine principles are divided into 100 broad and specific questions that use data from both publicly 

available information and key informant interviews. The framework includes interviews with private 

sector NGOs but does not include interviews with civil society organizations (CSOs) or measures of 

clinical governance from interviews with health system users. While the results help identify strengths 

and weaknesses, they are qualitative and not comparable between countries. The framework has been 

applied in nine countries, including Afghanistan, Egypt, Pakistan, and Sudan. The questionnaire is 

available, but we have not been able to find results of the country assessments.

PAHO also has developed a tool to assess government stewardship in providing “essential public health 

functions” (EPHF) (PAHO, 2008). The EPHF are the fundamental set of actions that governments should 

perform in order to improve the health of populations. PAHO has developed a methodology that allows 

health ministries to evaluate in a comprehensive manner their public health systems, including health 

infrastructure, management, and financing, and evaluate their performance in the 11 functions. It has 

practical use as a framework for understanding the MOH service delivery capacity and role, and can be 

used as an M&E tool by governments. It was initially conducted in 41 Latin American and Caribbean

countries in 2001 and 2002, and since then many countries have continued monitoring the status of EPHF 

not only at the national level but at the subnational level, as well.

USAID’s HSAA instructs on how to use indicators to gauge the overall government context and the 

capacity of the government in five dimensions (Islam, 2007). For the overall governance environment, the 

HSAA uses the six World Bank indicators of government effectiveness, as well as indicators from five 

dimensions of governance: information/assessment capacity, policy formulation and planning, social 

participation and system responsiveness, accountability, and regulation. Several indicators overlap with 

indicators for information systems, medicines, and financing. There are criteria and guidance given on 
59

how to weigh the information and produce a balanced assessment, although there are few questions that 

can be answered with quantitative data. The HSAA has far more indicators on regulation and social 

responsiveness than WHO and other agencies; however, unlike other groups, it does not include 

indicators of disparity between quintiles or regions. 

USAID Missions have included health governance indicators as part of results frameworks under the 

Democracy and Governance Strategic Objective. USAID/Guinea, for example, in its results framework 

measured progress toward “improved effectiveness, accountability, and transparency of government 

institutions in the productive and social sectors,” using the indicator “percentage of target CSOs which 

have formed effective partnerships with local government delivering services.” For the objective 

“Strengthened Civil Society and Advocacy,” one indicator used by the Mission was “Percent of CSOs 

that effectively monitor government public services.” These are examples of indicators that could be 

modified to be potentially collected across countries. The USAID Democracy and Governance Office also 

developed the Handbook of Democracy and Governance Program Indicators (Hyman & Silver, 1998). 

While there are no health sector indicators, it offers insight into the challenges of applying performance 

measurement indicators for governance. 

DfID has adopted the use of “country governance analysis” to help guide aid allocations to countries 

(DfID, 2008). These analyses examine the capability (stability, regulation, trade/growth, effectiveness, 

security); accountability (transparency, free media, rule of law, elections); and responsiveness 

(rights/liberties, pro-poor, equality, regulation, corruption) of the government. Assessments are based on 

existing national and international data. They are relatively new and have not been completed for many 

countries.

The Working Group on Health System Metrics comprises HMN, WHO, The World Bank, The Global 

Fund, and the GAVI Alliance uses The World Bank’s CPIA Index, which rates countries against a set of 

16 criteria grouped in four clusters: (a) economic management; (b) structural policies; (c) policies for 

social inclusion and equity; and (d) public sector management and institutions. In addition to 

macroeconomic management, the assessment criteria include trade and financial policies; business 

regulation; social sector policies; the effectiveness of the public sector; and transparency, accountability, 

and corruption. 

As a response to improve the health indicators of the Millennium Challenge Account (MCA), the CGD 

Global Health Indicators Working Group reviewed indicators for their overall utility and their relationship 

to poverty and good governance. Good governance and development policies are key considerations of 
60

aid allocations from the MCA. The Group’s report, Measuring Commitment to Health, specifies

indicators that reflect a government’s commitment and allocation of resources to reach the underserved 

and its selection of interventions (Becker, Pickett & Levine 2006). It chose eight proxy indicators 

(complete diphtheria-pertussis-tetanus immunizations [DPT3], government public health expenditure, 

under-5 mortality rate, contraceptive prevalence rate, unmet need for family planning, stunting, skilled 

birth assistance, and access to water) to reflect a government’s health policies and commitment to health. 

These are actually output indicators but are included here to demonstrate the different concepts of 

measuring governance. 

The Global Fund’s March 2008 addendum to its 2006 Monitoring and Evaluation Toolkit provides 

guidance on indicators of governance, including six examples (such as number of CSOs partnering with 

public/private providers to deliver services and frequency of audits or reviews of performance). Rather 

than specify indicators, it strongly encourages countries to use existing in-country indicators such as those 

that are part of a program-based approach (including the Sector-Wide Approach) or other national

strategic frameworks that meet Global Fund M&E requirements. Governance indicators may be indicators 

of equity of access and coverage of essential services by regions quintile or vulnerable population groups. 

Appropriate targets are set using country-specific baselines and resources (The Global Fund, 2008). 

The SWEF Research Network working group primarily focuses on governance of the policy process, 

fairness in resource allocation, and donor harmonization. It is one of the only sources to include indicators 

on the number of public/private partnerships for services and the number of new private providers, which 

measure private sector involvement in health care, the government’s understanding of the role of the 

private sector, and ease of entry into the market. These are goals of The Global Fund, and private sector 

involvement may not be a suitable indicator for all countries.

Other sources of governance indicators include “corruption evaluations” to help policymakers measure 

corruption in the health sector and identify possible ways to intervene to increase accountability, 

transparency, citizen voice, detection, and enforcement; control discretion; and reduce monopoly power 

(Vian, 2008). Sources of data to identify corruption include corruption perception surveys, expenditure 

surveys, qualitative data collection, and reviews of control systems. Other indicators that have been 

identified for capturing poor governance in health service delivery mismanagement include leakage of 

drugs and supplies, patients providing in-kind supplies and drugs, staff absenteeism, and informal 

payments (Lewis, 2006). Benchmarks of Fairness, a generic matrix for assessing health systems reform 

in developing countries (Daniels, et al., 2000), includes benchmarks for democracy, accountability, and 

empowerment. The democracy benchmark has eight field-tested and measureable criteria (indicators), 
61

such as the existence of procedures for evaluating services, resource allocation with transparency, and fair 

grievance. Because of the adaptive framework, it is a promising tool for cross-country comparability 

(Alliance for Health Policy and Systems Research, 2008). The USAID Strategic Framework for 

Foreign Assistance Governance Indicators quantify how USAID Missions are improving governance 

in USAID-assisted areas and at the national level (DOS, 2006). A corruption perceptions index (CPI), 

public sector opinion surveys, and corruption barometers are available from Transparency International 

(TI); however, they do not cover any health-specific indicators. Among the 11 institutions that provide 

data for the CPI, none are health institutions or organizations. TI and a partner organization, CIET 

International, have implemented public opinion surveys, institutional surveys, case studies, and social 

audits that measure governance of local health institutions or health boards. However, these case studies 

are typically qualitative assessments for specific purposes and are not comparable across countries. 

Another indicator used by Political Risk Services in its International Country Risk Guide is a measure of 

institutional/bureaucratic quality (PRS Group, 2009). This indicator captures the result of good 

governance, although not necessarily in the health sector. It has been used in research to analyze the 

relationship between governance and health and education. Its role, however, has not been clearly 

captured in such analyses.

Most Recommended Leadership and Governance Indicators

The indicators most frequently cited are measures of accountability and social participation. Government 

expenditure on health is also frequently used, although this overlaps with financing measures. Sources 

varied widely in the use of governance indicators, although several include the CPIA in their overall 

assessment. Often governance indicators must be established within local contexts and aligned with local 

goals. Some general descriptions include:

• WHO Policy Index. WHO proposes universal use of 10 yes/no indicators that assess if a country 

has adopted WHO-recommended policies, regulations, and strategies in relation to 10 areas of the 

health sector (WHO, 2008). The 10 policy items are rated as zero (adequate policy does not exist 

or cannot be assessed) or one (adequate policy is available). The proposed index does not aim to 

assess enforcement, as this may be captured by other indicators within the health system 

components. The indicators include the existence of a national health strategy; an essential 

medicines list and policy on drug procurement; a national policy on child, maternal, and 

reproductive health; national policies on malaria and TB; and participation in the United Nations 

General Assembly Special Session (UNGASS) composite policy index questionnaire.
62

• Existence of an up-to-date national health strategy linked to national needs and priorities. 

The strategy should follow internationally accepted policy standards and guidelines (e.g., the 

international code of marketing of breast milk substitutes, international standards of care). These 

measures indicate government stewardship and capacity to develop, implement, and monitor 

legislation and guidance on public health and health system issues. Comprehensive health policy 

and planning processes integrate health system information, public input, and evidence-based 

recommendations for action. The target is for each health goal to conform to standard 

international policies. Comparability is an issue, as the regulation and enforcement of policies, 

and the processes by which they are set, vary by country. Data are found within MOH documents 

or reports. This indicator is also part of the WHO Policy Index.

• Health worker absenteeism in public health facilities. This indicator is also an indicator of 

regulation (see below). 

• Proportion of government funds that reaches district-level facilities. The rationale for this is 

that governance in health financing can be assessed by monitoring not only overall levels of 

health spending but also equity in allocating budgets and efficiency in ensuring that spending 

reaches health facilities and the poor. Data can be gathered from Public Expenditure Tracking 

Surveys, NHAs, and/or MOH records.

• Health service delivery: Stockout rates of essential drugs in health facilities. This indicator is 

also part of the medicines component of health systems (see Medical Products, Vaccines, and 

Technologies section). 

• Pharmaceutical regulation: Proportion of pharmaceutical sales that consist of counterfeit 

drugs. This indicator is also part of the medicines component of health systems (see Medical 

Products, Vaccines, and Technologies section) and one of several often-used indicators of 

regulation (see below). 

• Accountability indicators. These measure the government’s ability to answer questions, meet 

reasonable expectations of the system, and address negligent or corrupt actions. There are many 

measures of this, and researchers approach accountability using different paradigms and measure 

different outcomes (Alliance for Health Policy and Systems Research, 2008). They include 
63

indicators of civil society participation in decisions, although the range and impact of civil 

involvement are difficult to compare. Some agencies (USAID among them) include the existence 

of a free and scientific press, watchdog organizations, and an independent judiciary as a 

requirement for accountability. Other measures of accountability include the existence of fair 

grievance procedures, such as dispute resolution, that are available to the public. Data likely come 

from interviews of stakeholders.

• Disparity in coverage between lowest- and highest-income groups/regions/rural/urban areas.

Equitable coverage demonstrates rational and transparent resource allocation. However, 

comparability is an issue, as the size of the different groups or regions affects result and may limit 

the usefulness of country comparisons of data. Data can be found in household surveys. 

• CPIA Index. The World Bank’s CPIA Index is based on a set of criteria captured in 16 

subcomponents grouped in four clusters: (a) economic management; (b) structural policies; (c)

policies for social inclusion and equity; and (d) public sector management and institutions (World 

Bank, 2007 and 2008). 

• Proportion of informal payments within the public health care system. Informal payments 

can reduce the utilization of services by patients who cannot pay and reduce the quality of care 

from loss of revenue at facilities. The frequency of informal payments is often used in studies 

and interventions. Data can be gathered from household surveys, corruption perception surveys, 

and key informant interviews, although the latter are subjective, subject to measurement error, 

and can only provide a rough indication of trends. 

• Regulation. The most frequently cited indicator is pharmaceutical regulation, which itself has 

many measures, including “the existence of adequate regulation to ensure the safety, efficacy, and 

quality of medicines” and “the proportion of pharmaceutical sales that consist of counterfeit.” 

Regulation of medicines and procurement overlap with the medicines sector indicators. Indicators 

also can include absenteeism, accreditation, and licensure of health professionals, although these 

may overlap with the workforce sector. Data on regulatory and enforcement policies may be 

available at the MOH; other sources include key informant interviews and facility surveys.

• Social responsiveness. One goal of many donor-funded health reforms is to increase the extent to 

which citizens of a country are able to participate in health policy decisions. These indicators 

measure the government’s ability to facilitate collaboration among government, civil society, and 
64

other stakeholders to participate in the planning, budgeting, and monitoring of activities in the 

health sector. They include voice and accountability and whether CSOs empower individuals to 

express their views to government bodies. Indicators frequently relate to the number of civil 

society members or civil groups trained. Indicators for this objective vary depending on the 

country context and may not be relevant in all situations. Data likely come from interviews of 

stakeholders.

One measure that overlaps with health service delivery is evaluation of staff performance. Evaluations 

can improve the accountability of the government in providing appropriate services and monitor the 

responsiveness of the public health sector. They also can measure staff absenteeism or adherence to 

regulations and protocols. Indicators on patient expenditure are not included here and would overlap with 

financing. Both the DHS and WHS can provide data on inequity, and WHS can provide data on 

expenditure. Neither the WHS or DHS questions distinguish between formal and informal payments. 

Issues and Concerns

Data availability is a critical concern when selecting indicators. Several indicators have data available for 

most countries in the public domain, using sources such as household surveys, World Bank indexes, and 

Transparency International’s CPI and Global Corruption Barometer survey. Currently, there is no healthspecific policy and institutional assessment tool, but WHO is proposing one based on measures in The 

World Bank’s CPIA (WHO, 2008). The CPIA itself is not yet suitable to be deconstructed into a healthspecific index (Bos, 2006). There are five health- and education-related indicators in the CPIA under the 

“building human resources” dimension. These five indicators are similar to the WHO Policy Index and 

account for 1.67 percent of the total score, with an overall correlation with the CPIA of .76. Recent 

discussions and recommendations for revising the CPIA included recommendations for evaluating 

national poverty reduction policies and social policies but not for increasing the significance of health 

sector in the score. There are other issues in creating a health-specific CPIA, including the inclusion of 

the overall country policy framework in any assessment of health governance. 

Another general concern is the simplistic nature of some indicators. For instance, the WHO indicator on 

the existence of a country health strategy is a yes/no indicator that does not capture the breadth of how 

governance may affect health outcomes. Governance indicators also are not always sector specific. As a 

result, it is not always possible to disaggregate them to be health sector specific. They often capture a 

composite picture of a country’s governance but not down to the sector level. A good example is the set 

of governance-related indicators that are part of the MCC category of Ruling Justly.
65

The comparability of measures of governance is also a challenge, as the definitions and contexts of 

governance vary across countries and depend on the degree of development. Moreover, in any external 

assessment, data for many indicators must still be gathered in collaboration with the MOH. Reliable 

reporting of data is also a concern. There have been no systematic reviews on governance or 

accountability in health systems, and research analytic frameworks and measurements vary widely

(Alliance for Health Policy and Systems Research, 2008). Research responses are often qualitative, which 

affects their quality. For example, the assessment of data reliability and the quality, timeliness, and extent 

of data use are graded according to qualitative answers on seven questions. The methodology and issues 

concerning corruption perception surveys also are currently being debated. The perceptions of the public 

regarding services will vary according to local circumstances, and perceptions can be different from 

actual behaviors. The level of decentralization and the development of information systems also affect 

many aspects of financing and policy measures. 
66

References 

Alliance for Health Policy and Systems Research. (2008). Neglected Health Systems Research: 

Governance and Accountability. Geneva: World Health Organization.

Alliance for Health Policy and Systems Research. WHO Web site. http://www.who.int/alliance-hpsr/en/ 

Becker, L., Pickett, J., & Levine, R. (2006). Measuring Commitment to Health: The Global Health 

Indicators Working Group Report. Washington, DC: Center for Global Development. 

Bennett, S. & Fairbank, A. (2003). The System-Wide Effects of The Global Fund to Fight AIDS, 

Tuberculosis and Malaria: A Conceptual Framework. Technical Report No. 031. Bethesda, MD: The 

Partners for Health Reform Plus Project, Abt Associates Inc.

Bertrand, J. & Escudero, G. (2002). Compendium of Indicators for Evaluating Reproductive Health 

Programs. Vol. I. USAID/MEASURE Evaluation Manual Series No. 6. Chapel Hill, NC: Carolina 

Population Center. 

Bos, E. (2006). Health Systems Metrics: The Country Policy and Institutional Assessment Indicator. 

Slide presentation at Health Systems Metrics Technical Meeting, Glion, Switzerland, 28–29 September 

2006: The World Bank. http://www.who.int/healthmetrics/documents/boscpia.ppt#257 

Brinkerhoff, D. (2007). Health Governance: Why, What, and How. Presentation at Global Health MiniUniversity, Washington, DC, October 2007. Bethesda, MD: Abt Associates, Health Systems 20/20. 

Cameron, A., Ewen, M., Ross-Degnan, D., Ball, D., & Laing, R. (2008). Medicine Prices, Availability, 

and Affordability in 36 Developing and Middle-income Countries: A Secondary Analysis. The Lancet,

373(9659):240–9. 

Capacity Project Web site. http://www.capacityproject.org/

Carandang, E.& Pierre-Jacques, M. (2005). Using Indicators to Measure Country Pharmaceutical 

Situations. Drug Action Program (DAP) and WHO Action Program on Essential Drugs and WHO 

Boston Collaborating Center on Pharmaceutical Policy. Slide presentation. 

http://archives.who.int/tbs/tbs2005/ indicators.ppt

Chen, L., Evans, T., Anand, S., Boufford, J., Brown, H., & Chowdhury, M. (2004). Human Resources for 

Health: Overcoming the Crisis. The Lancet, 364:1984–1990. 

Child Survival and Technical Support Plus Web site. http://www.childsurvival.com/rhfa_1.cfm

Christianson, J., Leatherman, S., & Sutherland, K. (2007). Financial Incentives, Healthcare Providers 

and Quality Improvements. London: The Health Foundation. 

Daniels, N., Bryant, J., Castano, R., Dantes, O., Khan, K., & Pannarunothai, S. (2000). Benchmarks of 

Fairness for Health Care Reform: A Policy Tool for Developing Countries. Bulletin of the World Health 

Organization, 78 (6).
67

DELIVER. (2006). Contraceptive Security Index 2006: A Tool for Priority Setting and Planning. Task 

Order 1 of the USAID | Health Policy Initiative. Arlington, VA: DELIVER, for the United States Agency 

for International Development. 

DELIVER. (2009). Logistics System Assessment Tool. (LSAT). Task Order. Arlington, VA: DELIVER, 

for the United States Agency for International Development.

Department of State. (2006). U.S. Foreign Assistance Performance Indicators for Use in Developing FY 

2007 Operational Plans. Washington, DC: Department of State. http://www.state.gov/documents/

organization/78557.pdf

de Vries, W. (1998). How Are We Doing? Performance Indicators for National Statistical Systems. 

Netherlands Official Statistics,13: 5–13. 

Department for International Development. (2008). Country Governance Analysis: A How-to Note. A 

DfID Practice Paper. Politics and the State Team. London: DfID. 

DfID Health Resource Center Web site. http://www.dfidhealthrc.org/

Doherty, J. & Gilson, L. (2006). Proposed Areas of Investigation for the KN: An Initial Scoping of the 

Literature. Geneva: WHO Commission on Social Determinants of Health. 

ECHIM. (2008). ECHI shortlist indicators. 

http://www.healthindicators.org/healthindicators/root/o13.html 

ECHIM. (2008a). ECHIM Comprehensive Lists. http://www.healthindicators.org/healthindicators/

object_document/o5415n28314.html 

ECHIM. (2008b). Links to Indicator Information. http://www.healthindicators.org/ 

healthindicators/object_document/o4963n28314.html 

Evans, D., Tandon, A., Murray, C., & Lauer, J. (2001). Comparative Efficiency of National Health 

Systems: Cross National Econometric Analysis. British Medical Journal, 323:307–310. 

Fresle, D. & Wolfheim, C. (1997). Public Education in Rational Drug Use: A Global Survey. Geneva: 

World Health Organization. 

Future Health Systems Consortium Web site. http://www.futurehealthsystems.org/ 

GAVI Alliance. (2007). GAVI Alliance Health System Strengthening (HSS) Applications, Geneva: 

GAVI Alliance. 

Global HIV/AIDS Initiatives Network Web site http://www.ghinet.org/index.htm

Gottret, P. & Schieber, G. (2006). Health Financing Revisited: A Practitioner’s Guide. Washington, DC: 

The World Bank. 

Hall, T. F. (2001). Human Resources for Health: Models for Projecting Workforce Supply and 

Requirements. Geneva: World Health Organization. 
68

Hanney, S., Gonzalez-Block, M., Buxton, M., & Kogan, M. (2003). The Utilization of Health Research 

in Policy-making: Concepts, Examples and Methods of Assessment. Health Research Policy and 

Systems, 1:2. 

Hare, L., Hart, C., Scribner, S., & Sheperd, C. (2004). SPARCHS: Strategic Pathway to Reproductive 

Health Commodity Security. A Tool for Assessment, Planning, and Implementation. Baltimore, MD: 

INFO Project/Center for Communication Programs, Johns Hopkins Bloomberg School of Public Health. 

Harvard School of Public Health Web site, International Health Systems Program. 

http://www.hsph.harvard.edu/ihsg/research.html 

Health Metrics Network. (2006). Health System Metrics: Report of a Technical Meeting. Glion, 

Switzerland, 28-29 September 2006: World Health Organization. 

Health Metrics Network. (2008). Framework and Standards for Country Health Information Systems. 

Geneva: World Health Organization. 

Health Metrics Network (2008a). Assessing the National Health Information System: Assessment 

Tool Version 4.0. Geneva: World Health Organization.

http://www.who.int/healthmetrics/tools/Version_4.00_Assessment_Tool.pdf

Health Systems 20/20 Web site. http://www.healthsystems2020.org/ 

Hutton, G. (2000). Indicators for Monitoring Health Sector Reform and the Sector-Wide Approach. 

Presentation at Sector-Wide Approach in Health Conference: Moving from Policy to Practice. 27–28 

November 2000, Amsterdam: Royal Tropical Institute.

Hyman, G. & Silver, R. (1998). Handbook of Democracy and Governance Program Indicators. 

Washington, DC: USAID Office of Democracy and Governance. 

Islam, M., (ed.). (2007). Health Systems Assessment Approach: A How-To Manual. Submitted to the 

United States Agency for International Development in collaboration with Health Systems 20/20, Partners 

for Health Reform Plus, Quality Assurance Project, and Rational Pharmaceutical Management Plus.

Arlington, VA: Management Sciences for Health. 

Kahn, M. & Hotchkiss, D. (2006). How Effective Are Health Systems Strengthening Programs in 

Reaching the Poor? A Rapid Assessments Approach. Bethesda, MD: Abt Associates, PHRplus.

Kruk, M. & Freedman, L. (2008). Assessing Health System Performance in Developing Countries: A 

Review of the Literature. Health Policy, 85(3):263–276. 

Laing, R.O., Hogerzeil, H.V., & Ross-Degnan, D. (2001). Ten Recommendations to Improve Use of 

Medicines in Developing Countries. Health Policy and Planning. 16(1): 13-20. 

Lewis, M. (2006). Governance and Corruption in Public Health Care Systems. Washington, DC: Center 

for Global Development. 

Management Sciences for Health/University Research Corporation (MSH/URC). (1995). Rapid 

Pharmaceutical Management Assessment: An Indicator-based Approach. Arlington, VA, and Bethesda, 

MD: Management Sciences for Health and University Research Corporation.
69

Management Sciences for Health/World Health Organization. (2006). Tools for Planning and Developing 

Human Resources for HIV/AIDS and Other Health Services. Cambridge, MA: Management Sciences for 

Health.

Marchal, B., Cavalli, A., & Kegels, G. (2009). Global Health Actors Claim to Support Health System 

Strengthening–Is This Reality or Rhetoric? PLoS Med, 6(4): e1000059. 

doi:10.1371/journal.pmed.1000059. 

Martinez, J. (2008). Sector Wide Approaches at Critical Times: The Case of Bangladesh. Technical 

Approach Paper. London: The HLSP Institute. http://www.hlspinstitute.org/files/project/196066/ 

BangladeshSWAp_Feb08.pdf 

McGlynn, E.A., et al. (eds.). (2000). Quality of Care Assessment Tools (5 vols.). Santa Monica: RAND 

Corporation. 

MEASURE DHS. Demographic and Health Surveys Web site. http://www.measuredhs.com 

MEASURE Evaluation. (2006). Profiles of Health Facility Assessment Methods: Report of the 

International Health Facility Assessment Network (IHFAN). Arlington, VA: USAID/MEASURE 

Evaluation. 

MEASURE Evaluation. (2007). Guidance for Selecting and Using Core Indicators for Cross-Country 

Comparisons of Health Facility Readiness to Provide Services. Health Facility Assessment Technical 

Working Group. September 2007. WP-07-97. Chapel Hill, NC: USAID/MEASURE Evaluation. 

Millennium Challenge Corporation. (2008). Guide to the MCC Indicators and the Selection Process, 

Fiscal Year 2008. Washington, DC: Millennium Challenge Corporation. 

Mosse, R.& Sontheimer, L.E. (1996). Performance Monitoring Indicators Handbook. World Bank 

technical paper; no. 334. The World Bank.

Mossialos, E., Allin, S., & Figueras, J. (2007). Health Systems in Transition: Template for Analysis. 

Copenhagen: WHO Regional Office for Europe on behalf of the European Observatory on Health 

Systems and Policies. 

Nandakumar, A., Bhawalkar, M., Tien, M., Ramos, R., & De, S. (2004). Synthesis of Findings from NHA 

Studies in Twenty-Six Countries. Bethesda, MD: Abt Associates, PHRplus.

Organization for Economic Cooperation and Development. (2000). A System of Health Accounts. 

Version 1.0. http://www.oecd.org/dataoecd/41/4/1841456.pdf

Pan American Health Organization/World Health Organization, Health Systems and Services Area. 

(2008). The Essential Public Health Functions as a Strategy for Improving Overall Health Systems 

Performance: Trends and Challenges Since the Public Health in the Americas Initiative, 2000–2007. 

Washington, DC: Pan American Health Organization. 

Perrin, E., Durch, J., & Skillman, S., (eds.). (1999). Health Performance Measurement in the Public 

Sector: Principles and Policies for Implementing an Information Network. Report of the Panel on 

Performance Measures and Data for Public Health Performance Partnership Grants, Committee on 

National Statistics, Commission on Behavioral and Social Sciences and Education, National Research 

Council. Washington, DC: National Academy Press.
70

Population Council Web site. http://www.populationcouncil.org 

PRS group. (2009). International Country Risk Guide. 

Rational Pharmaceutical Management Plus Web site. http://www.msh.org/projects/rpmplus/

Rockefeller Foundation. (2008). Strengthening Health Systems Capacity and Leadership. Background 

paper for meeting on Strengthening Health Systems Capacity and Leadership, Bellagio, October 2008. 

http://columbia-icap.org/bellagio2/

Ross, J.A. & Stover, J. (2001). The Family Planning Program Effort Index: 1999 Cycle. International 

Family Planning Perspectives (New York), 27 (3):119-129. 

Schneider, P. & Hanson, K. (2007). The Impact of Micro Health Insurance on Rwandan Health Centre 

Costs. Health Policy and Planning Advance, 22: 40–48.

Siddiqi, S. (2008). Governance of the Health System: Moving Ahead. Slide presentation. WHO/EMRO 

Regional Capacity Building Workshop on Health Systems Development, June 2008. 

http://gis.emro.who.int/HealthSystemObservatory/Workshops/2008/Presentations/Day%201/ 

3.%20Health%20system%20governance%20June%202008.ppt 

Sjönell, G. (1984). Relationship Between Use of Increased Primary Health Care and Other Out-patient 

Care in a Swedish Urban Area I. Utilisation of Public Primary Health Care. Scandinavian Journal of 

Primary Health Care, 2 (1): 33–39.

Stansfield, S., Walsh, J., Prata, N., & Evans, T. (2006). Information to Improve Decision Making

for Health. In D. Jamison, D., et al. (eds.), Disease Control Priorities in Developing Countries (pp. 1017–

1030). New York, NY, and Washington, DC: Oxford University Press and The World Bank.

System-Wide Effects (SWEF) Research Network. (2003). Common Research Protocol: Monitoring and 

Evaluating the Health System-Wide Effects of The Global Fund to Fight AIDS, Tuberculosis and Malaria. 

Bethesda, MD: Abt Associates, PHRplus. 

The ACQUIRE Project. Engenderhealth Web site. http://www.engenderhealth.org/our-work/majorprojects/acquire.php 

The Global Fund to Fight AIDS, Tuberculosis and Malaria. (2006). Monitoring and Evaluation Toolkit: 

HIV/AIDS, Tuberculosis and Malaria. 2nd edition. Geneva: The Global Fund to Fight AIDS, 

Tuberculosis and Malaria.

The Global Fund to Fight AIDS, Tuberculosis and Malaria. (2008). Monitoring and Evaluation Toolkit: 

HIV/AIDS, Tuberculosis and Malaria, Addendum March 2008. Geneva: The Global Fund to Fight AIDS, 

Tuberculosis and Malaria.

Transparency International Web site. http://www.transparency.org/policy_research/surveys_indices

United Nations. (2003). Indicators for Monitoring the Millennium Development Goals – Definitions, 

Rationale, Concepts and Sources. New York, NY: United Nations. 

United Nations. (2008). Delivering on the Global Partnership for Achieving the Millennium Development 

Goals: Millennium Development Goal 8: MDG Gap Task Force Report 2008. New York, NY: 

United Nations. 
71

University of Washington Institute of Health Metrics and Evaluation Web site. 

http://www.healthmetricsandevaluation.org/what/research.html 

USAID. (2001). Graduation Report on Increased Health Promotion and Access to Quality Health Care 

in the Europe and Eurasia (E&E) Region. Internal document, developed by E&E Bureau’s Program 

Objective Team for Strategic Objective 3.2.

USAID. (2008). Health Governance: Concepts, Experience, and Programming Options. Policy brief. 

Bethesda, MD: Abt Associates, Health Systems 20/20. 

USAID Program Objective Team 3.2. (2001). Strategic Objective 3.2 (2001., Increased Health 

Promotion and Access to Quality Health Care: Graduation Report. Washington, DC: USAID.

U.S. President’s Emergency Plan for AIDS Relief (PEPFAR). (2009). Next Generation Indicators 

Reference Guide. Version 1.0. Washington, DC: PEPFAR. 

Vian, T. (2008). Review of Corruption in the Health Sector: Theory, Methods and Interventions. Health 

Policy and Planning, 23(2):83–94; doi:10.1093/heapol/czm048. 

Wagner, A., Ross-Degnan, D., Reiss, S., & Johnson, A. (2008). Results of Feasibility Study Using World 

Health Survey Data to Measure Medicines Need, Use and Out-of-Pocket Expenditures in Eight Low and 

Lower-Middle Income Countries. Geneva: World Health Organization. 

Wagstaff, A. (2002). Reflections On and Alternatives to WHO’s Fairness of Financial Contribution Index. 

Health Econ, 11(2):103–15.

Wagstaff, A. & van Doorslaer, E. (2004). Overall Versus Socioeconomic Health Inequality: A 

Measurement Framework and Two Empirical Illustrations. Health Econ, 13(3):297–301. 

Walford, V. (2007). Indicators of Health System Performance: Report of a Review for DFID. London: 

DfID Health Resource Centre. 

WHO. (1999). Indicators for Monitoring National Drug Policies: A Practical Manual, 2nd edition. 

Geneva: World Health Organization.

WHO. (2000). The World Health Report 2000: Health Systems: Improving Performance. Geneva: World 

Health Organization. 

WHO. (2001). Macroeconomics and Health: Investing in Health for Economic Development: Report of 

the Commission on Macroeconomics and Health. Geneva: World Health Organization. 

WHO. (2001a). Report of the Scientific Peer Review Group on Health Systems Performance Assessment. 

Geneva: World Health Organization. 

WHO. (2002). Report of the Scientific Peer Review Group on Health Systems Performance Assessment. 

Geneva: World Health Organization. 

WHO. (2004). Developing Health Management Information Systems: A Practical Guide for Developing 

Countries. Geneva: World Health Organization. 
72

WHO. (2006). Using Indicators to Measure Country Pharmaceutical Situations: Fact Book on WHO 

Level I and Level II Monitoring Indicators. Geneva: World Health Organization. 

WHO. (2006a). The World Health Report 2006: Working Together for Health. Geneva: World Health 

Organization. 

WHO. (2007). Everybody’s Business: Strengthening Health Systems to Improve Health Outcomes: 

WHO’s Framework for Action. Geneva: World Health Organization. 

WHO. (2007a). Division of Health Systems. Health Workforce Supply and Requirements Projection 

Models. Presented at the HRH Workforce Planning Model Workshop, Washington, DC. December 2007. 

WHO. (2008). Toolkit on Monitoring Health Systems Strengthening (draft). Geneva: World Health 

Organization. http://www.who.int/healthinfo/statistics/toolkit_hss/en/index.html 

WHO. (2008a). WHO Medicines Strategy (2008–2013) (draft). Geneva: World Health Organization. 

http://www.who.int/medicines/publications/Medicines_Strategy_draft08-13.pdf 

WHO/AusAID. (2006). Measuring Transparency in Medicines Registration, Selection and Procurement: 

Four Country Assessment Studies. Geneva: World Health Organization. 

WHO/EMRO. (2007). Health System Governance for Improving Health System Performance. Report of a 

WHO Global Consultation. World Health Organization Regional Office for the Eastern Mediterranean. 

http://gis.emro.who.int/HealthSystemObservatory/PDF/Publications/Reports%20of%20Workshops%20an

d%20Meetings/PHP043healthsystemgovernancefinal.pdf

WHO/PAHO Collaborating Center in Pharmaceutical Policy Web site. http://www.whoccpp.org/research/

survey_analysis.asp

WHO/WPRO. (2004). Developing Health Management Information Systems: A Practical Guide for 

Developing Countries. Geneva: World Health Organization. 

World Bank. (2007). Country Political and Institutional Assessment: 2007 Assessment Questionnaire. 

Washington, DC: The World Bank. 

World Bank. (2008). Adaptation and Refinement of the World Bank’s “Country Policy and Institutional 

Assessment” (CPIA). Washington, DC: The World Bank. 

World Bank. Living Standards Measurement Surveys Web site. 

http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/EXTLSMS/0,,contentMD

K:21610833~pagePK:64168427~piPK:64168435~theSitePK:3358997,00.html

The World Bank. Statistical Capacity Indicator, Country Statistical Information Database Web site. 

http://web.worldbank.org/WBSITE/EXTERNAL/DATASTATISTICS/0,,contentMDK:20541648~menuP

K:64133152~pagePK:64133150~piPK:64133175~theSitePK:239419,00.html

Xu, K., Evans, D., Kawabata, K., Zeramdini, R., Klavus, J., & Murray, C. (2003). Household 

Catastrophic Health Expenditure: A Multi-country Analysis. The Lancet, 362: 111–117.
73

Appendix 1: Indicators by Organization

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Service Delivery

Number and Distribution 

of in-patient beds per 

10,000

No threshold set- A greater number of 

hospital beds suggests that there is a 

greater availability of inpatient health 

services. Conversely, some countries 

(e.g., members of the Organisation 

for Economic Cooperation and 

Development) have witnessed a 

downward trend in hospital beds per 

10,000 population as outpatient 

surgery increases. 

X Total # of 

Hospital 

beds per 

10,000 

pop.

Hospital 

bed 

density

Total # of 

Hospital 

beds per 

100,000 

pop.

Number and distribution 

of health facilities per 

10,000 pop.

Although few benchmarks are 

available, a comparison with key 

neighboring countries may be 

instructive. 

X # of 

primary 

care fac. 

per 

10,000 

pop

Hospitals 

per 

1,000 

pop.

Proportion of Health 

facilities that meet basic 

service capacity 

standards - general and 

specific

No threshold set - increases are 

considered positive movement.

X
74

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Number of out-patient 

dept. (OPD) visits per 

10,000 pop per year

This is related to the number of 

inpatient visits- in some developed 

countries outpatient visits increase 

because in patient visits are 

decreasing, reflecting the greater use 

of ambulatory procedures instead of 

procedures requiring admission. In 

some instances this reflects overuse, 

and a decrease in OPD visits would 

be positive movement. This would 

also be true in urban areas of 

developing nations when access to 

inpatient care is restricted, resulting in

overuse of OPD (Holdsworth, Garner 

and Harpham, 2006). However, In 

other instances, such as in countries 

where use is strikingly low (Uganda, 

Burundi, etc, possibly in the ballpark 

of 500/10,000 or less), an increase 

implies increases in primary care use.

Therefore, an increase in this number 

suggests positive movement (Sjonell, 

1984).

XXXX

Service Quality/Quality 

of Care indicators

Category of indicators; more 

definition of indicators is needed 

beforea threshold is set. 

*****

Percentage of people 

living within X kms of a 

health facility

No threshold set -No distance set. 

The values for this indicator will vary 

by type of geography. 

X ... within 

10km

Percentage of primary 

care facilities that are 

adequately equipped

No threshold set.X

Number and distribution of 

health facilities with basic 

service capacity per 

10,000

No threshold set.X

Basic and comprehensive 

obstetric care facilities per 

500,000 

Minimum of four basic emergency 

obstetric care facilities and one 

comprehensive emergency obstetric 

care facility per 500 000 population 

(WHO, 2005).

X
75

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Number of primary care or 

outpatient visits per 

person to health facilities 

per year

In most developing countries, a 

higher utilization rate of public sector 

health services (compared to the 

private sector) may be desirable, 

because it suggests access to 

facilities and a degree of trust in the 

public system. However, to interpret 

this indicator, a regional average is 

needed (Islam et al., 2007). 

X

Availability of updated 

clinical standards for MOH 

priority areas, high burden 

diseases areas,

and/or areas responsible 

for high morbidity and 

mortality

Yes/No - definition of "updated" not 

given. 

X

Number of new services 

offered by type of facilitiy

No threshold set. X

Average hours that priority 

services are offered per 

week

No threshold set. X

The ratio of health care 

professionals to the 

population

See workforce section. X

Financial access (select 

an indicator based on 

available data)

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

*

User fee exemptions and 

waivers

No threshold set. X

Private sector service 

delivery

High private sector use can indicate 

an unmet demand for health services 

in the public sector, perhaps due to 

poor quality issues or access 

constraints (Islam et al., 2007).

X

Existence of corporate 

social responsibility (CSR) 

programs that offer health 

services among

the country's largest 

employers

Yes/No X
76

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Quality Assurance system 

that monitors the service 

quality of health care 

facilities

No threshold set. X

Public /Private indicators This is a category of indicators.

Indicators need to be defined before

a threshold is set.

X

Pregnant women who 

received 1+ antenatal care 

visits (%)

X Pregnant 

women 

receiving 

4 ANC 

visits

Life expectancy at birth, 

total (years)

X

Mortality rate, infant (per 

1,000 live births) 

X

Maternal mortality ratio 

(per 100,000 live births)

X

Prevalence of HIV, total 

(% of population aged 15–

49)

X

Percentage of births 

attended by skilled health 

personnel per year

X

DPT3 immunization 

coverage: one-year-olds 

immunized with three 

doses of diphtheria, 

tetanus toxoid, and 

pertussis (DPT3) (%)

X

Contraceptive prevalence 

(% of women aged 15–49)

X
77

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Health Workforce

Number of health 

workers per 10,000 pop

Use the staffing requirement model 

developed for the WHO to determine 

thresholds (Hall, 2001), or the WPRO 

Workforce Projection Tool. The WHO 

found that a minimum of 2.3 

physicians, nurses, and midwives per 

1000 population were needed to meet 

80% coverage of skilled birth 

attendance (SBA). Another study 

found a similar threshold of 2.5 per 

1000 to reach 80% coverage for 

Skilled Birth attendants (SBA) or 

measles vaccination (WHO, 2006). 

These numbers have been cited by 

other sources as a general threshold 

for all basic health interventions, but 

perhaps more work is needed to test 

for other basic output indicators other 

than SBA and measles.

X X X X X

The ratio of doctors 

(physicians), nurses, 

midwives, pharmacists, 

and laboratory 

technicians, per 1,000 

population

Use the staffing requirement model 

developed for the WHO to determine 

thresholds (Hall, 2001), or the WPRO 

Workforce Projection Tool. The WHO 

found that a minimum of 2.3 

physicians, nurses, and midwives per 

1000 population were needed to meet 

80% coverage of skilled birth 

attendance (SBA). Another study 

found a similar threshold of 2.5 per 

1000 to reach 80% coverage for 

Skilled Birth attendants (SBA) or 

measles vaccination (WHO, 2006). 

These numbers have been cited by 

other sources as a general threshold 

for all basic health interventions, but 

perhaps more work is needed to test 

for other basic output indicators other 

than SBA and measles.

X Densities Nurses 

and MDs 

per 

1,000 

pop

X
78

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Distribution of health 

workers by profession/

specialization, region,

place of work, and sex

Use the staffing requirement model 

developed for the WHO to determine 

thresholds (Hall, 2001), or the WPRO 

Workforce Projection Tool.

X Mobility 

of 

professio

nal, tbd.

Annual number of 

graduates of health 

professions educational 

institutions per 100,000 

pop – by cadre

Use the staffing requirement model 

developed for the WHO to determine 

thresholds (Hall, 2001), or the WPRO 

Workforce Projection Tool.

X X

Indicators for 

monitoring country 

actions for 

strengthening the health 

workforce

This is a set of indicators; no 

threshold set. 

X

The distribution of health 

care professionals in 

urban and rural areas

No threshold set. The goal is to make 

sure that there are sufficient providers 

in rural areas. However, what is 

sufficient will vary by country, based 

on the labor-capital mix and disease 

burden. 

X X

Number of medical 

schools /nursing schools

No threshold set. X

HR data—Presence of 

human resources data 

system

Yes/No X

The existence of a 

functioning HR planning 

system

Yes/No X

HR dedicated budget Yes/No X

Proportion of health 

workers receiving training 

during previous year

No threshold set. X

% of health workers in 

rural areas receiving 

supervisory visits in last 

(X) months

No threshold set. X

Number of unfilled 

positions (rural vs. urban)

No threshold set. X
79

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Proportion of health 

workers at tertiary,

secondary, and primary 

levels

No threshold set. X

Ratio of GFATM to MOH 

funds covering per diems 

and salaries.

No threshold set. X

Information 

Systems

HISPIX Scale and scoring of the index are not 

yet known.

X 

(Toolkit)

Existence of national set 

of indicators with 

targets and annual 

reporting to inform 

health sector reviews 

and planning cycles (ie 

demographic and 

household surveys)

Yes/No X 

(Toolkit)

X

Percent of births (or 

deaths) registered in the 

country

<50% score 0

50–89% score 1

>=90% score 2

X 

(Toolkit)

X X

Percentage of districts 

that submit timely, 

complete, accurate 

reports to national level 

If the percentage is below 95 percent, 

then data quality is compromised. 

According to the WHO, "an accurate 

listing of public and private service 

delivery points should be available in 

all countries."

X 

(Toolkit)

% of 

districts 

represen

ted in 

reported 

info.

Completion of at least 

one national health 

account in last 5 years 

At least one national health account 

completed in last 5 years. 

X 

(Toolkit)

Existence of designated 

mechanisms charged 

with analysis of health 

statistics

Yes/No X (HMN), 

toolkit

X
80

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Availability of a national 

summary report which 

contains HIS 

information, analysis, 

and interpretation (most 

recent year)

Yes/No XX

The quality of health 

status Indicators as a 

proxy for the quality of the 

HIS (Mortality (Maternal 

mortality ratio, U5MR)

No threshold set. X (HMN)XXX

World Bank Statistical 

Capacity Indicator 

No threshold set. X (HMN)X

International Health 

Regulations implemented 

according to international 

standards

Yes/No X 

(Toolkit)

Availability of financial or 

physical resources (or 

both) to support the HIS 

within regional and district 

budgets

No threshold set.X

Two or more data points 

available for child mortality 

in the past 5 years

Threshold of two or more surveys 

within the past five years. 

X 

(Toolkit)

X

Health facilities or districts 

reporting all indicators 

according to national 

guidelines

No threshold set.XX

Health information 

systems that better inform 

policy-makers and facility 

managers on health status 

and trends: government 

implements a NHA, HMIS, 

and Clinical IS (EMR, etc.) 

No threshold set.X

Sentinel surveillance sites 

performing according to 

national standards 

(number and percentage)

No threshold set.X
81

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

ICD-10 used in district 

hospitals and causes of 

death reported to national 

level. 

WHO targets for ICD-10: 

<50% score 0; 

50–89% score 1; 

>=90% score 2

X

Percentage of disease 

surveillance reports 

received at the national 

level from districts

compared to the number 

of reports expected

No threshold set.X

Data quality* (e.g. 

presence of procedures to 

verify the quality of data 

reported, such as data 

accuracy checklists prior 

to report acceptance, 

internal data quality audit 

visits

WHO is one of the few sources to 

establish a target for data quality 

(“data quality assessment carried out 

and published within last 3 years; 

assessment should cover all routine 

data sources). 

**

Data analysis (e.g. 

designated mechanisms 

charged with analysis, 

utility, synthesis and 

validation of data from 

population and facility 

sources) 

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

*

Data dissemination policy 

(e.g. there is national 

commitment to 

transparency in data 

dissemination; a 

regulatory framework for 

the generation and use of 

health information

Yes/No XX

Percentage of private 

health facility data 

included in reported data

No threshold set.X

PRISM: indicators of the 

performance of HIS on 

behavioral, organizational, 

and technical 

determinants

This is a category of indicators;

qualitative continuum

X
82

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Medicines

Proportion of population 

with access to 

affordable essential 

drugs on a sustainable 

basis (e.g. availability of 

life-saving medicines)

MDG target is not specified, but the 

target is assumed to be 100%. 

X X

Availability and price of 

essential medicines

WHO global target: 80% availability; 

not more than 4 times the world 

market price. 

X * X

Percent of health 

facilities that have all 

tracer medicines and 

commodities in stock 

the day of visit, last 

three months (also 

average availability of 30 

selected essential 

medicines in public and 

private health facilities; 

varies time period of 

measurement).

WHO target: no stockouts. 

Global target: 80 percent.

Country-specific targets required. 

* 

(Toolkit)

* * *

Median proportion of 

tracer drugs that are in 

stock on the day of the 

visit, last three months

Supplement to above indicator. X 

(Toolkit)

Ratio of median local 

medicine price to 

international reference 

price for core list of 

drugs 

No threshold set - but the mark-up 

between manufacturers cost and 

consumer price has a rough target of 

below 70%. The target needs to be 

refined and needs to be country 

specific. 

X 

(Toolkit)

* % of 

average 

int'l price 

pd for 

last 

regular 

procure

ment of 

indicator 

drugs

X
83

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Existence and year of 

last update of a 

published national 

medicines policy

WHO target: Policy exists and was

updated within the last ten years. 

X 

(Toolkit)

X

Existence and year of 

last update of a 

published national list of 

essential medicines 

WHO target: Policy exists and was 

updated within the last two years. 

X 

(Toolkit)

X

Existence of Standard 

Treatment Guidelines 

e.g. measures of 

adherence to STGs.

WHO target: Policy exists and was

updated within the last five years. 

**

Percent of drugs 

purchased through 

competitive bidding over 

of the total 

pharmaceutical 

expenditure 

WHO target: 80 percentXXX

Appropriate prescribing 

practices and rational 

drug use

This is a category of indicators.

Indicators need to be defined before a threshold is set.

**

Existence and 

performance of key 

national pharmaceutical 

structures and processes 

(e.g. mechanisms for the 

licensing, inspection and 

control of (1) 

pharmaceutical personnel, 

(2) manufacturers, (3) 

distributors/importers, and 

(4) pharmacies/drug retail; 

a system for 

pharmaceutical 

registration; a national 

drug regulatory authority)

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

***

Legal provisions to allow 

generic substitution in the 

private sector

WHO target: Yes * 

(Toolkit)

*
84

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Are there formal standard 

operational procedures 

(SOPs) for conducting 

procurement

of pharmaceuticals in the 

public sector?

Yes/No X

Public and private per 

capita expenditure on 

medicine

No threshold set - country specific. X 

(Toolkit)

Percentage of population 

covered by health 

insurance

No threshold set - long term target is

100%. 

X X

Percent of facilities with 

expired items

No threshold given; assumed to be 

0%.

X X

Median consumer price

ratio of 30 selected 

essential medicines in 

public and private health 

facilities

WHO target: below 4x world market 

reference price. 

X 

(Toolkit)

X

Private expenditure on 

pharmaceuticals (per 

capita average exchange 

rate)

No threshold set - country specific. X

Drug use in health 

facilities

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

* *

Drug use in communities 

(e.g., most commonly 

used drugs, ideas about 

drug efficacy, drug 

sources, and treatments)

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

*

Government expenditure 

on medicines as a percent 

of the health budget

WHO established a target of 20 

percent, to be sustained over a threeyear period. 

X
85

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Financing

Total Health Expenditure 

(THE) per capita in 

international and US 

dollars

No clear threshold for this indicator 

(Carrin and Evans (2004). However,

Evans et al., (2001) found that health 

system performance sharply 

increases with total health 

expenditure up to about $80 (£53) per 

capita a year. Kruk and Freedman 

(2008) cite the Commission on 

Macroeconomics and Health (WHO, 

2001), that says "providing basic 

essential health care services would 

require expenditure in 2007 of at least 

$34 (30-40) USD per capita per year 

in low-income countries. Countries 

with relatively low per capita spending 

(e.g., below USD 30 per capita) are 

likely to have poor access, a low 

quality of health care, or both.

X Per 

capita

THE, at 

average 

exchang

e rate 

(USD)

X X X

THE as % of GDP No threshold set - Interpretation of 

this indicator is a challenge. In high 

income countries, the concern is 

overspending on health, while in low 

income countries, the primary issue is 

often under-spending on health. A 

target that specifies a minimum 

desired proportion of GDP (or of 

general government expenditure) for 

health spending is not very useful on 

scientific grounds (HMN, 2006).

X X X X X X X

General govt. health 

expenditure as a 

proportion of total 

government expenditure 

(GGHE/GGE)

Abuja Declaration- 15% of 

government budget to health. 

X X X X X X X
86

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

The ratio of HH OOP 

payments for health to 

total health expenditures

Lower figures are generally better. In 

poor countries, where the level of out 

of pocket expenditure is high, a 

decrease is desirable to increase 

protection for financial catastrophe

(HMN, 2006) On the other hand, it is 

not necessarily true that 0% out of 

pocket payment is desirable. Most 

countries impose some form of 

payment at point of service to 

discourage "over-use" (Carrin and 

Evans, 2004).

X Private 

exp. 

Health 

as % of 

THE

Amount of total donor 

spending on health as % 

of THE

A donor contribution of more than 10 

percent of a country’s total health 

spending is a problem for financial 

and institutional sustainability if the 

donor contributions are withdrawn 

(Islam et al., 2007). 

X External 

$ for 

health as 

% of 

THE

ODA as 

percent 

of GNI 

(not 

related to 

health)

X(w and 

w/o 

GFATM 

funds)

% of Government health 

financing that reaches 

the poorest income 

quintile

No threshold set. This is an indicator 

of equity, but appropriate allocation 

varies by country (Walford, 2007) 

X Share of 

exp. To 

poorest 

25% 

districts

X

Selected indicators for 

monitoring country 

actions for 

strengthening health 

financing

This is a set of indicators; no 

threshold set. 

X
87

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

% households 

impoverished annually by 

Out of Pocket spending/ % 

of population with 

catastrophic health 

expenditure, by 

expenditure quintile

The WHO defines catastrophic health 

expenditures as 40% or more of a 

family's income after basic needs are 

met (Kruk and Freedman, 2008). The 

World Bank threshold for this 

measure is 20% of all expenditures 

over a set period, a year for example

(World Bank, 2003). Others have 

classified expenditures as 

catastrophic when they put a family 

below the poverty line. “While it

is acknowledged that the choice of 

threshold is arbitrary, 10 per cent of 

total expenditure has been a common 

choice (Pradhan and Prescott, 2002; 

Ranson, 2002; Wagstaff and van 

Doorslaer, 2003); with the rationale 

that this represents an 

approximate threshold at which the 

household is forced to sacrifice other 

basic needs, sell productive assets, 

incur debt, or be impoverished 

(Russell 2004)”, cited in van 

Doorslaer and O’Donnell (2008). 

However, there is significant 

disagreement on such an arbitrary 

distinction, particularly across 

different income level countries. 

Other alternatives suggested include

% of non-food expenditures, 

concentration indices, etc.

X Fairness 

of 

financial 

contributi

on to 

Health 

system 

X X

% of population covered 

by health insurance

Private 

pre-paid 

plans as 

% of 

PHE

X X

Out-of-pocket spending as 

% of private expenditure 

on health

No threshold set; lower percentages 

indicate greater availability of private 

risk pooling mechanisms.

X X
88

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Public (government) 

spending on health as % 

of total health expenditure

If the percentage is below 

approximately 40 percent, it can 

reflect either a low tax capability of 

the country’s government, or a 

government philosophy of their

limited role in health (i.e., that public 

spending should not play a large role 

in financing or providing health 

services for the population), or both. 

A low value for this indicator also 

means that the government has 

limited ability to act to address equity 

issues. (Islam et al., 2007).

X Public 

HE as % 

of GDP

Total Public health 

expenditures 

disaggregated by source 

( by donor)

No threshold set. X

Govt. expenditure on 

wages and salaries as % 

of GGHE

No threshold set - but on average a 

country devotes just over 42% of total 

general government health 

expenditure to paying its health 

workforce (WHO, 2006) Governments 

in the 14 African countries have 

devoted a much lower proportion of 

their total expenditures to the health 

workforce (29% on average) than 

those in the other 50 countries (45% 

on average).

X Ratio of 

wages 

paid by 

Govt: by 

GFATM

Effectiveness of payment 

systems for primary and 

secondary health care 

providers

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

*

Sustainability of funds, 

Pace of disbursement, 

and verticality of program 

indicators

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

*
89

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Governance

Policy Index (10 

components)

X 

(Toolkit)

• Existence of an up-todate national health 

strategy linked to 

national needs and 

priorities

The target for is the existence of the 

standard international policy for each 

health goal.

* 

(Toolkit)

*

• Existence of an 

essential medicines list 

updated within the last 

five years and 

disseminated

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

* 

(Toolkit)

*

• Existence of policies 

on drug procurement 

which specify the most 

cost effective drugs in 

the right quantities; and 

open, competitive 

bidding of suppliers for 

quality products

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

* 

(Toolkit)

*

• Existence of a national 

strategic plan for TB that 

reflects the Stop TB 

Strategy

Yes/No X 

(Toolkit)

• Existence of a national 

malaria strategy/policy 

which includes drug 

efficacy monitoring, 

vector control, and 

insecticide resistance 

monitoring

Yes/No X 

(Toolkit)

• Completion of the 

UNGASS National 

Composite Policy Index 

Questionnaire

Yes/No X 

(Toolkit)
90

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

• Existence of 

comprehensive 

reproductive health 

policy

Yes/No (taking into account whether 

the plan has been systematically 

developed and costed) (WHO, 2005). 

X 

(Toolkit)

• Existence of an 

updated comprehensive, 

multi-year plan for 

childhood immunization

Yes/No X 

(Toolkit)

• Existence of key health 

sector documents, 

which are published and 

disseminated annually 

(also: health system 

goals, objectives, and 

performance targets 

clearly communicated to 

the public by the MOH)

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

* 

(Toolkit)

*

• Existence of 

mechanisms, such as 

surveys, for obtaining 

timely client input on the 

existence of appropriate, 

timely and effective 

access to health

services.

Yes/No X 

(Toolkit)

X

HRH: Health worker 

absenteeism in public 

health facilities

WHO marker indicator (no target 

given; assumed to be 0%)

X

Health Financing: 

Proportion of 

government funds which 

reach district-level 

facilities

WHO marker indicator (no target 

given; assumed to be 100%)

X

Health Service Delivery: 

Stock-out rates 

(absence) of essential 

drugs in health facilities

WHO marker indicator (no target 

given; assumed to be 0%)

X
91

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Health Service Delivery: 

Proportion of informal 

payments within the 

public health care 

system

WHO marker indicator (no target 

given; assumed to be 0%)

X

Pharmaceutical 

Regulation: Proportion 

of pharmaceutical sales 

that consist of 

counterfeit

drugs

WHO marker indicator (no target 

given; assumed to be 0%); 

X

Voice & Accountability: 

Existence of effective 

civil society 

organizations in 

countries with 

mechanisms in place for 

citizens to express 

views to government 

bodies

WHO marker indicator. Yes/No X

Disparity in coverage 

between lowest/ highest 

income groups (for 

DPT3 or other 

indicators)

Difficult to set a global or national 

threshold- will vary by region/group. 

X (HMN) X

World Bank Country 

Policy and Institutional 

Assessment (CPIA)

0-6 score, 6 is maximum. X X

Worldwide Governance 

Indicators: Civil Liberties, 

Political Rights, Voice and 

Accountability, 

Government 

Effectiveness, Rule of 

Law, Control of Corruption 

Measured in units ranging from about 

-2.5 to 2.5, with higher values 

corresponding to better governance 

outcomes. 

X X X X X

International Country 

Risk Guide (Political 

Risk Services Group)

0 to 100, 100 is maximum. X
92

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Voice and 

Accountability: (e.g. 

rules for public disclosure 

of information; existence 

of effective civil society 

organizations who can 

express views to 

government bodies; 

existence of policies that 

defines the role and 

responsibilities of the 

public health sector) Also 

part of World Bank 

governance indicators.

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

* *

Strengthening civil 

society/Social 

responsiveness (e.g., 

collaboration between 

government, civil society 

and other stakeholders in 

the planning, budgeting, 

and monitoring of activities 

in the health sector; 

number of civil society 

members trained; 

existence of active citizen 

health boards; MOH 

outreach with information, 

education, and 

communication.

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

* * *
93

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Pharmaceutical 

regulation (e.g., 

proportion of 

pharmaceutical sales that 

consist of counterfeit 

drugs; (USAID= Adequate 

regulation to ensure the 

safety, efficacy, and 

quality of medicines, as 

well as the 

appropriateness and 

accuracy of product 

information)

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

* *

Percentage of publicly 

funded supplies and 

services leaked* (e.g., 

sale of publicly funded 

drugs in private markets, 

fraudulent insurance 

billing practices)

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

*

Proportion of informal 

payments within the public 

health care system

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

X

Percent of government 

spending devoted to 

health

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

X X X X

Government budget 

process (e.g., explicit 

procedures for resource 

allocation with 

transparency and 

rationales)

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

* * *

Public/private mix (e.g., 

absolute number of private 

and public health care 

providers

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

* * *

Proportion of government 

funds which reach districtlevel facilities

No threshold set. X
94

HS Building Block/

Indicator Threshold/Benchmark/Target WHO [a]

USAID 

HS 20/20 

[b]

MCC/MC

A (long 

list) [c]

Literatur

e review 

[d] MDG [e]

World 

Bank [f]

SWEF 

[g]

ECHIM 

[h] DFID [i]

USAID 

E&E 

Grad.[j]

Share of expenditure to

poorest 25% of districts

See Finance section. X

Government coordination 

or harmonization of donor 

inputs

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

* *

DPT3 immunization 

(proxy)

X X

Management capacity 

indicators

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

*

Health worker 

absenteeism in public 

health facilities

This is a category of indicators.

Indicators need to be defined before

a threshold is set. 

* *

Stock-out rates (absence) 

of essential drugs in 

health facilities

No stock-outs (assumed to be 0%). * * *

Provider/institutional 

report cards* (e.g. human 

resources management, 

supervision)

Yes/No X X

Legislative and Policy 

Reforms are enacted, with 

increased focus on 

community-based primary 

health care

Yes/No X

Guide to Appendix 1:

Bolded: Indicates these indicators are discussed in greater detail in the text

* : Indicates that this source mentions indicators on this topic but presents it in a different form (ie uses multiple indicators, 

suggests topic area but does not present specific indicator, index vs. indicator, etc.)

Yellow highlighting: Indicates that this indicator is a health output indicator

Blue highlighting: Indicates the indicator overlaps with a financing, workforce, or medicines indicator

Pink Highlighting : Identifies WHO's proposed list of ten rules-based indicators for key aspects of health policy
95

Notes: 

The Global Fund, referenced in the text of this literature review encourages the use of existing in-country indicators used to monitor health systems performance. For example, 

the specific indicators that are part of a program-based approach (including SWAp) performance matrix or other national strategic framework (The Global Fund, 2008). 

Sources for Indicators:

[a] WHO. (2008). Toolkit on Monitoring Health Systems Strengthening. DRAFT. Geneva: World Health Organization, June 2008; Health Metrics Network (HMN), WHO

[b] Islam, M., ed. 2007. Health Systems Assessment Approach: A How-To Manual. Submitted to the U.S. Agency for International Development in collaboration with Health 

Systems 20/20, Partners for Health Reformplus, Quality Assurance Project, and Rational Pharmaceutical Management Plus. Arlington, VA: Management Sciences for Health.

[c] Millennium Challenge Corporation, Guide to the MCC Indicators and the Selection Process, Fiscal Year 2008

[c] Becker, L., Pickett, J. & Levine, R. (2006), Measuring Commitment to Health: The Global Health Indicators Working Group Report.. Washington, DC: Center for Global 

Development. 

[d] Kruk, M. & Freedman, L. (2008). Assessing Health System Performance in Developing Countries: A Review of the Literature. Health Policy, 85(3):263–276.

[e] United Nations. (2003). Indicators for Monitoring the Millennium Development Goals--Definitions, Rationale, Concepts and Sources. New York, NY: United Nations. 

[f] World Bank CPIA Index Web site, 

http://web.worldbank.org/WBSITE/EXTERNAL/EXTABOUTUS/IDA/0,,contentMDK:20948754~menuPK:2625191~pagePK:51236175~piPK:437394~theSitePK:73154,00.html

[g] GFATM (2003). Evaluating the Health System-Wide Effects of the Global Fund to Fight AIDS, Tuberculosis and Malaria. System wide Effects of the Fund (SWEF) Research 

Network, November 2003.

[h] The ECHIM website has full descriptions of specific indicators; http://www.healthindicators.org/healthindicators/object_document/o4963n28314.html . Also the strategy is 

outlined in Kramers, P.G.N. (2003). European Journal of Public Health 13(Supplement 1): 101-106. 

[i] Walford, V. (2007). Indicators of Health System Performance: Report of a Review for DFID. London: DfID Health Resource Centre.

[j] USAID. (2001). Graduation Report on Increased Health Promotion and Access to Quality Health Care in the Europe and Eurasia (E&E) Region. Internal document, developed 

by E&E Bureau’s Program Objective Team for Strategic Objective 3.2.
96

Citations for thresholds:

Carrin ,G. & Evans, D. (2004). Monitoring Health Financing Changes: Key Indicators. Paper presented at a WHO/World Bank meeting on Health System Metrics: Monitoring 

the Health System in Developing Countries, Glion, Switzerland. October 6–7 2004. 

Evans, D., Tandon, A., Murray, C., & Lauer, J. (2001). Comparative Efficiency of National Health Systems: Cross National Econometric Analysis. British Medical Journal, 

323:307–310.

Hall, T. F. (2001). Human Resources for Health: Models for Projecting Workforce Supply and Requirements. Geneva: World Health Organization. 

Health Metrics Network. (2006). Health System Metrics: Report of a Technical Meeting. Glion, Switzerland, 28-29 September 2006: World Health Organization.

Holdsworth, G., Garner, P. A., & Harpham, T. (2006). Crowded Outpatient Departments in City Hospitals of Developing Countries: A Case Study from Lesotho. The International 

Journal of Health Planning and Management, 8(4): 315–324.

Kruk, M. & Freedman, L. (2008). Assessing Health System Performance in Developing Countries: A Review of the Literature. Health Policy, 85(3):263–276.

Pradhan, M. & Prescott, N. (2002). Social Risk Management Options for Medical Care in Indonesia. Health Economics. 11(5): 431-446. 

Ranson, M. K. (2002). Reduction of Catastrophic Health Care Expenditures by a Community-based Health Insurance Scheme in Gujarat, India: Current Experiences and 

Challenges. Bulletin of the World Health Organisation, 80(8): 613-21.

Russell, S. (2004). The Economic Burden of Illness for Households in Developing Countries: A review of Studies Focusing on Malaria, Tuberculosis, and Human 

Immunodeficiency Virus/Acquired Immunodeficiency Syndrome. American Journal of Tropical Medicine and Hygiene, 71(Supp. 2): 147-155.

Sjönell, Göran (1984). Relationship Between Use of Increased Primary Health Care and Other Out-patient Care in a Swedish Urban Area I. Utilisation of Public Primary Health 

Care. Scandinavian Journal of Primary Health Care, 2 (1), 33-39. 

van Doorslaer, E. & O’Donnell, O. (2008). Measurement and Explanation of Inequality in Health and Health Care in Low-Income Settings, UNU-WIDER Discussion Paper No. 

2008/04.

Wagstaff, A. & van Doorslaer, E. (2003). Catastrophe and Impoverishment in Paying for Health Care: With Applications to Vietnam 1993-1998. Health Economics, 12 (11):921-

34.

Walford, V. (2007). Indicators of Health System Performance: Report of a Review for DFID. London: DfID Health Resource Centre.

World Bank. (2003). Catastrophic Health Care Payments. Quantitative Techniques for Health Equity Analysis—Technical Note # 18. Washington, DC: The World Bank. 
97

WPRO Workforce Projection Tool Web page. http://www.wpro.who.int/NR/rdonlyres/0DEFEE72-65C3-4C20-B374-0F4687DA97D8/0/WPROWorkforceProjectionTool.pdf 

WHO. (2001). Macroeconomics and Health: Investing in Health for Economic Development: Report of the Commission on Macroeconomics and Health. Geneva: World Health 

Organization.

WHO. (2005). The World Health Report 2005: Making Every Mother and Child Count. Geneva: World Health Organization.

WHO. (2005). Resolution WHA 58. 31: Working Towards Universal Coverage of Maternal, Newborn and Child Health Interventions. Geneva: World Health Organization.

WHO. (2006). The World Health Report 2006: Working Together for Health. Geneva: World Health Organization.
98

Appendix 2: Health Output Indicators

Technical Area Output Indicators

Tuberculosis TB case detection rate (new smear-positive)

TB treatment success rate (new smear-positive)

Malaria Insecticide-treated net (ITN) Use

Intermittent preventive treatment (IPT)

Indoor residual spraying (IRS)

Artemisinin-based drug treatment of malaria

Maternal and Child 

Health 

Number of antenatal care visits (1 visit)

Number of antenatal care visits (4 visits)

Births attended by skilled health personnel

Delivery at a health facility

Proportion of 1-year-old children immunized against measles

Percentage of 1-year-olds immunized with the third dose of DPT

Percent of children ages 12 to 23 months fully immunized before age 1

Pneumonia care-seeking

Oral rehydration therapy use

Vitamin A supplementation

Exclusive breastfeeding

Family Planning and 

Reproductive Health

Modern contraceptive prevalence rate

Unmet need for family planning

HIV/AIDS Condom use rate of the contraceptive prevalence rate

Condom use at last high-risk sex

Percentage of population ages 15 to 24 with comprehensive correct knowledge of 

HIV/AIDS

Proportion of population with access to affordable, essential drugs on a sustainable 

basis

Other Public Health 

Threats

Treatment of neglected tropical diseases in endemic areas

Bacterial resistance rates in documented infections

Detection and treatment of noncommunicable diseases