Options for evaluating the impact of malaria control efforts on
mortality in African countries with high malaria burden.

An analysis conducted for the U.S. President's Malaria Initiative.

Final version

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

[]{.mark}

Primary author and contact person for this document: Alexander Rowe,
Malaria Branch, Division of Parasitic Diseases, National Center for
Infectious Diseases, Centers for Disease Control and Prevention,
Mailstop F22, 4770 Buford Highway, Atlanta, Georgia, United States.

Telephone: +1 770-488-3588; Fax: +1 770-488-7761; Email: axr9@cdc.gov.

[]{.mark}

[Other contributors include (listed alphabetically):]{.underline} Fred
Arnold (ORC Macro), Protik Basu (World Bank), Peter Bloland (CDC), Don
de Savigny (Swiss Tropical Institute), Patricia Graves (CDC), Bill
Hawley (CDC), Ali Khan (CDC), Patrick Kachur (CDC), Eline Korenromp
(GFATM), Matt Lynch (USAID), Bernard Nahlen (CDC/GFATM), Rob Newman
(CDC), Samantha Rowe (CDC), Trent Ruebush (USAID), Allan Schapira (WHO),
Larry Slutsker (CDC), Rick Steketee (PATH), Tessa Wardlaw (UNICEF), and
Carla Winston (CDC).

Not all contributors have read and agreed to the content and
recommendations of this draft.

Abbreviations

ACM All-cause mortality

ACT Artemisinin-based combination therapy

CDC Centers for Disease Control and Prevention

CI Confidence interval

DHS Demographic and Health Survey

DSS Demographic surveillance system

EIR Entomological inoculation rate

GFATM Global Fund to Fight AIDS, Tuberculosis, and Malaria

HIS Health information system (or health and management information
system)

HIV Human immunodeficiency virus

INDEPTH International Network for the continuous Demographic Evaluation
of Populations

> and Their Health

IPTp Intermittent preventive treatment for pregnant women

IRS Indoor residual spraying

ITN Insecticide-treated net

MACEPA Malaria Control and Evaluation Partnership

MARA Mapping Malaria Risk in Africa

M&E Monitoring and evaluation

MERG Monitoring and Evaluation Reference Group

MTIMBA Malaria Transmission Intensity and Mortality Burden Across Africa

MICS Multiple Indicator Cluster Survey

MIS Malaria Indicator Survey

PBT Preceding birth technique

PECM Prompt and effective case management

PMI President's Malaria Initiative

RBM Roll Back Malaria

SAVVY Sample vital registration with verbal autopsy

Under 5s Children under 5 years old

UNICEF United Nations Children's Fund

VA Verbal autopsy

WHO World Health Organization

Executive summary

Introduction

The goal of the President's Malaria Initiative (PMI) is to reduce
malaria-associated mortality by 50% in a group of African countries with
a high malaria burden. To determine if this goal has been achieved, an
evaluation method is needed. As weak vital registration systems in
countries targeted by the PMI do not produce valid estimates of malaria
mortality, other options must be considered. This document describes
these other options, including their costs, advantages, and
disadvantages, and makes recommendations on how the PMI should evaluate
its impact. Methods for monitoring the PMI's progress on an ongoing
basis are mentioned briefly. The concepts and recommendations in this
document are also applicable to the evaluation of other malaria
initiatives that are part of the Roll Back Malaria partnership.

## Specifying some assumptions about the evaluation

First, it is assumed that the direct measurement of malaria mortality is
not possible, and therefore, a conceptual bridge is needed to link the
"unmeasurable" goal of reducing malaria mortality (directly measured) to
measurable indicators. Second, mortality reduction will be measured
relative to the mortality level of a "baseline" (pre-PMI) time period.
Third, the evaluation will take a "pooled multi-country" approach in
which the focus will be to evaluate mortality reductions in the combined
populations of all PMI-funded countries, as opposed completely separate
evaluations in each country. It is recognized, however, that data will
be collected from all PMI-funded countries and that country-specific
results will always be presented alongside the pooled results. Fourth,
non-malarious areas of PMI-funded countries should be excluded from the
evaluation whenever possible. Fifth, as other malaria initiatives might
be operating in PMI-funded countries, no specific attempt will be made
to evaluate the separate contributions of each initiative in the same
country.

## Description of the evaluation options

Eight evaluation options are presented in detail, including advantages,
disadvantages, and estimated cost. The options can be used singly or in
combination. Most options use the same basic approach. First, an
indicator of malaria mortality is measured over a period of time.
Second, data are collected concurrently on factors that could influence
malaria mortality, including malaria-program factors (e.g., coverage of
interventions to control malaria) and non-malaria-program factors (e.g.,
rainfall). Much of this information can be collected from representative
household surveys, but some information could come from other sources.
Third, trends in the coverage of interventions are analyzed to confirm
that coverage has increased to a high level. Finally, trends in the
indicator of malaria mortality are interpreted taking into account
changes in the prevalence of non-malaria-program factors, as these could
confound the relationship between malaria mortality and the coverage of
interventions that a malaria control program has implemented over time.
A key limitation of all options is that they have relatively weak
designs for proving causality. The use of multiple options may mitigate
the weakness of a single option. Also, to complement indicators of
malaria mortality, it is useful to follow trends in anemia prevalence,
which is a measure of malaria morbidity and in the causal pathway
between programmatic activities and reductions in malaria mortality.

The following are the eight evaluations options.

1.  In representative populations, measure trends in all-cause child
    mortality, coverage of malaria control interventions, and other
    factors that might influence child mortality.

2.  In demographic surveillance systems (DSSs), measure trends in
    malaria-specific mortality, coverage of malaria control
    interventions, and other factors that might influence malaria
    mortality.

3.  Use results on malaria-specific mortality and all-cause mortality
    (ACM) from DSSs to interpret trends in ACM from population-based
    surveys, controlling for coverage of malaria control interventions,
    and other factors that might influence malaria mortality.

4.  In representative populations, measure trends in malaria-specific
    mortality (by adding verbal autopsies to population-based surveys),
    coverage of malaria control interventions, and other factors that
    might influence malaria mortality.

5.  Use results on malaria-specific mortality and ACM from DSSs to
    interpret trends in ACM from population-based surveys, controlling
    for coverage of malaria control interventions, and other factors
    that might influence malaria mortality; [also]{.underline}, measure
    trends in malaria mortality with verbal autopsies added to
    population-based surveys.

6.  In community-based sentinel sites, measure trends in
    malaria-specific mortality, ACM, coverage of malaria control
    interventions, and other factors that might influence malaria
    mortality.

7.  Estimate malaria mortality trends from a mathematical model based on
    coverage of malaria control interventions and other factors that
    might influence malaria mortality.

8.  Measure trends in malaria-specific mortality by analyzing data from
    a country's health information system or data abstracted from
    hospital records, coverage of malaria control interventions, and
    other factors that might influence malaria mortality.

## Dissemination of results

A dissemination plan should be developed that includes: 1) publishing
the evaluation plan; 2) communicating results in a timely fashion to
countries and other partners; 3) coordinating with other malaria
initiatives; 4) creating a website with data from surveys and other
evaluation activities; and 4) publishing results in scientific articles.
A philosophy of complete transparency should be adopted.

## Discussion and recommendations

There is no simple, valid way to evaluate the impact of malaria control
efforts in countries with a high malaria burden in Africa. All options
described in this document have important shortcomings. As malaria
initiatives have similar donors and are likely to work in the same
countries, it would be efficient and desirable for all initiatives to
use the same evaluation plan. This approach agrees with the "Three Ones"
principle (one country-level monitoring and evaluation system).

Two issues complicate the development of simple recommendations: the
degree to which evaluations will be coordinated among countries has not
yet been decided, and different countries have different constraints
with regards to funding and which types of surveys can be conducted at a
particular time. To address these complications, an algorithm was
developed to help choose among the evaluation options. In addition to
the algorithm, a detailed generic country-specific evaluation plan is
presented.

The plan assumes a multi-country evaluation, mortality surveys with
verbal autopsy, the availability of a DSS-based model, and no system of
sentinel sites (option 6). Given these assumptions and the algorithm, it
is recommended that PMI-funded countries use evaluation option 5 (plus
complementary estimates provided by options 1 and 2). The primary
benefits of option 5 are that it produces results that are nationally
representative, the impact indicator is malaria-attributed mortality,
and it uses multiple data sources such that each data source makes up
for the limitations in other data sources. The plan also has three
elements that are not, strictly speaking, necessary for the final
evaluation of impact on mortality, but which serve other important
needs: 1) intervention coverage surveys every 1--2 years, 2) operations
research projects, and 3) capacity-building activities related to
conducting surveys.

The analysis in this document permits several other practical
recommendations to be made. First, a dedicated evaluation team is
needed. Second, the PMI should re-consider the programmatic targets of
85% coverage for malaria control interventions. Third, in the future,
quantitative goals of large-scale public health initiatives should be
stated in measurable terms that are appropriate for the health issue to
be addressed. Finally, operations research should be conducted to fill
knowledge gaps that exist for the evaluation methods; and as new
knowledge becomes available, the evaluation plan should be adjusted
accordingly.

The analysis in this document has several limitations. First, numerous
assumptions are required, many of which are not supported by
high-quality data. Second, cost estimates might be inaccurate. Third,
the measurement of changes smaller than the desired 50% mortality
reduction is not discussed. Fourth, little has been said about malaria
risk mapping, although better maps are needed. Fifth, new interventions
are not mentioned, such as intermittent preventive treatment for
infants. Sixth, certain important monitoring issues are not addressed,
such as antimalarial resistance, insecticide resistance, and adverse
drug reactions. Finally, other methods for measuring malaria burden,
such as measuring the prevalence of the hemoglobin S gene, are not
mentioned because too little is known about their practical application
to evaluating programs.

**I. Purpose of this document and the intended audience**

[]{.mark}

The goal of the President's Malaria Initiative (PMI) is to reduce
malaria-associated mortality by 50% in a group of African countries with
a high malaria burden. Building on the Abuja targets, the PMI plans to
achieve 85% coverage of the most vulnerable groups (children under five
years of age, pregnant women, and people living with HIV/AIDS) with key
preventive and therapeutic measures. The achievement of these targets is
expected to result in a 50% reduction in malaria-associated deaths after
three years of full implementation in each country (i.e., 2010 for the
first three countries). To determine if the mortality reduction goal has
been achieved, an evaluation (see Box 1 for definitions) method is
needed. The most simple and logical method---measuring trends in the
total number of malaria deaths per year (or a rate based on the number
of malaria deaths)---cannot be used because the vital registration
systems of the countries targeted by the PMI have relatively low
coverage (i.e., many deaths are missed) and uncertain reliability (i.e.,
it is difficult to assess the accuracy of reported causes of deaths) and
thus do not produce valid estimates of the number of malaria deaths
\[Mathers, 2005; page 1272 and Table 1 of Murray and Lopez, 1997; Table
3 (page 8) of Murray et al, 2001\]. Other methods for evaluating impact
on malaria mortality, however, have been developed. The purpose of this
document is to describe these other methods, including their estimated
costs, advantages, and disadvantages, and to make recommendations on how
the PMI should evaluate its impact on malaria mortality. This document
is not intended to address the important issue of which methods could be
used for the ongoing monitoring (Box 1) of malaria mortality; however
several evaluation options might contribute to monitoring efforts, and
these are mentioned. Moreover, this document does not address how to
evaluate the impact of malaria control efforts on other "impact"
measures of malaria's burden, such as the burden of morbidity and the
economic burden.

Although the primary purpose of this document relates to the evaluation
of the PMI, the concepts and recommendations are applicable to other
malaria initiatives that are part of the Roll Back Malaria (RBM)
\[Nabarro, 1998; RBM website\] partnership (e.g., the Global Fund to
Fight AIDS, Tuberculosis, and Malaria \[GFATM\] \[GFATM website\]; The
World Bank's Malaria Booster Program \[World Bank Booster website; World
Bank strategy document, 2005\]; and the Malaria Control and Evaluation
Partnership \[MACEPA\] \[PATH website\]) and are potentially relevant to
organizations with more general interests in monitoring, evaluation, and
health information (e.g., the Health Metrics Network
\[http://www.who.int/healthmetrics/en/ (accessed January 18, 2006)\]).
The concepts and recommendations in this document may also be useful in
developing an evaluation plan for the Millennium Development Goal 6,
target 8 (to halt by 2015 and begin to reverse the incidence of malaria)
\[MDG website\].

The intended audience for this document includes health program officers
and researchers interested in a highly detailed discussion of methods
for evaluating the impact of malaria control programs on mortality.
Shorter, less detailed versions of this document will be produced for
the general international public health community.

+-----------------------------------------------------------------------+
| # Box 1. Definitions used in this document                            |
|                                                                       |
| *Consequential malaria mortality.* Consequences of clinical           |
| management (e.g., inadvertent exposure to HIV during a blood          |
| transfusion for non-life threatening severe anemia) or sequelae of a  |
| malaria infection (e.g., epilepsy resulting from brain insults during |
| cerebral malaria) lead to a child's death \[Page 18 of Snow et al.    |
| DCPP, 2003\].                                                         |
|                                                                       |
| *Demographic surveillance system.* A study site in which relatively   |
| small populations (e.g., generally 5,000--140,000 people) are         |
| monitored closely to identify all births, deaths, in-migration, and   |
| out-migration \[INDEPTH, 2002; Byass, 2002\].                         |
|                                                                       |
| *Direct malaria mortality.* Deaths in which malaria was the           |
| underlying cause (defined as "the disease or injury which initiated   |
| the train of morbid events leading directly to death" \[page 30 of    |
| WHO, 1993\]).                                                         |
|                                                                       |
| *Entomological inoculation rate*. Number of infective mosquito bites  |
| per unit time. It is a measure of malaria transmission intensity.     |
|                                                                       |
| *Evaluation.* The periodic assessment of whether objectives are being |
| achieved, often requiring special surveys or studies.                 |
|                                                                       |
| *Indirect malaria mortality.* Deaths in which malaria was a           |
| contributing cause (e.g., a child with malaria-associated anemia dies |
| after developing pneumonia: neither the anemia nor pneumonia would    |
| have been fatal in isolation; both diseases were necessary to be      |
| fatal). Another example is a child whose death is partly caused by    |
| low birth weight (a major risk factor for infant mortality            |
| \[Steketee, 2001\]), when the low birth weight was caused by malaria  |
| (and malaria-related anemia) in the mother during pregnancy.          |
|                                                                       |
| *Monitoring.* The ongoing measurement of progress towards an          |
| objective, often using routinely collected data.                      |
|                                                                       |
| *Positive predictive value.* The probability that a death identified  |
| as a malaria death by a particular method (e.g., verbal autopsy or    |
| HIS) is truly a malaria death. For example, if the HIS identifies 100 |
| malaria deaths and of these, 90 are found to be true malaria deaths,  |
| then the positive predictive value of the HIS is 90% (90/100).        |
|                                                                       |
| *Sensitivity.* The probability that true malaria deaths are           |
| identified by a particular method (e.g., verbal autopsy or HIS) as    |
| malaria deaths. For example, if there are 1000 true malaria deaths in |
| a country in a year and the HIS identifies 100 of these 1000 malaria  |
| deaths, then the sensitivity of the HIS is 10% (100/1000).            |
|                                                                       |
| *Verbal autopsy.* A method for determining causes of deaths in which  |
| relatives of the deceased person are asked about the signs and        |
| symptoms of the child's terminal illness, usually 1 to 6 months after |
| the death \[Garenne, 1990; Anker, 1999\]. To attribute causes of      |
| deaths, results from these interviews are analyzed either by an       |
| algorithm, a single clinician, or several clinicians who decide on    |
| causes of death by majority vote.                                     |
+-----------------------------------------------------------------------+

# II. Specifying the goal 

II.A. *Issue 1: The outcome of malaria mortality*

PMI has a goal of reducing malaria-associated mortality by 50%. RBM has
stated its goals in two ways: halve malaria-associated mortality by 2010
\[Nabarro, 1998\], and more recently, halve the burden of malaria by
2010 \[Brochure on RBM website,
<http://www.rbm.who.int/docs/rbm_brochure.pdf>, accessed February 10,
2006\]) A fundamental challenge for these initiatives is that in all
African countries with a high malaria burden, it is practically
impossible to count all malaria deaths (see section I). Indeed, as most
deaths occur at home, it is not even feasible to obtain a representative
sample of deaths with enough clinical and laboratory information to
identify the true cause with certainty. Therefore, a dilemma exists: if
malaria deaths cannot be counted, how can one determine whether the
number has been reduced by 50%? \[Attaran, 2005\]

There are at least two options for resolving the dilemma. First, one
could alter the phrasing of the goal (e.g., "the goal is to
substantially reduce the burden of malaria mortality") and then define
"substantial reduction in malaria mortality" in terms of measurable
indicators. Second, one could retain the existing goal, but describe the
measurement problem and state an operational definition of how
reductions in malaria mortality will be assessed using measurable
indicators. These two options are similar because rigorous evaluation
methods (which all donors and stakeholders desire) require measurable
indicators, so either way leads to the same methodological conclusion.
However, there are differences between the two options. The first option
changes what policy-makers have said. The advantage is that it
completely solves the technical dilemma---one is free to define a
qualitative outcome (e.g., "substantial reduction") in measurable terms.
The disadvantage is it may be politically unacceptable because it may
sound as if a promise is being altered. Someone might complain that
resources were mobilized based on a promise of a very specific result,
and now it is not clear if that result will be delivered. In contrast,
the second option does not change what policy-makers have said. The
advantage is that the potential political problem mentioned above is
avoided. The disadvantage is that it is easy to see a shift in the
outcome, which might lead to the same criticism mentioned above. In
addition, given the precise language of the goal, someone might perceive
such a shift as misleading (e.g., the goal has been altered in the "fine
print"). That said, with the second option, it is also quite possible
that most policy-makers and others in the public health community will
appreciate that some specification of the goal is needed for sound
technical reasons. Assuming policy-makers want to avoid the political
problems associated with changing an often published and widely
recognized goal, and are willing to accept a shift in the outcome, the
second option is preferable.

II.B. *Issue 2: Specifying the method for measuring mortality reduction*

There are at least two methods of measuring mortality reduction \[Watt &
Dye, 2000 (page 28)\]. First, one could choose a year (e.g., 1998, the
beginning of RBM; or 2005, the beginning of PMI) as the baseline year,
and then determine whether malaria mortality is 50% lower in an
end-of-project year (e.g., 2010). Second, if trend data are available,
one could estimate the number of cases that would have occurred in the
end-of-project year in the absence of the malaria initiative, and then
assess whether actual mortality is 50% lower relative to the
end-of-project year projection.[^1] Admittedly, most African countries
with a high malaria burden will not have reliable trend data for malaria
mortality. Therefore, for the purposes of this document, the first
method described above will be used.

The above methods focus primarily on changes in mortality over time. A
complementary method could compare mortality trends in PMI-funded
countries to trends in malarious countries without funding from PMI (or
other malaria initiatives), controlling for numerous potential
confounding effects. Controlling for confounding, however, might be a
challenge as some confounders might be collinear with the characteristic
"received funding from PMI or other malaria initiatives" (e.g., presence
of civil war, extremely weak infrastructure, or mismanagement of public
funds).

II.C. *Issue 3: Should the evaluation focus on individual countries or a
multi-country region?*

As the PMI will support interventions in multiple countries, there are
at least 2 approaches for collecting and synthesizing data. First, an
individual country approach would involve a separate evaluation in each
country, and then a synthesis of all country-specific data to evaluate
the PMI as a whole. For PMI to be considered a success, a 50% reduction
in malaria mortality would be required in each country. Second, a pooled
multi-country approach would involve considering the populations of all
PMI-funded countries as a single population. For PMI to be considered a
success, a 50% reduction in malaria mortality would be required in the
single pooled multi-country population.

The choice of approach is important for several practical reasons.
First, compared to the individual country approach, the pooled
multi-country approach will probably require smaller sample sizes and
thus fewer resources. Second, certain evaluation options only make sense
for a particular approach. For example, option 6 (see section III.G,
below) involves data collection from sentinel sites in multiple
countries, and the results for the small number of sites in any single
country will probably have limited value. Thus, evaluation option 6
essentially requires the pooled multi-country approach. Third, the
pooled multi-country approach will probably require better coordination
among countries funded by PMI (and probably among other initiatives, as
other initiatives will probably be asked to fund some evaluation
activities). As a result, this approach is riskier because if certain
evaluation activities are not conducted well and on time, the entire
evaluation could suffer. Fourth, the assessment of change over time is
more complex with the pooled multi-country approach because countries
will begin scaling up coverage of malaria control interventions at
different times. If, for example, three countries begin in 2006 and five
other countries begin in 2008, it would be inappropriate to assess
impact in all eight countries by comparing results from 2005 to results
from 2010. However, given that PMI countries are expected to demonstrate
mortality reductions after three years of full implementation, the
pooled multi-country approach could compare baseline results to results
after three years of full implementation---regardless of the particular
year when this milestone is reached (unless it is long after the end of
the PMI). Finally, the pooled multi-country approach would be less
likely to demonstrate statistically significant results for each
individual country, which might be a concern for donors and in-country
partners (especially, for national malaria control programs). To address
this concern, one might define "success" for individual countries with a
lower confidence level (e.g., 90% confidence) than the 95% confidence
level used for the pooled multi-country area (for justifications of this
approach, see note 3 in Annex 1).

The PMI goal does not specify which approach should be used. However, a
draft PMI proposal document does mention that PMI intends to reduce
malaria mortality for 175 million Africans, which suggests that the
total population is more important than specific countries. The way
populations are divided among countries is, after all, an accident of
history. Other justifications of this approach are that the sample size
requirements will be lower and that the PMI will not be judged a failure
if a single country misses its targets (although other countries would
need to surpass their targets). For the purposes of this document, the
primary approach will be the pooled multi-country approach. However, it
is acknowledged that results must be presented for individual countries;
and wherever possible, methods will be recommended that have adequate
statistical power to demonstrate when the impact goal has been achieved
for individual countries. It is also acknowledged that the decision
might not rest with the PMI alone---other malaria initiatives might need
to be consulted.

II.D. *Issue 4: Non-malarious areas in malarious countries*

Several African countries with a high malaria burden (e.g., Kenya) have
sizeable populations living in areas with essentially no malaria
transmission. For these countries, the evaluation will focus on only
malarious areas. If the population in non-malarious areas is small
(\<5%), an effort should be made to exclude these populations (e.g.,
from intervention coverage surveys). However, if such exclusions are
thought to over-complicate survey designs, then the populations in
non-malarious areas could probably be ignored (so the country's
population is considered a single population). Non-malarious areas that
are to be excluded should be specified in country-specific evaluation
plans before the evaluation begins.

II.E. *Issue 5: Attributing credit for mortality reductions to multiple
malaria control initiatives*

As several malaria control initiatives are investing in Africa, it is
likely that any given country or group of countries will benefit from
more than one initiative. In such cases, a relevant question is whether
the evaluation of an individual initiative should require an assessment
of its particular impact---separated from the impact of other
initiatives working in the same area. For example, in a single country,
if initiative A contributes \$30 million and initiative B contributes
only \$1 million and the program reduces malaria mortality by 50%,
should the evaluation of initiative B count the success fully? If yes,
then small projects could be piggybacked on larger ones and perhaps make
unfair claims about their impact.

The PMI goal does not specify whether impact should be evaluated
separately from other initiatives operating in PMI-funded countries.
However, the PMI emphasizes the importance of working with in-country
partners, which suggests that credit for successes should be shared. For
the purposes of this document, evaluation options will not consider the
contributions of different initiatives separately when operating in the
same country. However, the level of financial support that each major
initiative provides should be noted for all countries, so readers can
make their own judgments of who deserves credit and can detect unfair
claims of impact by small projects.

**III. Description of options, estimated costs, advantages, and
disadvantages**

## III.A. General considerations for all options

Eight evaluation options are presented in this section (see summary in
Table 1). For a given country or group of countries, options can be used
singly or in combination. Most options use the same basic approach.
First, an indicator of malaria mortality is measured over a period of
time. Second, data are collected concurrently on factors that could
influence malaria mortality, including malaria-program factors (e.g.,
coverage of insecticide-treated nets \[ITNs\], prompt and effective case
management \[PECM\] of malaria, intermittent preventive treatment of
pregnant women \[IPTp\], and indoor residual spraying \[IRS\]) and
non-malaria-program factors (e.g., rainfall, proportion of the
population living in urban areas, HIV prevalence, coverage of
interventions to prevent maternal-to-child transmission of HIV, and
indicators of wealth such as gross domestic product). Much of this
information can be collected from representative, population-based
surveys, but some information could come from other sources. For
example, meteorological databases could be used to assess rainfall[^2],
antenatal clinic data could be used to measure HIV prevalence and IPTp
coverage, and administrative data could be used to measure IRS coverage.
Third, trends in the coverage of ITNs, PECM, IPTp, and IRS are analyzed
to confirm that coverage has increased to a high level (e.g., at least
70%)[^3]. Finally, trends in the indicator of malaria mortality are
interpreted taking into account changes in the prevalence of
non-malaria-program factors, as these could confound the relationship
between malaria mortality and the coverage of interventions that a
malaria control program has implemented over time. This generalized
method is presented schematically in Figure 1 (a description of the role
of anemia measurement and use of plausibility arguments is provided
below).

Figure 1. A schematic depiction of the evaluation method

![](media/image1.wmf)

Table 1. Summary of the attributes of each evaluation option

+-----------+-----+-----+-------+-----+-------+-----+-------+------+
| Attribute | E   |     |       |     |       |     |       |      |
|           | val |     |       |     |       |     |       |      |
|           | uat |     |       |     |       |     |       |      |
|           | ion |     |       |     |       |     |       |      |
|           | opt |     |       |     |       |     |       |      |
|           | ion |     |       |     |       |     |       |      |
+-----------+-----+-----+-------+-----+-------+-----+-------+------+
|           | 1   | 2   | 3     | 4   | 5     | 6   | 7     | 8    |
|           |     |     |       |     |       |     |       |      |
|           | ACM | DSS | DSS + | ACM | DSS + | Se  | Model | He   |
|           | s   | o   |       | s   |       | nti | based | alth |
|           | urv | nly | ACM   | urv | ACM   | nel | on    | faci |
|           | eys |     | su    | eys | su    | si  | in    | lity |
|           |     |     | rveys | w   | rveys | tes | terve | data |
|           |     |     |       | ith | with  |     | ntion |      |
|           |     |     |       | VA  | VA    |     | cov   | (e   |
|           |     |     |       |     |       |     | erage | .g., |
|           |     |     |       |     |       |     |       | from |
|           |     |     |       |     |       |     |       | HIS) |
+-----------+-----+-----+-------+-----+-------+-----+-------+------+
| Abb       | *   | *   | *ACM  | *   | *     | *Se | *     | *    |
| reviation | ACM | DSS | surv  | ACM | DSS + | nti | Model | HIS* |
| (shown in | su  | on  | eys + | sur | ACM   | nel | only* |      |
| italics   | rve | ly* | DSS*  | vey | surv  | sit |       |      |
| in the    | ys* |     |       | s + | eys + | es* |       |      |
| rest of   |     |     |       | VA* | VA*   |     |       |      |
| this      |     |     |       |     |       |     |       |      |
| document) |     |     |       |     |       |     |       |      |
+-----------+-----+-----+-------+-----+-------+-----+-------+------+
| Represent | Exc | F   | Good  | Exc | Good  | G   | Good  | Fair |
| ativeness | ell | air | to    | ell | to    | ood |       |      |
|           | ent |     | exce  | ent | exce  |     |       |      |
|           |     |     | llent |     | llent |     |       |      |
+-----------+-----+-----+-------+-----+-------+-----+-------+------+
| Validity  | Not | G   | Good  | F   | Good  | G   | Not   | Good |
| of deaths | a   | ood |       | air | for   | ood | appli |      |
| a         | ppl |     |       | to  | DSS;  |     | cable |      |
| ttributed | ica |     |       | G   | fair  |     |       |      |
| to        | ble |     |       | ood | to    |     |       |      |
| malaria   |     |     |       |     | Good  |     |       |      |
|           |     |     |       |     | for   |     |       |      |
|           |     |     |       |     | s     |     |       |      |
|           |     |     |       |     | urvey |     |       |      |
|           |     |     |       |     | VAs   |     |       |      |
+-----------+-----+-----+-------+-----+-------+-----+-------+------+
| Validity  | Not | G   | Good  | G   | Good  | G   | De    | D    |
| of trends | a   | ood |       | ood |       | ood | pends | iffi |
| in        | ppl |     |       |     |       |     | on    | cult |
| malaria   | ica |     |       |     |       |     | m     | to   |
| mortality | ble |     |       |     |       |     | odel; | know |
| rate      |     |     |       |     |       |     | could |      |
|           |     |     |       |     |       |     | be    |      |
|           |     |     |       |     |       |     | good  |      |
+-----------+-----+-----+-------+-----+-------+-----+-------+------+
| Includes  | No  | Y   | Yes,  | No  | Yes,  | Yes | Yes,  | Yes  |
| older     |     | es, | pot   |     | pot   |     | pot   |      |
| children  |     | p   | entia |     | entia |     | entia |      |
| and       |     | ote | lly\* |     | lly\* |     | lly\* |      |
| adults    |     | nti |       |     |       |     |       |      |
|           |     | all |       |     |       |     |       |      |
|           |     | y\* |       |     |       |     |       |      |
+-----------+-----+-----+-------+-----+-------+-----+-------+------+
| Outcome   | Co  | D   | Di    | D   | Di    | D   | Com   | Dir  |
| is        | mbi | ire | rect, | ire | rect, | ire | bined | ect, |
| "direct   | ned | ct, | or    | ct, | or    | ct, |       | or   |
| malaria   |     | or  | com   | or  | com   | co  |       | comb |
| mo        |     | co  | bined | co  | bined | mbi |       | ined |
| rtality," |     | mbi |       | mbi |       | ned |       |      |
| or        |     | ned |       | ned |       |     |       |      |
| "combined |     |     |       |     |       |     |       |      |
| direct +  |     |     |       |     |       |     |       |      |
| indirect  |     |     |       |     |       |     |       |      |
| malaria   |     |     |       |     |       |     |       |      |
| m         |     |     |       |     |       |     |       |      |
| ortality" |     |     |       |     |       |     |       |      |
| (or       |     |     |       |     |       |     |       |      |
| option    |     |     |       |     |       |     |       |      |
| can       |     |     |       |     |       |     |       |      |
| produce   |     |     |       |     |       |     |       |      |
| both)     |     |     |       |     |       |     |       |      |
+-----------+-----+-----+-------+-----+-------+-----+-------+------+
| Cost      | Hig | Hig | H     | Hig | H     | V   | I     | Ine  |
|           | hly | hly | ighly | hly | ighly | ery | nexpe | xpen |
|           | var | var | vari  | var | vari  | exp | nsive | sive |
|           | iab | iab | able† | iab | able† | ens |       |      |
|           | le† | le† |       | le† |       | ive |       |      |
+-----------+-----+-----+-------+-----+-------+-----+-------+------+
| Comment   | Hig | Few | Based |     | Ben   | La  | Many  | Requ |
|           | hly | ex  | on    |     | efits | rge | a     | ires |
|           | dep | ist | d     |     | from  | und | ssump | va   |
|           | end | ing | ata + |     | use   | er- | tions | lida |
|           | ent | D   | model |     | of    | tak | ne    | tion |
|           | on  | SSs |       |     | mul   | ing | eded; |      |
|           | pla |     |       |     | tiple |     | req   |      |
|           | usi |     |       |     | data  |     | uires |      |
|           | bil |     |       |     | so    |     | valid |      |
|           | ity |     |       |     | urces |     | ation |      |
|           | ar  |     |       |     |       |     |       |      |
|           | gum |     |       |     |       |     |       |      |
|           | ent |     |       |     |       |     |       |      |
+-----------+-----+-----+-------+-----+-------+-----+-------+------+

[Footnotes:]{.underline}

ACM = All-cause mortality, DSS = demographic surveillance system, VA =
verbal autopsy, HIS = health information system.

\* Not all DSSs conduct verbal autopsies for individuals 5 years of age
and older.

† Depends on who pays for the survey or DSS. If the malaria program
pays, then very expensive; however mortality surveys (e.g., Demographic
and Health Surveys) and DSSs are often funded by sources external to the
malaria program.

All options require data on the coverage of malaria interventions.
Although a thorough discussion of monitoring systems is outside the
scope of this document, it should be mentioned that such data are the
foundation of a monitoring system. Some data can be collected through
routine systems (e.g., number of ITNs distributed, sold, or retreated;
quantity of antimalarials distributed; number of pregnant women
receiving IPTp) and some can be collected from household surveys. All of
this information can be quite useful for managing programs. However for
the purpose of evaluation, the options in this document generally
require intervention coverage data from representative household
surveys. The reason is that the results of such surveys can be
generalized to large populations of interest (e.g., the entire country).

Given the importance of intervention coverage data from household
surveys for the evaluation options in this document, a few points should
be made on how these surveys should be conducted. First, the surveys
should use probability sampling. Second, non-malarious parts of the
country should be omitted from the sampling frame, unless they represent
a small proportion (e.g., \<5%) of the population. Third, questions
should be standardized so results can be compared over time and between
countries. Standard questions have been developed by RBM and can be
found in the Malaria Indicator Survey (MIS) \[MERG website\]. Fourth,
regarding sample size, see details in Annex 1. Finally, when surveys are
done periodically, they should be done at about the same time of the
year so trends can be assessed without the potential confounding effect
of seasonality. For countries with distinct rainy and dry seasons,
several points should be made: 1) rainy season measurements, when
malaria transmission is highest, are better linked to public health
outcomes, but probably overestimate coverage throughout the year (and
RBM recommends ITN use throughout the year); 2) dry season measurements
are the most conservative because they avoid overestimation, but they
are likely to underestimate coverage when it counts the most; and 3)
because of the above points, there is some logic to measuring coverage
in rainy and dry seasons (and there is precedent for this approach:
parasite prevalence studies for assessing transmission intensity
sometimes include measurements in rainy and dry seasons). For most
countries, it would be too expensive to conduct more than one survey per
year. Therefore, to obtain information with the greatest public health
relevance, it is recommended that surveys be conducted no more than once
a year during the rainy season.

To complement indicators of malaria mortality, regardless of which
evaluation options are used, it is useful to follow trends in anemia
prevalence \[World Malaria Report, 2005 (page 84)\] (hemoglobin \<8
g/dl) \[minutes of RBM-MERG Anemia Task Force\]. A recent meta-analysis
concluded that anemia is a reliable indicator of malaria morbidity that
can reflect the impact of malaria control interventions \[Korenromp,
2004 TMIH\]. Moreover, compared to indicators of malaria mortality,
anemia is easier and less expensive to measure. When added to household
surveys, such as a Demographic and Health Survey \[DHS\] \[ORC Macro
website\], the cost of anemia testing is about \$50,000--70,000 for a
survey of 8000 households. Anemia should be measured in children 6--59
months old during the rainy season or within 6 weeks of its end, and
serial anemia measurements should occur at about the same time of the
year. Practically speaking, given that anemia and ITN use will often be
measured by the same survey and that surveys might take a month or more
to complete, a reasonable time to conduct such surveys is during the
second half of the rainy season or up to 2 weeks after the end of the
rains (as mosquitoes and malaria transmission persist for several weeks
after the end of the rains). This timing would allow for a rainy season
measurement of ITN use and a measurement of anemia that reflects
children's exposure to at least half of the peak malaria transmission
season. Another approach that has been considered is measuring anemia at
immunization clinics. At present this approach is still being
investigated (see Box 2 for operations research priorities related to
monitoring and evaluation of malaria control programs) and is not
currently recommended.

+-----------------------------------------------------------------------+
| # Box 2. Priority o                                                   |
| perations research issues for improved evaluation methods and results |
|                                                                       |
| #                                                                     |
|                                                                       |
| # *[Highest priority issues]{.underline}*                             |
|                                                                       |
| 1\. What is the sensitivity and specificity of verbal autopsies for   |
| identifying malaria deaths when the verbal autopsy "interview" has    |
| been simplified to a few questions asked in a household survey 1--5   |
| years after the death (e.g., if verbal autopsies are added to DHS)?   |
|                                                                       |
| 2\. How can a final estimate of malaria mortality be made when two or |
| more estimates are available---i.e., "triangulation" (e.g., for       |
| evaluation option 5 \[*DSS + ACM surveys + VA\**\])?                  |
|                                                                       |
| 3\. What is the best way to interpret changes in a mortality rate     |
| (all-cause or malaria-specific) in a single country or group of       |
| countries that quantitatively accounts for changes in the values of   |
| multiple potential confounders (e.g., rainfall, HIV prevalence, urban |
| residence, wealth, etc)? Also, which potential confounders must be    |
| considered and how they should be measured?                           |
|                                                                       |
| 4\. What is the true burden of indirect malaria mortality? Similarly, |
| given that the level of effort to control other diseases that         |
| influence the indirect malaria mortality burden is likely to change   |
| over time in a particular country or group of countries, how can one  |
| quantify the reduction in indirect malaria mortality that is          |
| attributable to malaria control efforts?                              |
|                                                                       |
| 5\. What is the sensitivity and specificity of verbal autopsies       |
| ("traditional" verbal autopsies as well as verbal autopsies conducted |
| as part of survey) for identifying malaria deaths when the coverage   |
| of malaria control interventions is relatively high? Similarly, what  |
| is the sensitivity and specificity of verbal autopsies for            |
| identifying malaria deaths when malaria transmission is low?          |
|                                                                       |
| 6\. What is the burden of malaria mortality for children              |
| [\>]{.underline}5 years old and adults?                               |
|                                                                       |
| 7\. What is the validity of a model (based on DSS data or other data) |
| for predicting malaria mortality when applied to representative       |
| household surveys that include mortality, such as DHS (e.g., for      |
| evaluation option 3 \[*ACM surveys + DSS\**\])? Similarly, what is    |
| the validity of a model (based on DSS data or other data) for         |
| predicting malaria mortality when applied to representative household |
| surveys of malaria intervention coverage that do [not]{.underline}    |
| include mortality (e.g., for evaluation option 7 \[*model only\**\])? |
|                                                                       |
| # *[Other priority issues]{.underline}*                               |
|                                                                       |
| 8\. What is the best way to map malaria risk?                         |
|                                                                       |
| 9\. What are the validity, utility, and cost of health facility-based |
| data on malaria cases and deaths, and on programmatic activities      |
| (e.g., quality of case management performed by health workers, and    |
| ITN use collected during routine immunization visits, during sick     |
| child visits, or estimated from inpatient data) when used to monitor  |
| and evaluate the impact of malaria control interventions?             |
|                                                                       |
| 10\. What is the utility of measuring parasite prevalence during      |
| household surveys?                                                    |
|                                                                       |
| 11\. What is the validity of measuring anemia at immunization visits? |
|                                                                       |
| \* Italicized text indicates an abbreviation for an evaluation option |
| (see first row of Table 1).                                           |
+-----------------------------------------------------------------------+

Another complementary indicator of malaria morbidity is parasite
prevalence \[World Malaria Report 2005 (page 84)\]. There are two main
reasons for using this indicator. First, most malaria impact measures
considered in this document (all-cause mortality \[ACM\], verbal
autopsy-defined malaria deaths, and anemia) are indirect measures;
whereas parasite prevalence is a direct, malaria-specific measure.
Second, as reducing transmission is a goal of malaria control programs,
transmission should be measured; and parasite prevalence is a rough
measure of transmission. Also, when transmission levels are low (e.g.,
an entomological inoculation rate \[EIR\] \<1 bite per person per year),
"gold standard" measures of transmission (such as the EIR) are less
sensitive to change, and malaria-attributable anemia largely disappears.
In this situation, parasite prevalence may be the only available measure
of transmission that is still sensitive to change.

However, there is an important reason why parasite prevalence might not
be useful. Studies of malaria control interventions have often found
that despite reductions in mortality, parasite prevalence changes
little. For example, a Cochrane review of ITN efficacy found that in
areas of stable malaria transmission (EIR \>1 bite per person per year),
ITNs reduced the prevalence of *Plasmodium falciparum* infection by only
10% (summary of 3 trials in which control groups had untreated nets) to
13% (summary of 7 trials in which control groups had no nets)
\[Lengeler, 2004, Table 10\]. At present, the utility of this indicator
is still being examined (Box 2), so it is not currently recommended for
widespread use.

[]{.mark}

There are several limitations common to all evaluation options in this
document. First, they all have relatively weak designs for proving
causality. The primary reasons are: 1) stronger designs, such as
randomized controlled trials, are not feasible; and 2) it is recognized
that even if the weak designs do introduce small-to-moderate amounts of
bias, it will probably not have important public health consequences.
The use of multiple options, however, may mitigate the weakness of a
single option. Watt and Dye have proposed 5 groups for categorizing
strength of inference, which might be useful in communicating final
results \[Watt & Dye, 2000\].[^4]

The second limitation is that correctly measuring and accounting for
non-programmatic factors such as rainfall and urban residence, which may
confound the relationship between malaria mortality and the coverage of
malaria interventions implemented over time, may be difficult to do
reliably. A precise characterization of potential confounding factors
that must be considered, how they should be measured in a standardized
way, and how changes in malaria mortality should be adjusted to account
for the changing prevalence of such factors are high- priority issues
(Box 2). Third, the surveys used to measure intervention coverage and
ACM have limitations. For example, the adequacy of PECM is difficult to
measure because most household surveys do not assess aspects of case
management such as correct dosage. Other methods, such as health
facility surveys, could serve as complementary data sources for
examining the quality of PECM.

III.B. *Option 1. In representative populations, measure trends in
all-cause child mortality*[^5]*, coverage of malaria control
interventions, and other factors that might influence child mortality.
\[Abbreviation = ACM surveys\]*

III.B.1. [Description.]{.underline}

All-cause child mortality (e.g., ~5~q~0~, the probability that a
live-born child dies before his or her fifth birthday) is measured over
a period of time, using either representative, population-based surveys
(e.g., DHS, UNICEF's Multiple Indicator Cluster Surveys \[MICS\]
\[UNICEF website\]), or another valid method (e.g., a census, or the
preceding birth technique \[PBT\] \[Hill, 1988; Hill, 1990\]). Each
year, the Interagency Child Mortality Estimates Group (including WHO,
UNICEF, The World Bank, and the United Nations Population Division)
review all available data on all-cause child mortality to make
country-by-country mortality estimates \[UNICEF website; Hill, 1999\],
which are published in the State of the World's Children Report and
other reports. The method to synthesize mortality data has recently been
refined such that standard errors can be estimated and the statistical
significance of mortality trends can be determined \[personal
communication, Edilberto Loaiza, UNICEF, January 31, 2006\]. This method
could be used by individual countries to assess mortality trends. For
countries with sizable populations living in non-malarious areas (e.g.,
Kenya), only the mortality of children in malarious areas would be
considered (and in such countries, the WHO/UNICEF/World Bank/UN method
mentioned above might be difficult to implement because it might not be
possible to stratify results for past mortality surveys by malarious
versus non-malarious areas).

There are several justifications for this indicator. First, in
sub-Saharan Africa, a relatively large proportion (estimated as about
18%) of deaths among children under 5 years old ("under 5s") are
directly attributable to malaria (Box 1), and a substantial proportion
of deaths are indirectly related to malaria (Box 1) \[Rowe (in press)\]
(see Annex 2 for a discussion of this burden). Second, most malaria
deaths in sub-Saharan Africa occur in the under-5 age group. From the
results of a recent study, one can estimate a proportion of 63.6%
(679,658 under-5 malaria deaths / 1,068,505 all-age malaria deaths
\[Hay, 2005 (Table 3)\]) for the year 2000, although this proportion
might be an underestimate because malaria mortality rates for older
children and adults might have been overestimated.[^6] Third,
randomized-controlled trials have demonstrated that increasing the
coverage of malaria control interventions can significantly reduce
all-cause child mortality \[Lengeler, 2004; Kidane 2000\]. In fact, the
primary impact indicator for ITN trials was ACM (when supported by other
indicators such as anemia and measures of malaria transmission). Other
advantages are listed below (see section III.B.3). The use of ACM as an
indicator of the impact of malaria control programs has been mentioned
in several articles and monitoring and evaluation reports \[Molineaux,
1997; Watt & Dye, 2000; World Malaria Report (page 21); RBM M&E
Framework, 2000; Global Fund M&E Toolkit, 2005; RBM-AFRO M&E, 2004\] and
is recommended by RBM's Monitoring and Evaluation Reference Group
(MERG), a group of experts that provide technical advice to the RBM
Partnership. The RBM-MERG also recognizes the need to have more in-depth
data from a few representative sentinel sites in at least a few
countries to assist with interpretation of ACM trends \[MERG Mortality
Task Force minutes\] (see option 3 \[*ACM surveys + DSS*\] and 5
\[*DSS + ACM surveys + VA*\], below).

This option assumes that malaria mortality reductions among older
children and adults (i.e., individuals [\>]{.underline}5 years old) will
be half (a conservative estimate, see below) of the reductions for under
5s (e.g., if interventions are found to reduce the under-5 malaria
mortality by 30%, it is assumed that malaria mortality for older
children and adults was reduced by 15%). In general, this assumption is
supported by the notion that malaria control interventions can protect
individuals of all ages. For example, the use of ACTs in health
facilities should prevent malaria deaths in all age groups; and ITNs,
even if distribution targets young children and pregnant women, are
likely to protect others in the same household or nearby households if
coverage levels are relatively high \[Hawley, 2003; Gimnig, 2003\]. More
specifically, this assumption is supported by results from a
multi-faceted health systems intervention project in Tanzania in which
it appears that from 1999--2003, ACM (per 1000 persons per year)
decreased significantly by about 55% among under 5s (from 36.8 to 16.5
per 1000 person years), decreased non-significantly by about 46% among
children 5--15 years old (from 2.6 to 1.4 per 1000 person years), and
decreased non-significantly by about 22% among people 15--59 years old
(from 8.5 to 6.6 per 1000 person years). However, across the whole age
group from 5--59 years, mortality declined significantly by 26% (from
6.2 to 4.6 per 1000 person years) \[personal communication, D. de
Savigny, February 5, 2006\].[^7] Also, results from the Pare-Taveta IRS
study in Kenya and Tanzania showed that decreases in ACM ratios (deaths
per 1000 live births for infants, and deaths per 1000 at risk for older
children and adults; comparing the period from 1954--56, when malaria
transmission was still "intense or was just beginning to decline," to
1956--58, when transmission was "negligible") were about 48% (from
\~216.7 to \~113.0) for infants, about 52% (from \~26.1 to \~12.4) for
children 1--4 years old, about 46% (from \~5.7 to \~3.1) for children
5--14 years old, about 45% (from \~9.4 to \~5.2) for adults 15--39 years
old, and about 38% (from \~23.1 to \~14.4) for adults 40 years and older
\[Bradley, 1991\]. As a result of this assumption (and the further
assumption that 63.6% of all malaria deaths occur among under 5s \[Hay,
2005\]), then to have an overall reduction in malaria mortality of 50%
(the PMI goal), malaria mortality reductions for under 5s must be 61.1%
(Figure 2, point A). Alternatively, supposing that 80% of all malaria
deaths occur among under 5s, then to have an overall reduction in
malaria mortality of 50%, malaria mortality reductions for under 5s must
be 55.6% (Figure 2, point B).

Figure 2. The percent reduction in under-5-year old malaria mortality
needed to show a 50% reduction in malaria mortality for all ages, for
various values of the assumed reduction in malaria mortality for older
children and adults relative to children under 5 years old (for two
assumptions on the percentage of all malaria deaths occurring among
children under 5 years old)

![](media/image2.wmf)

More conservative assumptions could be used, however. For example,
assuming that malaria control interventions have no impact on older
children and adults (and that 63.6% of all malaria deaths occur among
under 5s), to have an overall reduction in malaria mortality of 50%,
malaria mortality reductions for under 5s must be 78.6% (Figure 2, point
C). Furthermore, it is possible that as malaria transmission decreases,
older children and adults will be more susceptible to dying from
malaria. Thus in the long term, reductions in malaria deaths among under
5s could be offset by increases in malaria mortality among older
children and adults. It is important to note that this entire discussion
of the relative impact of malaria interventions on children versus
adults is based on little evidence. Therefore, improving our
understanding of malaria mortality among older children and adults
should be a priority (Box 2).

In summary, for the purposes of this document, to demonstrate a
reduction in malaria mortality of 50% for all ages (the PMI goal), one
would need to measure a malaria mortality reduction of 55.6% or 61.1%
(depending on assumptions) among under 5s.

To implement option 1, coverage of malaria control interventions among
vulnerable populations (i.e., children under 5 years old and pregnant
women), the prevalence of other factors that might influence malaria and
non-malaria child mortality (e.g., vaccination coverage, breastfeeding,
access to improved water sources and sanitation, proportion of births
attended by a health professional, malnutrition, and other factors
described in section III.A), and anemia are concurrently measured with
data from population-based surveys and other valid data sources. To
evaluate the impact of malaria control activities over the time period,
trends in ACM, anemia, coverage of malaria control interventions, and
other factors are examined together. If statistically significant
reductions in ACM and anemia are found [and]{.underline} coverage of
malaria control interventions has increased to levels high enough to
affect child mortality [and]{.underline} other factors that might
influence child mortality have not changed substantially (or have
changed in a direction that would lead to an increase in child
mortality, e.g., vaccination coverage has decreased), then one can make
a plausibility argument \[Habicht, 1999; Victora, 2004\] that the
malaria control activities caused or contributed to the reduction in
mortality.[^8] As reductions in ACM and anemia get larger and the
coverage of malaria control interventions increases, the conclusion that
malaria control activities had a positive public health impact becomes
more plausible.

The plausibility argument might be strengthened if age-specific
reductions in ACM matched an expected profile. For example, the Asembo
Bay ITN trial \[Phillips-Howard, 2003\] found that in the
intention-to-treat analysis, ACM for children 1--11 months old was
reduced by 23% and ACM for children 12--59 months old was reduced by 7%;
and in the analysis of ITNs retreated within 6 months, ACM for children
1--11 months old was reduced by 26% and ACM for children 12--59 months
old was reduced by 14%. Perhaps this profile can be considered the
"signature" of a reduction in malaria-specific mortality caused by ITNs
and could be used when interpreting ACM reductions in a program. For
example, if ACM among under-5s decreased by the target (e.g., 15%) after
increasing ITN coverage, but most of the mortality reduction was among
neonates, one might conclude that the prevented deaths were
[unlikely]{.underline} to have been malaria deaths prevented by the
ITNs. Age-specific mortality estimates could be obtained from surveys
such as DHS (e.g., in 3 categories: neonates, 1--11 months, and 12--59
months).

How much ACM reduction should be expected? There are two ways to answer
this question. First, one can estimate ACM reduction assuming that
malaria mortality is reduced by 50% (PMI's goal). However, as it is not
known exactly what coverage levels of malaria control interventions are
needed to reduce malaria mortality by 50%, a second approach is to
estimate how much ACM would decrease if intervention coverage increased
from a low to a reasonably high level. Annex 2 shows that if malaria
mortality among under 5s decreases by 50%, under-5 ACM would decrease by
about 16.8% (precision estimate: 14.3--19.2%) (left side of Figure 3)
(see Annex 2 for a complete description of assumptions made regarding
the proportion of all-cause child mortality directly and indirectly
related to malaria). A sensitivity analysis based on minimum assumed
intervention efficacy found that under-5 ACM would decrease by 14.0%
(precision estimate: 11.9--16.0%), and a sensitivity analysis based on
maximum assumed intervention efficacy found that under-5 ACM would
decrease by 22.4% (precision estimate: 19.1--25.6%). Annex 3 shows that
if the coverage of malaria interventions increased from low levels
(\<5%) to a reasonably high level (70%), under-5 ACM would decrease by
19.5% (precision estimate: 13.5--24.8%) (right side of Figure 3). A
sensitivity analysis based on maximum assumed intervention efficacy
found that under-5 ACM would decrease by 21.6% (precision estimate:
15.6--26.9%). It is worth noting that estimates from these two methods
are roughly similar; thus if the estimates are correct, one could
conclude that increasing intervention coverage to 70% should decrease
malaria mortality by about 50%. It is also worth noting that a recently
published analysis found that a pair of DHSs with a "typical" sample
size (4000--8000 women 15--49 years old) would have adequate statistical
power to detect a reduction in all-cause under-5-year mortality of 15%
or more \[Korenromp et al, IJE 2004\], so typical DHSs probably have
adequate sample size to evaluate the PMI (assuming one can wait several
years for results, see disadvantages for option 1).

[]{.mark}

[]{.mark}

Figure 3. Estimates of the reduction in all-cause mortality for children
0--59 months old that could be expected if malaria mortality is reduced
by 50%, and if the coverage of malaria control interventions is
increased from a low (\<5%) to a high (70%) level

![](media/image2.wmf)

III.B.2. [Estimated cost.]{.underline}

Survey costs depend on many factors. The following are rough estimates
for several types of surveys that would provide the data needed for
option 1 \[Personal communications from F. Arnold, ORC Macro, September
25, 2005; and M. Vaessen, ORC Macro, December 23, 2005\]. For a standard
"full-scale" DHS (including anemia testing, numerous questionnaire
modules not related to malaria, publishing and distributing a 400-page
final report, a national seminar to disseminate results,
capacity-building activities, purchasing computer equipment, etc.) with
a typical sample size of about 10,000 women 15--49 years old (which is
adequate for estimating ~5~q~0~ for the 5 years before the survey), the
cost would be about \$2.1 million. This cost, however, is usually shared
among multiple partners. For a DHS with a larger sample size of about
17,000 women (to estimate ~5~q~0~ for the 3 years before the survey),
the cost would be about \$500,000 more than a standard DHS with 10,000
women (i.e., about \$2.1 million + \$500,000 = \$2.6 million). Whenever
data can be used that were collected mainly for other purposes in a
full-scale DHS, the cost of the data collection to the malaria program
is no more than what the malaria program put into the cost of the DHS.
For a DHS-type survey that only measured malaria indicators (including
anemia) and all-cause child mortality (with a sample size of 10,000
women, enough to estimate ~5~q~0~ for the 5 years before the survey),
the cost would be between \$600,000 and \$1 million. Such a survey would
not include the extra benefits of a standard DHS (i.e., only a short
report, and no extra dissemination or capacity-building activities). The
estimated cost of a MICS is \$250,000 -- \$400,000 \[personal
communication, Tessa Wardlaw, January 31, 2006\].

III.B.3. [Advantages.]{.underline}

-   The option provides results that are representative of, and thus can
    be generalized to, large populations of interest (e.g., an entire
    country, or a part of a country, in case the PMI selects a country
    such as Kenya, which has a sizable population living in areas with
    no malaria risk).

-   The option captures the impact of malaria control interventions on
    direct, indirect, and consequential malaria mortality (Box 1).

-   ACM can be measured reliably. The option does not depend on verbal
    autopsies or other methods for determining causes of death, which
    may not be reliable (see explanation below in the disadvantages of
    option 2 \[*DSS only*\]).

-   By using ACM as the outcome indicator, the option has a broad public
    health perspective. To illustrate the point, consider a country in
    which the ideal indicator of malaria deaths could be reliably
    measured. Imagine that this indicator showed a 50% reduction in
    malaria deaths, but also imagine that other data showed that ACM had
    changed little because people who had been saved from malaria were
    dying a few months later from pneumonia and diarrhea. One could
    argue that the battle was won but the war was lost. In contrast, if
    the outcome indicator of ACM shows a mortality reduction and
    indicators of the coverage of malaria control interventions show
    substantial increases, one could be satisfied that the overarching
    goal of saving lives (the implicit goal of all disease-specific
    control programs) had been achieved---as well as the
    disease-specific goal of preventing malaria deaths.

-   If implemented over the right time frame in the right places, the
    option would probably not be very expensive for the PMI because it
    uses data from surveys that have been already completed and surveys
    that are already planned and funded by sources external to the PMI
    (e.g., DHS). However, it should be noted that if the PMI requires
    results more quickly, more surveys might be needed or surveys might
    need larger sample sizes---all of which would require additional
    resources.

-   If ACM could be measured frequently (e.g., PBT provides monthly
    estimates of ~2~q~0~, the probability a live-born child dies before
    the second birthday), such data could be used to monitor malaria
    mortality, although ACM might lack the "face validity" that
    malaria-specific mortality has (see disadvantages of option 1,
    below).

III.B.4. [Disadvantages.]{.underline}

-   Use of an outcome indicator that is not malaria mortality might lead
    to two problems. The first potential problem is that ACM has many
    causes besides malaria. Each cause has its own determinants, and the
    prevalence of these determinants may change over time. Thus, as
    coverage of malaria control interventions increases, ACM could
    conceivably increase, decrease, or remain unchanged, depending on
    trends in the determinants of non-malaria mortality. Not only might
    determinants of non-malaria mortality potentially confound the
    relationship between malaria intervention coverage and ACM, but
    factors such as rainfall and urbanization (non-programmatic factors
    that can influence malaria mortality) might also be confounders.
    Measuring and accounting for these factors is explicitly described
    in the method; however in practice, measuring and accounting for
    these factors may be difficult to do reliably. Thus, if malaria
    intervention coverage increases and ACM decreases (the ideal
    situation), it may be difficult to be sure whether the mortality
    decreases were truly [caused]{.underline} by the improved coverage
    of the malaria control interventions. Also, if malaria intervention
    coverage increases and ACM does not decrease as expected, it would
    be difficult to be sure whether the malaria control activities were
    ineffective: perhaps non-malaria mortality was increasing for
    unrelated reasons (or perhaps malaria is a "competing risk," \[RBM
    M&E Framework, 2000; Watt & Dye, 2000 {page 26}\] so although
    children are protected against dying from malaria, they might soon
    die from another cause), and these increases in non-malaria
    mortality offset true decreases in malaria mortality.

-   A related issue is that the method depends on an ecologic
    association, and thus is particularly susceptible to bias known as
    the "ecologic fallacy" \[Morgenstern, 1982\]. That is, the outcome
    (ACM) and the explanatory factors (coverage of malaria control
    interventions and prevalence of other factors influencing mortality)
    are measured at the population level---not the individual level.
    Thus, even if the coverage of malaria control interventions
    increases [and]{.underline} ACM decreases [and]{.underline} other
    factors that influence mortality are measured and accounted for in
    the analysis, the mortality decrease could have been truly caused by
    the non-intervention factors acting at the individual level.[^9]

-   The second potential problem caused by the use of an outcome
    indicator that is not malaria mortality is that the results from
    this method might suffer from a lack of face validity. In other
    words, even if success is found with this method, some people might
    simply not believe that the true objective of reducing malaria
    deaths by 50% had been achieved.

-   The use of a plausibility argument (needed both because the outcome
    indicator is not malaria mortality and because the method depends on
    an ecologic association) is a disadvantage because it is, in a
    sense, a qualitative method---i.e., a qualitative interpretation of
    quantitative data. As such, precision is difficult to define.
    Usually the concept of precision (e.g., as measured with a
    confidence interval) relates to sampling error; however with a
    plausibility argument, in addition to sampling error, a key concern
    is bias (and biases from sources that may be difficult to measure
    and account for). Thus, if there is a statistically significant
    increase in the coverage of malaria control interventions from 5% to
    70% and there is a statistically significant decrease in ACM from a
    ~5~q~0~ of 100 to 80 (i.e., results that indicate success by this
    method), it may be difficult or impossible to quantify the
    confidence one can have in the conclusion that malaria mortality was
    reduced by 50%. It should be mentioned, however, that a working
    group of the Child Health Epidemiology Reference Group (CHERG)
    \[Bryce, 2005\] is developing methods for quantifying precision.

-   If currently existing surveys that measure mortality (e.g., DHS) are
    used, the method does not capture mortality trends among older age
    groups, and relies on the rough assumption that mortality reductions
    among older age groups is equal to some proportional reduction among
    under 5s. The failure to measure mortality among older age groups
    might miss the impact of changes in the underlying malaria
    epidemiology caused by successful interventions that shifts the
    burden of mortality to older age groups. Although such changes are
    unlikely to occur in the relatively short, 5-year lifespan of PMI
    (underlying acquire immunity may not change dramatically within that
    timeframe), in theory, one could see a 50% reduction among under 5s
    that, if measured, was partially or mostly offset by increased
    mortality among children over 5 years old and adults.

-   For DHSs with a typical sample size, national-level mortality
    estimates are based on deaths during the 5 years before the survey.
    Thus, if national scale-up of interventions occurred from
    2006--2007, one would need to wait until 2012 to conduct the
    follow-up survey to have a 5-year period before the survey in which
    interventions were fully implemented. If the follow-up survey were
    conducted earlier (e.g., 2010), the mortality estimate would cover 2
    years of the scaling-up period (2006--2007) with relatively higher
    mortality and 3 years (2008--2010) of full implementation. This
    mixing of programmatic time periods would lead to an underestimation
    of impact. To avoid this problem somewhat, a survey could be
    conducted in 2010 with extra sample size so that mortality could be
    measured for the 3 years before the survey. Alternatively, results
    from multiple countries could be pooled (see section III.A) and
    deaths in the 3 years before the surveys could be analyzed. In
    either case, mortality results would not be available until the end
    of the malaria initiative, which may be too long to wait for donors
    who are anxious to see proof of the impact of their investments.
    This disadvantage justifies consideration of the use of
    complementary evaluation options to be used in at least some
    places---e.g., PBT or a demographic surveillance system (DSS) (see
    Box 1 and option 2 \[*DSS only*\]), which can produce results in a
    more timely fashion (e.g., monthly or annually). Note that countries
    with sizable populations not at risk for malaria would need an
    increased sample to have adequate precision for malarious areas.

III.C. *Option 2. In DSSs, measure trends in malaria-specific mortality,
coverage of malaria control interventions, and other factors that might
influence malaria mortality.*

*\[Abbreviation = "DSS only"\]*

III.C.1. [Description.]{.underline}

The mortality rate of deaths attributed to malaria is measured over a
period of time in a series of DSSs (Box 1). There are over 20 DSSs in
Africa[^10], most of which are part of the International Network for the
continuous Demographic Evaluation of Populations and Their Health in
developing countries (INDEPTH), a multi-national consortium of DSSs
\[INDEPTH, 2002; INDEPTH website; Email from Osman Sankoh, INDEPTH,
September 30, 2005\]. Among DSS currently operating in the middle part
of sub-Saharan Africa, where nearly all malaria deaths occur
\[Rowe-CHERG, 2005\], the median DSS population size under surveillance
is 40,000 (range: 5000--140,000). Causes of deaths are determined using
verbal autopsies (Box 1) \[Garenne, 1990; Anker, 1999\]. Coverage of
malaria control interventions among vulnerable populations, the
prevalence of other factors that might influence malaria mortality (see
section III.A), and anemia are concurrently measured with data from
population-based surveys and other valid data sources. Potentially, some
data could be collected from DSS censuses, as was done in Tanzania
\[Armstrong Schellenberg, 2001 (page 1243)\]. To evaluate the impact of
malaria control activities over the time period, trends in
malaria-specific mortality, anemia, coverage of malaria control
interventions, and other malaria-related factors are examined together.
If a statistically significant reduction of [\>]{.underline}50% in
malaria-specific mortality is found [and]{.underline} a statistically
significant reduction in anemia prevalence is found [and]{.underline}
coverage of malaria control interventions has increased to levels high
enough to affect malaria mortality (e.g., at least 70% of under 5s sleep
under ITNs) [and]{.underline} other factors that might influence malaria
mortality have not changed substantially (or have changed in a direction
that would lead to an increase malaria mortality, e.g., rainfall has
increased), then one can conclude that the goal of reducing malaria
mortality by 50% was achieved.

Regarding the timeliness of results and sample size requirements, for a
DSS in an area with high levels of malaria transmission and a population
of 60,000 (a medium-sized DSS), one could detect a 50% reduction in the
malaria mortality rate at the 95% confidence level after one year of
follow-up (see Annex 4).

III.C.2. [Estimated cost.]{.underline}

Estimates range from \$0.80 \[Byass, 2002\] up to \$10 \[personal
communication from L. Slutsker, CDC on September 19, 2005\] per person
under surveillance per year, although typical values are \$2 to \$3 per
person per year \[Rommelmann, 2005; personal communication from P.
Kachur, CDC on September 16, 2005; personal communication from O.
Sankoh, INDEPTH on October 8, 2005\], assuming 3--4 surveillance rounds
per year \[de Savigny, 2004\]. Start-up costs of about \$100,000 should
be added to the budget for the first year \[personal communication from
P. Bloland, CDC\].

III.C.3. [Advantages.]{.underline}

-   Verbal autopsy-based malaria mortality (i.e., malaria-attributed
    mortality), which is still a proxy for true malaria deaths, is the
    impact indicator. ACM is also available.

-   Data on the coverage of malaria control interventions could be
    collected at the individual level (although such data are probably
    not currently collected in most DSSs).

-   All ages included.

-   As a data source, DSSs could be used to monitor malaria mortality.

III.C.4. [Disadvantages.]{.underline}

-   Results are not based on representative samples of large "target"
    populations of interest (e.g., a national population); thus, results
    cannot be generalized to such target populations unless an
    assumption is made that the DSSs are approximately representative of
    the target populations. Such an assumption is unlikely to be valid
    because DSSs typically include populations that are selected for
    specific reasons (e.g., a rural population in an area with unusually
    high malaria transmission) and that are studied extensively \[Byass,
    2002\]. Indeed, intervention trials are conducted in some DSSs, and
    it is not clear how to interpret malaria-specific mortality (or even
    ACM) from populations exposed to investigational interventions. A
    recent analysis of malaria mortality in Africa found that the ACM
    rate among under-5s in DSSs in the 36 countries in the middle part
    of Africa (not every country had a DSS) from 1990--2000 (31.5 deaths
    per 1000 children per year) was considerably (28.2%) lower than the
    rate estimated for the entire pooled population of these 36
    countries (43.9 deaths per 1000 children per year) during the same
    time period \[Rowe-CHERG, 2005\].

-   In a related issue, large areas of sub-Saharan Africa have no DSSs
    (e.g., central Africa) \[Rudan, 2005\]. If this option were used, it
    might be necessary to create and maintain new DSSs, which would be
    costly.

-   Verbal autopsies may become less reliable for correctly identifying
    malaria as the cause of death in areas with increasing coverage of
    malaria control interventions. Specifically, if the interpretation
    of verbal autopsies remains unchanged, even if malaria were
    eradicated, verbal autopsy's high false-positive rate[^11] would
    show erroneously that malaria is present (i.e., the positive
    predictive value \[Box 1\] would probably decrease). As coverage
    increases and the malaria burden is reduced and the false-negatives
    disappear, one is left with "unpreventable" malaria-like deaths
    (e.g., false-positives caused by deaths from bacterial infections).
    Such results would underestimate the impact of malaria control
    efforts \[Rowe-CHERG, 2005\]. Theoretically, one way to avoid this
    problem would be to conduct serial validation studies as the
    coverage of malaria control interventions increases and then use the
    results to adjust the verbal autopsy results. Past attempts,
    however, at adjusting verbal autopsies have not been wholly
    successful and probably should not be done unless new methods are
    developed and validated \[Rowe IJE, 2005\].

-   The option only measures the impact of malaria control interventions
    on deaths directly attributable to malaria and would miss the impact
    on indirect and consequential malaria mortality. Such results would
    underestimate the impact of malaria prevention and control efforts.
    However, as ACM is also measured in DSSs, an analysis like that
    described in option 1 \[*ACM surveys*\] should also be performed.
    Such an analysis would provide estimates for a combination of
    direct, indirect, and consequential malaria mortality. Note that the
    use of two options simultaneously (option 2 \[*DSS only*\] and an
    analysis of ACM \[similar to option 1\]) will mean that two
    estimates of mortality reduction will be produced. This situation
    leads to a methodological question: How can a final estimate of
    malaria mortality be made when two or more estimates are
    available---i.e., "triangulation"? Answering this question is a high
    priority (Box 2).

-   Data on coverage of malaria interventions might only be collected at
    the population level; therefore the results would be subject to the
    limitations of an ecologic analysis (see disadvantages of option 1
    \[*ACM surveys*\]).

-   Regarding the use of DSS data to monitor malaria mortality, in
    addition to other disadvantages already mentioned, there is usually
    a time lag in the availability of data and results. In other words,
    the mortality results are not necessarily "timely."

III.D. *Option 3. Use results on malaria-specific mortality and ACM from
DSSs to interpret trends in ACM from population-based surveys,
controlling for coverage of malaria control interventions, and other
factors that might influence malaria mortality.*

*\[Abbreviation = "ACM surveys + DSS"\]*

III.D.1. [Description.]{.underline}

This option combines aspects of options 1 \[*ACM surveys*\] and 2 \[*DSS
only*\]. The under-5 ACM rate, the mortality rate of deaths attributed
to malaria, coverage of malaria control interventions, and the
prevalence of other factors that might influence malaria mortality are
measured over a period of time in DSSs. Populations under surveillance
in DSSs would only be considered if they were not the intervention arm
of a study designed to affect mortality. There are two ways to implement
option 3. First, a more local, qualitative approach is to consider only
the DSSs in a particular country. Over a period of time, one examines
increases in intervention coverage and decreases in the under-5 ACM rate
from representative surveys. At the same time, one examines trends in
intervention coverage and decreases in under-5 ACM in the DSS in the
country. If increases in intervention coverage are similar in the entire
country and in the DSS (a real possibility given that the programmatic
goal of the national malaria control program is to increase intervention
coverage to relatively high---i.e., 85% coverage or more) and decreases
in the under-5 ACM rate are similar in the entire country and in the
in-country DSSs, then one could make a general conclusion that the
decreases in malaria-specific mortality in the DSSs are likely to be
similar for the entire country. Or stated another way, the DSS results
add to the plausibility that malaria control efforts caused or at least
contributed to reductions in under-5 ACM and malaria-specific mortality
in the entire country.

A second approach uses data from DSSs inside and outside a given
country. The data would be used to construct a statistical model to
predict malaria mortality (outcome of the model) as a function of all
the other factors mentioned above (predictors of the model). A
random-effects model could be used to account for differences among
DSSs. Meanwhile, over the life of the PMI project in a given country or
group of countries, a series of population-based household surveys are
conducted to measure the "predictor" factors mentioned above. To
estimate the percent reduction in malaria mortality, values of the
predictors measured in the household surveys are entered into the
DSS-based model (see Figure 4 below). To account for the possibility
that the impact of malaria control interventions might vary depending on
the level of malaria transmission intensity, new DSSs might need to be
established in carefully chosen sites. Also, the validity of this option
might be improved if there were at least one DSS in the country. A
variant of this approach is to construct a model that prediction malaria
mortality given reductions in anemia prevalence.

Figure 4. Using DSS data to create a model for interpreting
country-specific data from representative surveys.

![](media/image1.wmf)

If option 3 is selected and multiple DSSs are used to develop a
multi-country prediction model, the evaluation should probably use the
"pooled multi-country" approach (see section II.C, above) because the
cost of new DSSs (if needed) would probably need to be shared among
multiple country-specific programs. Also, pooling populations might
"average out" differences among the populations of individual countries,
and therefore stabilize results and help prevent extreme predicted
values for individual countries. Some analyses of multiple DSSs has
begun (e.g., the Malaria Transmission Intensity and Mortality Burden
Across Africa \[MTIMBA\] project \[Thompson, 2002\]), however, results
have not been finalized.

To take advantage of the fact that this option is essentially a variant
of option 1 \[*ACM surveys*\], option 3 \[*ACM surveys + DSS*\] should
include an analysis using option 1. An RBM report recommends that both
ACM and malaria-specific mortality be used because: 1) there are
circumstances where changes in ACM can be reliably measured, but changes
in malaria mortality cannot: deaths counted in surveys sometimes cannot
accurately be attributed to malaria; 2) interventions against malaria
can have indirect as well as direct benefits, reducing deaths partly
attributable to other conditions, and it is highly desirable to quantify
these additional, indirect benefits; 3) where ACM is high, malaria is
sometimes a "competing risk", in which case fewer malaria deaths are
offset by more deaths from other causes, and when there is no measurable
change in ACM, we need to distinguish between two possible
explanations---the failure of malaria control and compensating
mortality; and 4) a number of RBM interventions will not be specific to
malaria, e.g. management of anemia in pregnancy \[RBM M&E Framework,
2000 (page 11)\].

In addition, to provide a source of information for monitoring impact,
DSS data could be used to construct a model to estimate
malaria-attributed mortality based on all the predictors described above
[except]{.underline} ACM (i.e., the model would be based primarily on
the coverage of malaria control interventions, similar to option 7
\[*Model only*\] \[see section III.H, below\]). Intervention coverage
data would presumably be collected every 1--2 years, depending on the
schedule of household surveys.

III.D.2. [Estimated cost.]{.underline}

The cost for option 3 includes the costs for option 1 (for
population-based mortality surveys) and option 2 (for DSSs).

III.D.3. [Advantages.]{.underline}

-   Verbal autopsy-based malaria mortality (i.e., malaria-attributed
    mortality), which is still a proxy for true malaria deaths, is the
    impact indicator.

-   The option provides results that are representative of, and thus can
    be generalized to, large populations of interest.

-   Impact can be estimated for all ages, although only a small amount
    of representative population-based data are collected for older
    children and adults.

-   Provides potential source for monitoring trends in
    malaria-attributed mortality.

III.D.4. [Disadvantages.]{.underline}

-   The option relies on a key assumption that relationships between
    predictors and malaria mortality in DSSs are the same as those in
    entire countries.

-   Verbal autopsies may become less reliable for correctly identifying
    malaria as the cause of death in areas with increasing coverage of
    malaria control interventions (see disadvantages for option 2 \[*DSS
    only*\]).

-   Option 3 \[*ACM surveys + DSS*\], by itself, only measures the
    impact of malaria control interventions on deaths directly
    attributable to malaria, which misses indirect and consequential
    malaria mortality and would probably underestimate program impact.
    As recommended above, an option 1 analysis of ACM should also be
    performed; although this combined approach will produce multiple
    estimates of mortality reduction that would have to be reconciled
    (see disadvantages for option 2).

-   Data on coverage of malaria interventions might only be collected at
    the population level; therefore the results would be subject to the
    limitations of an ecologic analysis (see disadvantages of option 1
    \[*ACM surveys*\]).

-   If new DSSs were needed, it would take time to establish them.

III.E. *Option 4. In representative populations, measure trends in
malaria-specific mortality (by adding verbal autopsies to
population-based surveys), coverage of malaria control interventions,
and other factors that might influence malaria mortality.*

*\[Abbreviation = "ACM surveys + VA"\]*

III.E.1. [Description.]{.underline}

The mortality rate of deaths attributed to malaria is measured
retrospectively for a defined time period by adding verbal autopsies to
representative, population-based surveys of mortality (e.g., DHS).
Although surveys such as DHS do not routinely include verbal autopsies,
it has been done twice in Bangladesh \[Baqui, 1998; Baqui, 2001\], twice
in Egypt (results not published, personal communication from Prof.
Robert Black, Johns Hopkins University, April 7, 2003), once in two
rural districts of Sierra Leone \[Amin, 1996\], and in numerous
countries as part of the WHO's World Health Surveys \[World Health
Survey website: <http://www.who.int/healthinfo/survey/en/index.html>,
accessed on September 14, 2005\]. Verbal autopsies could be performed
either by adding questions to a mortality survey (as done in the Sierra
Leone study and in the World Health Surveys), or by sending specialized
verbal autopsy teams after the survey to households in which the survey
identified a child death (as done in the Bangladesh and Egypt studies).
The former approach is probably preferable because it is much less
expensive, although the latter approach might be the only option if the
survey has already been done or verbal autopsy questions cannot be added
to the survey.

Modifying survey tools to include verbal autopsy questions might be
relatively simple, especially if the goal is to distinguish between
probable malaria deaths and all other causes. For example, one verbal
autopsy validation study of children 1--59 months old in Uganda found
that malaria deaths could be identified with the verbal autopsy
algorithm "convulsions or stopped responding to voice"; this algorithm
had a sensitivity of 55% and a specificity of 69% \[Anker, 1999 {page
46}\]. A study from Namibia found that the algorithm "fever and either
convulsions or loss of consciousness" had a sensitivity of 45% and a
specificity of 87% \[Mobley, 1996\]. More complex algorithms might have
better test characteristics. For example, a study of children 1--59
months old in Kenya found the following algorithm had a sensitivity of
71% and a specificity of 80%: (duration of illness \<4 days and no
measles) or (duration of illness 4--7 days and no measles and no
diarrhea) \[Quigley, 1996\]. Another approach is to acknowledge that is
will be difficult to identify malaria deaths reliably and thus to use
the following verbal autopsy categories: 1) obviously not a malaria
death (e.g., injuries), and 2) possibly a malaria-associated death; or
perhaps: 1) obviously not a malaria death, 2) possibly a direct malaria
death, and 3) possibly an indirect malaria death.[^12]

To implement this option, coverage of malaria control interventions
among vulnerable populations, the prevalence of other factors that might
influence malaria mortality (see section III.A), and anemia are
concurrently measured with data from population-based surveys and other
valid data sources. As with option 2 \[*DSS only*\], to evaluate the
impact of malaria control activities over the time period, trends in
malaria-specific mortality, anemia, coverage of malaria control
interventions, and other malaria-related factors are examined together.
If a statistically significant reduction of [\>]{.underline}50% in
malaria-specific mortality is found [and]{.underline} a statistically
significant reduction in anemia prevalence is found [and]{.underline}
coverage of malaria control interventions has increased to levels high
enough to affect malaria mortality [and]{.underline} other factors that
might influence malaria mortality have not changed substantially (or
have changed in a direction that would increase malaria mortality), then
one can conclude that the goal of reducing malaria mortality by 50% was
achieved. Note that a DHS with a typical sample size might not have
enough malaria deaths to estimate a stable malaria mortality rate
(especially if wide variation in underlying transmission rates in a
country). The goal of reducing malaria mortality by 50% could also be
demonstrated by showing a significant reduction in ACM [and]{.underline}
a statistically significant reduction of [\>]{.underline}50% in the
proportion of deaths attributed to malaria [and]{.underline} a
statistically significant reduction in anemia prevalence
[and]{.underline} a statistically significant increase in the coverage
of malaria control interventions to levels high enough to affect malaria
mortality [and]{.underline} no substantial change in other factors that
might influence malaria mortality (or have changed in a direction that
would increase malaria mortality). As with option 1, this option assumes
that malaria mortality reductions among older children and adults will
be half the reductions seen among children under 5 years old.

[]{.mark}

To take advantage of the fact that this option is essentially a variant
of option 1 \[*ACM surveys*\], option 4 \[*ACM surveys + VA*\] should
include an analysis using option 1 (see discussion in section III.D).
Results from option 4 estimate trends in direct malaria-attributable
mortality, and results from option 1 reflect trends in the sum of
direct, indirect, and consequential malaria-attributable mortality.

[]{.mark}

III.E.2. [Estimated cost.]{.underline}

The cost for option 4 includes the cost of the survey (see costs for
option 1) plus about \$100,000 to conduct the verbal autopsies if a
separate team performs the verbal autopsies (depending on local costs
for surveyors and transportation). If verbal autopsies are performed
during the course of the survey interview, as is done with the WHO's
World Health Surveys \[World Health Survey website\], then the only cost
would be that of including additional questions in the survey. (However,
as concerns exist about the actual method and questions used in the
World Health Surveys, this specific method is not recommended at
present.)

III.E.3. [Advantages.]{.underline}

-   Verbal autopsy-based malaria mortality (i.e., malaria-attributed
    mortality), which is still a proxy for true malaria deaths, is the
    impact indicator. ACM is also available.

-   The method provides results that are representative of, and thus can
    be generalized to, large populations of interest.

III.E.4. [Disadvantages.]{.underline}

-   Verbal autopsies may become less reliable for correctly identifying
    malaria as the cause of death in areas with increasing coverage of
    malaria control interventions (see disadvantages for option 2 \[*DSS
    only*\]). In addition, it is not clear how the quality and
    reliability of the verbal autopsy method is affected by performing
    verbal autopsy interviews long (e.g., years) after the death. Such a
    method may need validation. The verbal autopsy method used in the
    World Health Surveys have not yet been validated, and analyses of
    verbal autopsy data from the first round of surveys has not yet been
    completed; thus, this verbal autopsy methodology should be
    considered investigational until some results can be reviewed.

-   If currently existing surveys that measure mortality (e.g., DHS) are
    used, the method does not capture mortality trends among children
    [\>]{.underline}5 years old and adults, and relies on the rough
    assumption that mortality reductions among older age groups is equal
    to some proportional reduction among under 5s (see disadvantages of
    option 1 \[*ACM surveys*\], in section III.B.4, above).

-   Option 4 \[*ACM surveys + VA*\], by itself, only measures the impact
    of malaria control interventions on deaths directly attributable to
    malaria, which misses indirect and consequential malaria mortality
    and would probably underestimate program impact. As recommended
    above, an option 1 analysis should also be performed; although this
    combined approach will produce multiple estimates of mortality
    reduction that will have to be reconciled (see disadvantages for
    option 2 \[*DSS only*\]).

-   Data are collected at the population level, thus an ecologic
    analysis is required, which may be susceptible to bias (see
    disadvantages for option 1).

-   Verbal autopsies could not be added to the MICS unless the
    questionnaire was substantially modified.

-   This option may not provide results in a timely fashion if typical
    survey methods (e.g., typical DHS methods) are used. See the last
    disadvantage of option 1.

III.F. *Option 5. Use results on malaria mortality and ACM from DSSs to
interpret trends in ACM from population-based surveys, controlling for
coverage of malaria control interventions, and other factors that might
influence malaria mortality; also, measure trends in malaria mortality
with verbal autopsies added to population-based surveys.*

*\[Abbreviation = "DSS + ACM surveys + VA"\]*

III.F.1. [Description.]{.underline}

This option combines options 3 \[*ACM surveys + DSS*\] and 4 \[*ACM
surveys + VA*\]. Option 3 is used, as described above (see section
III.D); and in addition, trends in malaria-attributable mortality are
estimated from verbal autopsy results collected from population-based
mortality surveys, as described in option 4 (see section III.E).
Therefore, this method provides two complementary sets of results on
malaria-specific mortality. The primary justification for this option is
that it helps address key disadvantages of option 3 (the assumption that
relationships between predictors and malaria mortality in DSSs are the
same as those in entire countries) and option 4 (verbal autopsies used
with surveys have not been validated, and estimates of malaria mortality
reductions for older children and adults are based solely on
assumptions). Also, option 5 is not much more expensive than option 3,
assuming the verbal autopsy questions are added to mortality surveys and
separate verbal autopsy teams are not needed.

[]{.mark}

To take advantage of the fact that this option is essentially a variant
of option 1 \[*ACM surveys*\], option 5 should include an analysis using
option 1 (see discussion in section III.D). Results from option 4
estimate trends in direct malaria-attributable mortality, and results
from option 1 reflect trends in the sum of direct, indirect, and
consequential malaria-attributable mortality. Also, as with option 3
\[*ACM surveys + DSS*\], to provide information for monitoring impact,
DSS data could be used to construct a model to estimate
malaria-attributed mortality based on all predictors
[except]{.underline} ACM (i.e., based primarily on the coverage of
malaria interventions, similar to option 7 \[*Model only*\] \[see
section III.H, below\]). Intervention coverage data would presumably be
collected every 1--2 years, depending on the schedule of household
surveys.

[]{.mark}

III.F.2. [Estimated cost.]{.underline}

The cost for option 5 includes the costs for options 3 \[*ACM surveys +
DSS*\] and 4 \[*ACM surveys + VA*\] (see section III.D.2 and section
III.E.2 for details on these costs).

III.F.3. [Advantages.]{.underline}

-   Verbal autopsy-based malaria mortality (i.e., malaria-attributed
    mortality), which is still a proxy for true malaria deaths, is the
    impact indicator. ACM is also available.

-   The method provides results that are representative of, and thus can
    be generalized to, large populations of interest.

-   Impact can be estimated for all ages, although only a small amount
    of representative population-based data are collected for older
    children and adults.

-   Provides potential source for monitoring trends in
    malaria-attributed mortality.

III.F.4. [Disadvantages.]{.underline}

-   Verbal autopsies may become less reliable for correctly identifying
    malaria as the cause of death in areas with increasing coverage of
    malaria control interventions (see disadvantages for option 2 \[*DSS
    only*\]). In addition, it is not clear how the quality and
    reliability of the verbal autopsy method is affected by performing
    verbal autopsy interviews years after the death (see disadvantages
    for option 4).

-   Option 5, by itself, only measures the impact of malaria control
    interventions on deaths directly attributable to malaria, which
    misses indirect and consequential malaria mortality and would
    probably underestimate program impact. Therefore, an option 1
    analysis \[*ACM surveys*\] should also be performed; although this
    combined approach will produce multiple estimates of mortality
    reduction that will have to be reconciled (see disadvantages for
    option 2).

-   Data on coverage of malaria interventions might only be collected at
    the population level; therefore the results would be subject to the
    limitations of an ecologic analysis (see disadvantages of option 1).

III.G. *Option 6. In community-based sentinel sites, measure trends in
malaria-specific mortality, ACM, coverage of malaria control
interventions, and other factors that might influence malaria
mortality.*

*\[Abbreviation = "Sentinel sites"\]*

III.G.1. [Description.]{.underline}

The mortality rate of deaths attributed to malaria is measured over a
period of time in multiple community-based sentinel sites.
Community-based sentinel sites already exist in some countries: for
example, Zambia has 10 sentinel districts \[Zambia Ministry of Health,
2005\]. There are at least two approaches for using sentinel sites to
evaluate programmatic impact on mortality. First, a probability sample
of enumeration areas is selected (e.g., 24 "clusters" are selected, each
with about 20,000 people), and the populations in these areas are
closely monitored with village recorders and annual censuses for births,
deaths, and migrations. Causes of death are determined using verbal
autopsies, and other population characteristics (e.g., coverage of
malaria control interventions) could be measured at the individual
level. This approach is called sample vital registration \[Setel, 2005\]
(also, sample vital registration with verbal autopsy \[SAVVY\]) and is
conceptually similar to establishing numerous small DSSs. The second
approach involves choosing a smaller number of medium-sized populations
(e.g., 4 districts per country, each district with a population of
30,000), however all demographic data are collected by annual surveys
(not censuses). Causes of death are determined using verbal autopsies. A
key difference between the two approaches is that the latter approach is
less "invasive" and therefore less likely to influence behaviors in the
sentinel sites. In both approaches, data would also be collected from
health facilities on malaria morbidity, and studies would be conducted
to measure malaria transmission.

Coverage of malaria control interventions among vulnerable populations,
the prevalence of other factors that might influence malaria mortality
(see section III.A), and anemia are concurrently measured from the
sentinel sites, as well as with data from population-based surveys and
other valid data sources. To evaluate the impact of malaria control
activities over the time period, trends in malaria-specific mortality,
anemia, coverage of malaria control interventions, and other
malaria-related factors are examined together. If a statistically
significant reduction of [\>]{.underline}50% in malaria-specific
mortality is found [and]{.underline} a statistically significant
reduction in anemia prevalence is found [and]{.underline} coverage of
malaria control interventions has increased to levels high enough to
affect malaria mortality (e.g., at least 70% of under 5s sleep under
ITNs) [and]{.underline} other factors that might influence malaria
mortality have not changed substantially (or have changed in a direction
that would lead to an increase malaria mortality, e.g., rainfall has
increased), then one can conclude that the goal of reducing malaria
mortality by 50% was achieved.

The justification for this option is that it addresses a major
disadvantage of option 2 \[*DSS only*\]---the lack of representativeness
of DSSs. However, as program managers would know where the sentinel
sites are located, it is possible that areas with sentinel sites could
receive relatively greater support for malaria control activities, thus
introducing the very bias that option 6 was designed to avoid. To detect
such extra attention, the coverage of malaria control interventions in
the sentinel sites could be compared coverage levels measured from
nationally representative surveys.

III.G.2. [Estimated cost.]{.underline}

For the first approach (SAVVY) with 24 clusters (total population =
20,000 people per cluster x 24 clusters = 480,000) would cost about
\$3.7 million for the first 5 years \[personal communication from Philip
Setel, University of North Carolina, on November 4, 2005\]. In year 1,
the first 4 clusters would be established (cost = \$600,000); in years
2--5, progressively more clusters would be established (year 2 cost =
\$500,000; year 3 cost = \$800,000; year 4 cost = \$1 million; year 5
cost = \$800,000). Ongoing maintenance cost = \$800,000 per year. For
each cluster that is newly created, it would take 12--18 months before
high-quality data would be consistently reported. Of note, SAVVY is not
intended to be funded by a single disease-specific program; it could
collect information relevant to many health programs (as well as other
sectors) and thus its cost is intended to be shared.

III.G.3. [Advantages.]{.underline}

-   Results might be representative of, and thus can be generalized to,
    large populations of interest.

-   Verbal autopsy-based malaria mortality (i.e., malaria-attributed
    mortality), which is still a proxy for true malaria deaths, is the
    impact indicator. ACM is also available.

-   Data on the coverage of malaria control interventions could be
    collected at the individual level.

-   All ages included.

-   Results from sentinel sites could be used to monitor
    malaria-attributed mortality.

III.G.4. [Disadvantages.]{.underline}

-   As mentioned above, in the long term, sentinel surveillance sites
    might receive extra attention and no longer be representative of the
    entire country or group of countries.

-   Verbal autopsies may become less reliable for correctly identifying
    malaria as the cause of death in areas with increasing coverage of
    malaria control interventions (see disadvantages for option 2 \[*DSS
    only*\]).

-   It might take up to several years for a sentinel site system to
    become operational.

-   Option 6, by itself, only measures the impact of malaria control
    interventions on deaths directly attributable to malaria, which
    misses indirect and consequential malaria mortality and would
    probably underestimate program impact. Therefore, an analysis of ACM
    trends (similar to option 1 \[*ACM surveys*\]) should also be
    performed; although this combined approach will produce multiple
    estimates of mortality reduction that will have to be reconciled
    (see disadvantages for option 2).

-   Data on coverage of malaria interventions might only be collected at
    the population level; therefore the results would be subject to the
    limitations of an ecologic analysis (see disadvantages of option 1).

III.H. *Option 7. Estimate malaria mortality trends from a mathematical
model based on coverage of malaria control interventions and other
factors that might influence malaria mortality.*

*\[Abbreviation = "Model only"\]*

III.H.1. [Description.]{.underline}

This option, which has been alluded to in previous sections, involves
measuring coverage of malaria control interventions (with
population-based surveys) and other factors that might influence malaria
mortality, and entering the data into a mathematical model. The outcome
of the model is malaria mortality. Such a model has already been
developed \[Jones, 2003; UNICEF, 2005\]; although it was developed by
synthesizing results from trials that generally tested individual
interventions (not combinations of interventions), and it has not been
validated. New models could be developed using DSS results in places
that are scaling up (as suggested in option 3) and past studies of
intervention efficacy (e.g., ITN trials). As this method does not
actually involve measuring mortality (all-cause or malaria-specific), to
help avoid misinterpretation, it might be best to state the results of
this option are mortality *predictions*, not mortality *estimates*.

Although this option is not as robust as measuring intervention coverage
[plus]{.underline} mortality, it has several justifications. First, as
malaria intervention studies have demonstrated large decreases in
mortality, one can be relatively sure of the "end points" of the model
(i.e., low intervention coverage = no mortality reduction, and high
intervention coverage = mortality reduction target achieved \[see Annex
3\]). Thus, even if the validity of the middle part of the model is
unknown, a model could probably produce reasonably valid results
regarding whether substantial progress was being made and whether the
mortality reduction target has been achieved. Second, the option is
practical: it is inexpensive, rapid, and applicable at the sub-national
level. Third, it can be used for monitoring program impact (although it
must be stressed that it is likely to produce fairly crude estimates).
Fourth, it can be used in situations where mortality surveys cannot be
conducted, and when malaria control programs complete scale-up late
(i.e., after 2008). This last point is particularly important because
even large-scale mortality surveys have to measure mortality for the
3--5 years before the survey; so if the end-of-project survey is just
after the end of 2010, then scale-up must be complete by the end of
2008. If a program encounters difficulties with scaling up, then the
mortality estimates of the end-of-project survey would reflect mortality
for a mix of partial and complete scale up, which would probably
underestimate program impact.

A variant of option 7 is to use a model that predicts changes in ACM as
a function of changes in the coverage of malaria control interventions.
If fact, several such models have been developed that account for
changes in the coverage of malaria control interventions and other child
survival interventions \[Jones, 2003; UNICEF, 2005; Morel, 2005\]. Use
of the Marginal Budgeting for Bottlenecks model by UNICEF demonstrated
that this approach is acceptable to at least one international donor
\[UNICEF CIDA report, 2005\]. Concerns do exist, however, about the
validity of these models.

III.H.2. [Estimated cost.]{.underline}

The cost for option 7 is fairly low, as it requires only monitoring data
on intervention coverage and a statistical model for estimating malaria
mortality.

III.H.3. [Advantages.]{.underline}

-   Inexpensive

-   Rapid

-   Applicable at the sub-national level. In fact, if survey data were
    not available, very rough mortality estimates at the district-level
    might be made by applying the model to administrative estimates of
    ITN coverage (e.g., based on number of ITNs distributed).

-   Can be used for monitoring program impact

-   Can be used in situations where mortality surveys cannot be
    conducted

-   Can be used when malaria control programs complete scale-up late

-   The outcome is malaria-attributed mortality

-   All ages included

III.H.4. [Disadvantages.]{.underline}

-   Depends on an unvalidated mathematical model that could provide
    biased results. As such, it might lack persuasiveness for
    policy-makers.

III.J. *Option 8. Measure trends in malaria-specific mortality (by
analyzing data from a country's health information system \[HIS\] or
data abstracted from hospital records), coverage of malaria control
interventions, and other factors that might influence malaria
mortality.*

*\[Abbreviation = "HIS"\]*

III.J.1. [Description.]{.underline}

Trends in the malaria-specific mortality rate are measured over a period
of time by analyzing data on malaria deaths from a country's HIS or data
abstracted from hospital records. HISs typically include deaths that
occur and are recorded in public health facilities and hospitals, and
thus miss most deaths, which occur outside such facilities. For example,
Benin's HIS reported 564 malaria deaths among children under 5 years old
in 2000 \[<http://www.afro.who.int/malaria/country-profile/benin.pdf>,
accessed on February 24, 2005\], and an epidemiologic model estimated
that the number of deaths in under 5s in Benin for the year 2000 that
were directly attributable to malaria was 11,318 (precision estimate:
9669--12,967) \[Rowe-CHERG, 2005\]. Thus the sensitivity (Box 1) of the
HIS was estimated as 5% (564/11,318). An exception might be in several
countries in southern Africa, such as South Africa, which have good
access to health care throughout the country and a fairly complete and
good-quality reporting system for malaria \[Snow, 1999\]; these
countries however have a relatively low malaria burden and would
probably not be part of the PMI. If, however, the sensitivity (Box 1)
and positive predictive value (Box 1) of the data are constant, then the
[trend]{.underline} from the HIS data is a valid estimate of the true
malaria mortality trend at the community level. If the HIS is not
functioning well enough (e.g., completeness of reporting is low) to
provide data from hospitals, then staff can be sent to hospitals to
abstract data from inpatient records on deaths due to malaria (or deaths
due to severe anemia among children, which are probably caused by
malaria). These hospitals can be considered as sentinel surveillance
sites. In addition to following trends in malaria mortality, one could
also follow trends in the proportion of all deaths that are caused by
malaria \[World Malaria Report 2005 (page 21)\]. This measure might be
less susceptible to bias caused by changes in hospital utilization,
although it is sensitive to the incidence of other diseases, e.g., a
measles epidemic. Coverage of malaria control interventions among
vulnerable populations, the prevalence of other factors that might
influence malaria mortality (see section III.A), factors such as
hospital utilization that might affect hospital-based estimates of
malaria-specific mortality, and anemia are concurrently measured with
data from population-based surveys and other valid data sources. As with
option 2 \[*DSS only*\], to evaluate the impact of malaria control
activities over the time period, trends in malaria-specific mortality,
anemia, coverage of malaria control interventions, other malaria-related
factors, and indicators of hospital utilization (e.g., number of
admissions per month for non-malaria conditions) are examined together.
If a statistically significant reduction of [\>]{.underline}50% in
malaria-specific mortality is found [and]{.underline} a statistically
significant reduction in anemia prevalence is found [and]{.underline}
coverage of malaria control interventions has increased to levels high
enough to affect malaria mortality [and]{.underline} other factors that
might influence malaria mortality have not changed substantially (or
have changed in a direction that would increase malaria mortality),
[and]{.underline} hospital utilization is constant or increasing, then
one can conclude that the goal of reducing malaria mortality by 50% was
achieved.

III.J.2. [Estimated cost.]{.underline}

Theoretically, collecting HIS data would have no cost for the PMI, as
such data are routinely collected by the government. However, in
practice, ensuring that data are of usable quality might be costly (see
disadvantages below) and take time. If staff must be sent to hospitals
to abstract data from hospital records, the cost might be about \$70,000
over a 5-year period (depending on local costs for staff and
transportation).

III.J.3. [Advantages.]{.underline}

-   Malaria mortality is the impact indicator.

-   All ages are included.

-   HISs are a potentially important data source for monitoring malaria
    mortality (i.e., the data are continuously collected and could be
    made available quickly), although the data have several important
    limitations (see disadvantages below).

III.J.4. [Disadvantages.]{.underline}

-   As HIS data are health facility-based, the data are unlikely to be
    complete (e.g., the estimated 5% sensitivity of Benin's HIS for
    child malaria deaths in the year 2000) or representative of the
    country. Specifically, deaths at health facilities occur among
    people who seek care at health facilities; such populations are
    likely to be wealthier than the population as a whole.

-   The validity of trends from HIS data depends on constant sensitivity
    and positive predictive value, as well as proper transmission of
    data from health facilities to the national level government agency
    that collates the HIS data. For sensitivity to remain constant,
    utilization of health facilities and hospitals and the willingness
    to keep patients hospitalized until death[^13] needs to be constant.
    Such an assumption does not seem valid because numerous factors
    influence care seeking practices and care seeking could change over
    time. Changes in care-seeking is especially pertinent in the context
    of massive efforts to scale-up malaria interventions because as
    services improve, health facility utilization may increase
    \[Arifeen, 2004; de Savigny, 2004 (pages 226-7)\]. Regarding
    changing sensitivity, it would be difficult or impossible to monitor
    sensitivity (i.e., it would be difficult or impossible to know
    whether sensitivity was constant). For positive predictive value to
    remain constant, a standard definition for a malaria death needs to
    be applied consistently; however, at present, it is unknown whether
    such diagnostic practices occur (and it seems unlikely that they
    do). Efforts could be made to improve and standardize such
    practices, and then to monitor diagnostic practices over time. Such
    efforts, however, may be costly. Regarding the proper transmission
    of data from health facilities to the national level, it is
    certainly possible that national level data could suffer from
    transcription errors, data loss, and data fabrication. Efforts could
    be made to verify HIS data by routinely checking national level data
    against the source data in health facilities, but such efforts may
    be costly (although it would be less expensive if data were only
    collected from a limited number of sentinel surveillance sites). For
    HIS data, for example, one recommendation is that completeness of
    reporting should be evaluated regularly (e.g., annually) \[World
    Malaria Report 2005\].

-   Other factors could affect trends. For example, if transfusion
    services improve over time, inpatient malaria deaths could decrease
    regardless of the malaria mortality rate in the community because
    transfusion is a highly efficacious treatment for severe anemia
    \[Schellenberg, 2004\].

-   Regarding malaria morbidity, when malaria transmission is reduced
    (e.g., by increasing coverage of malaria interventions),
    hospital-based estimates of the incidence of clinical malaria could
    either decrease or paradoxically increase. In the short term (e.g.,
    1--2 years), studies have found that when the coverage of malaria
    interventions (in particular, ITNs) increases, the incidence of
    uncomplicated malarial episodes and severe malaria decreases
    \[Lengeler, 2004\]. However, other studies have found that hospital
    admission rates for malaria do not steadily decrease with decreasing
    malaria transmission \[Schellenberg, 2004; Snow, 1997\]. For
    example, a study of 5 areas with different levels of malaria
    transmission found that as transmission decreased from high levels
    (parasite prevalence of 74--83%) to moderate levels (parasite
    prevalence of 37--49%), hospitalizations for malaria among children
    0--9 years old [increased]{.underline} from about 17 admissions per
    1000 children per year to about 26 admissions per 1000 children per
    year; as transmission further decreased to very low levels (parasite
    prevalence of 2%), hospitalizations decreased dramatically to 3.9
    admissions per 1000 children per year \[Snow, 1997\]. Snow and
    colleagues explained that this phenomenon might be due to a
    combination of malaria transmission intensity, the acquisition of
    passive immunity, and the development of active immunity early in
    life.[^14]

-   Option 8 \[*HIS*\], by itself, only measures the impact of malaria
    control interventions on deaths directly attributable to malaria,
    which misses indirect and consequential malaria mortality and would
    probably underestimate program impact. Therefore, an analysis of the
    trends of ACM (similar to option 1 \[*ACM surveys*\]) could also be
    performed; although this combined approach will produce multiple
    estimates of mortality reduction that will have to be reconciled
    (see disadvantages for option 2 \[*DSS only*\]).

# IV. Dissemination of results

A general dissemination plan should be developed. It is recommended
that, at a minimum, such a plan should include: 1) publishing the
evaluation plan via the Internet and an article in a peer-reviewed
scientific journal; 2) communicating results in a timely fashion to
Ministries of Health, other implementing partners, and donors; 3)
coordinating, whenever possible, with other malaria initiatives to avoid
contradictions; 4) creating an Internet website with data from surveys
and other evaluation activities; and 5) publishing final results in
peer-reviewed scientific articles. A philosophy of complete transparency
should be adopted---even if the evaluation reveals potentially
embarrassing results. Without complete transparency, the entire
evaluation is at risk for being dismissed as biased and invalid.

**V. Discussion and recommendations**

Clearly, there is no simple, valid way to evaluate the impact of malaria
control efforts in countries with a high malaria burden in Africa. All
options described in this document have important shortcomings. However,
despite these shortcomings, given the substantial resources that the PMI
and other malaria initiatives are mobilizing and the high priority
placed on showing real public health impact, it is equally clear that a
rigorous evaluation plan must be adopted and implemented. In addition,
as malaria initiatives have similar donors and are likely to work in the
same countries, it would be efficient and desirable for all initiatives
to use the same evaluation plan. This approach agrees with the "Three
Ones" principle that has been used by AIDS programs (one national AIDS
framework, one national AIDS authority, and one system for monitoring
and evaluation) \[UNAIDS, 2005\].

A number of articles and reports have been published on evaluating the
impact of malaria control programs. Most have addressed the topic in
more conceptual terms \[Molineaux, 1997; Watt & Dye, 2000; RBM M&E
Framework, 2000; de Savigny & Binka, 2004; World Malaria Report 2005\];
and to a large extent, this document has simply refined and
operationalized the concepts already described. Among these articles and
reports, there is considerable agreement. Most authors: 1) emphasize the
importance of linking data from different sources through meta-analysis
and statistical modeling; 2) express similar cautions regarding the use
of health facility-based data; and 3) recommend a fairly consistent set
of impact indicators. A typical list for areas with highly endemic
malaria is: ACM among under 5s, anemia prevalence among under 5s,
parasite prevalence from community surveys during transmission season,
and malaria-attributed deaths in sentinel DSSs \[World Malaria Report
2005 (Table 9)\].

A different approach was taken in one article, which modeled mortality
among four sub-populations[^15] as a function of malaria's epidemiologic
characteristics and estimates of program effectiveness \[Cohen, 1988\].
However, this method requires data inputs that are probably difficult to
obtain reliably. Also, it was created before most of the currently
recommended malaria control interventions had been developed (or before
their efficacy had been established) and does not account for the fact
that intervention coverage can generally be measured locally for
populations of interest and usefully incorporated into estimates of
program impact.

## Limitations

The analysis presented in this document has several important
limitations. First, numerous assumptions are required, many of which are
not supported by high-quality data. To avoid gross errors, sensitivity
analyses have been performed whenever possible. Second, it is difficult
to estimate costs accurately. Not only are cost estimates for specific
evaluation options sometimes merely rough approximations, a further
complication is taking into account the possibility that costs will be
shared with other malaria initiatives (as in the case of intervention
coverage surveys) or with initiatives for other diseases (as in the case
of DHSs). Thus, costs to any one initiative (e.g., to PMI) are likely to
be lower than the totals at the bottom of Annex 5.

Third, as the focus of this document is on methods for determining
whether a specific goal (50% reduction) has been achieved, little has
been said about how to measure smaller changes. Of course all efforts
should be made to achieve the PMI's goal; however, it should be
recognized that a smaller reduction might still be an important public
health accomplishment if unexpected things occur (e.g., political
instability in a PMI country, the rapid spread of resistance to
insecticides or antimalarials, or scale-up simply goes more slowly than
anticipated). Thus, as a final evaluation plan is developed, some
thought should be given to how the plan could be adjusted if smaller
reductions are anticipated. For example, as plans are implemented, if it
appears that intervention coverage targets will be missed, the design of
end-of-project surveys could be modified[^16]. Fourth, little has been
said about malaria risk mapping. For certain countries (e.g., with
malarious and non-malarious areas), reliable maps of malaria risk will
be quite important for designing country-specific evaluation plans.
Currently, maps from the Mapping Malaria Risk in Africa (MARA) project
are assumed to be the primary source for malaria risk, but these maps
have important limitations; and these limitations could affect the
evaluation. Clearly, better maps are needed (Box 2). Fifth, this
analysis has not mentioned innovative interventions, such as
intermittent preventive treatment for infants \[Egan, 2005\]. If new
interventions are recommended in time for the PMI, then the evaluation
plan might need adjustment. At a minimum, surveys for measuring
intervention coverage should be modified to include the new
interventions. Sixth, certain important monitoring issues have not been
addressed at all, such as antimalarial resistance, insecticide
resistance, and adverse drug reactions. Finally, other methods for
measuring malaria burden, such as measuring the prevalence of the
hemoglobin S gene \[Molineaux, 1988\], are not mentioned because too
little is known about their practical application to evaluating
programs.

## Recommendations

Two issues complicate the development of simple recommendations. First,
the degree to which evaluations will be coordinated among countries has
not yet been decided. For example, RBM could sponsor one evaluation of
all malarious countries in Africa. A broad evaluation might be more
valid and less expensive (per country) than separate evaluations of
smaller areas because country-specific evaluation plans would be
harmonized, continent-wide evaluation options (e.g., option 6
\[*Sentinel sites*\]) could be used that complement survey-based
evaluations, and country-specific surveys could have smaller sample
sizes (but combined at the continental level). Such a combined dataset
would be very large and might allow for analyses such as mortality in
the 1 or 2 years before end-of-project surveys. Countries with less data
would probably benefit from epidemiologically similar countries with
more data. Also, a regional plan might stimulate more resources, public
health action, and cooperation, as policy-makers recognize that all
countries succeed or fail together. However, as such a broad evaluation
has numerous implications for the evaluation design, not knowing whether
it is desired complicates the task of making recommendations. The second
issue that different countries have different constraints with regards
to funding and which types of surveys can be conducted at a particular
time. For example, although it might be best for the evaluation to
conduct a DHS in a particular year, a country might not permit it for
political reasons (but might agree to conduct an MIS).

To address these complications, an algorithm was developed (Figure 5) to
help choose among the evaluation options presented in this document. To
use the algorithm, begin with the first decision point in the upper left
corner (multi-country versus individual country evaluation), and follow
the path indicated by the arrows by answering questions at subsequent
decision points until an evaluation option is reached.

Figure 4. An algorithm for selecting evaluation options in areas with a
high malaria burden

![](media/image1.wmf)

[Footnotes:]{.underline}

ACM = All-cause mortality, DSS = demographic surveillance system, VA =
verbal autopsy.

a Although option 2 is less likely to be representative (compared with
option 6), it seems reasonable to at least examine option 2 results.

b Option 1 (or an "option 1-like" analysis) should also be used to
provide complementary results on ACM. Such results reflect the impact of
interventions on direct [plus]{.underline} indirect [plus]{.underline}
consequential malaria mortality.

c Although smaller sample sizes reduce the likelihood that results for
individual countries would be statistically significant.

d Depends on whether enough DSSs have been established, adequate
scale-up is occurring in DSSs, the data can be shared, and a reasonable
model has been developed.

NB 1. Option 1 = ACM surveys; option 2 = DSS; option 3 = ACM surveys +
DSS; option 4 = ACM surveys + VA; option 5 = DSS + ACM surveys + VA;
option 6 = sentinel sites; option 7 = model; option 8 = health facility
data.

NB 2. Although not currently recommended as a primary method for
evaluating impact in most countries, trends in malaria mortality from
health information systems can be examined (option 8) in all countries.
Trends should be interpreted with great caution because of concerns
about completeness of reporting, quality of diagnostics and cause of
death information, and low and potentially varying proportion of deaths
that occur in health facilities.

+-----------------------------------+-----------------------------------+
| # Box 3. R                        | -   Option 5 = DSS + ACM          |
| eminder of the evaluation options |     surveys + VA                  |
|                                   |                                   |
| -   Option 1 = ACM surveys        | -   Option 6 = Sentinel sites     |
|                                   |                                   |
| -   Option 2 = DSS only           | -   Option 7 = Model only         |
|                                   |                                   |
| -   Option 3 = ACM surveys + DSS  | -   Option 8 = HIS                |
|                                   |                                   |
| -   Option 4 = ACM surveys + VA   |                                   |
+-----------------------------------+-----------------------------------+

Several explanatory points should be mentioned about the algorithm.
First, for multi-country evaluations, the algorithm always recommends at
least two options: option 2, perhaps option 6, and one of the options on
the right-hand side of the figure (i.e., options 1, 3, 4, 5, or 7).
Second, as mentioned in footnote "b" of Figure 4, options that estimate
impact in terms of malaria-specific mortality (e.g., option 5) recommend
that another option (option 1, or a similar analysis of ACM) also be
performed, as the data will already be available and as a second set of
results in terms of ACM complements the results on malaria-specific
mortality (see section III for the justification). Third, the algorithm
does not include option 8. The reason is that the disadvantages (see
section III.J.4, and references such as de Savigny & Binka, 2004) raise
important questions about its validity. This is not to say the data are
without use. Under the right conditions, they can be helpful for
measuring the burden on the health system, detecting epidemics in areas
with unstable transmission, estimating case-fatality rates, and
monitoring trends in the incidence of severe malaria. The issue for this
document, however, is whether such data can be used for evaluating the
impact of malaria initiatives on malaria mortality; and at present, the
answer is that it is not clear. For this reason, studying the question
is listed as a priority operations research topic (Box 2). Perhaps in
the future, as more research is done to understand the data and refine
the way it is analyzed and interpreted, option 8 could be considered.
Ultimately, when vital registration systems achieve high coverage and
accuracy with regards to identifying causes of death, all African
countries could use this option.

In addition to the algorithm, a more detailed generic country-specific
evaluation plan was developed for the PMI (Annex 5). The plan assumes a
multi-country evaluation, mortality surveys with verbal autopsy, the
availability of a DSS-based model, and no system of sentinel sites
(option 6). The reasons for not using sentinel sites are: 1) given the
high cost, this option should be reserved for a continent-wide or even
global level evaluation (i.e., the option seems less suited for
evaluating a single malaria initiative); and 2) as mentioned above (see
section III.G.1), sentinel sites could attract disproportionately extra
support for malaria control activities, such that they are no longer
representative of large populations of interest. Given these assumptions
and the algorithm (Figure 4), it is recommended that PMI-funded
countries use evaluation option 5 (plus complementary estimates provided
by options 1 and 2). The primary benefits of option 5 are that it
produces results that are nationally representative, the impact
indicator is malaria-attributed mortality, and it uses multiple data
sources such that each data source makes up for the limitations in other
data sources. For example, verbal autopsies from DSSs are probably of
better quality than verbal autopsies linked to surveys, but survey
results are nationally representative while DSSs are not.

Several explanatory points should be mentioned about the plan. First, as
it is a generic plan, it must be tailored to a particular country.
Second, it has a budget item for one DSS for the country, but the
purpose of the DSS is not specifically to provide results for the
country. Rather, the multi-country evaluation will likely require
several (perhaps up to four) new DSSs whose results will contribute to
the evaluation of all PMI-funded countries. Therefore, if PMI supports
up to 15 countries, not every one will have a new DSS (for this reason,
one set of budget estimates in Annex 5 excludes DSS costs). Also, the
selection of the DSS requires input from country-level teams as well as
whoever is managing the multi-country evaluation. Third, as shown in the
top row of Annex 5, the plan requires that full scale up is needed for 3
years before the end-of-project mortality survey. The reason is that, to
keep the mortality survey affordable, it has an adequate sample size to
measure mortality for the 3 years before the survey (instead of the
usual 5 years). If scale up is not complete until 1 or 2 years before
the survey, the survey will measure a mix programmatic time periods
(partial scale up and full scale up), which would lead to an
underestimation of impact (see the last disadvantage for option 1, in
section III.B.4). Fourth, regarding PBT (a method for monitoring
all-cause early childhood mortality, see row 6 of Annex 5, and option
1), it should only be used when certain conditions are met: 1) high
proportion of births occur at health facilities, 2) high fertility rate,
and 3) low use of contraceptives. If these conditions are not met, PBT
can be omitted from the plan (for this reason, one set of budget
estimates in Annex 5 excludes PBT costs). Fifth, with regards to
monitoring impact before the end-of-project mortality survey, estimates
of impact can be made with a prediction model (as described in option 7,
see section III.H) based on estimates of coverage of malaria control
interventions from annual household surveys or perhaps based on data
from DSSs (if malaria control interventions are being scaled up in the
DSSs). Sixth, the plan assumes that the process starts in 2006 and
continues until 2010 (with the end-of-project mortality survey in 2011).
If programs start later than 2006, then dates of evaluation activities
must be pushed back. There is a potential problem, however, if the "end"
date of 2010 is held fixed because of RBM targets. In this case, the
evaluation plan would need to be condensed somehow, which might be quite
problematic. For countries starting as late as 2008, obtaining
good-quality evaluation results will be difficult. Perhaps the only
option is to measure coverage (and demonstrate dramatic improvements in
coverage) and use a model (option 7) to estimate impact on mortality.
For the PMI, however, this issue is unlikely to be a problem because
mortality reductions are only expected after 3 years of full
implementation---even if the 3-year period ends after 2010. Finally, the
cost of the plan is intended to be shared among initiatives working in a
particular country. Therefore, the cost to any one initiative (e.g., to
PMI) is should be lower than the totals at the bottom of Annex 5.

The plan in Annex 5 also has 3 elements that are not, strictly speaking,
necessary for the final evaluation of impact on mortality, but which
serve other important needs. First, intervention coverage surveys are
recommended every 1--2 years. The reason is that scaling up is likely to
be a challenging endeavor, and annual coverage estimates that are
nationally representative can improve planning and help stimulate
action. Also, towards the end of the 5-year project, if coverage
estimates are not approaching target levels, then a decision might be
made to postpone or cancel a large, expensive mortality survey that
would not have otherwise been conducted. Second, the plan recommends
several operations research projects, including validation studies of
the verbal autopsy method; the evaluation of a quality improvement
system to support health facility activities; and the evaluation of the
validity, cost, and utility of health facility-based data. These
projects are relatively inexpensive, will help inform the evaluation
process, and strengthen programmatic activities. Third, the plan
supports capacity-building activities related to conducting surveys. In
fact, a secondary goal of the PMI should be that by the end of the
project, country teams are able to conduct high-quality surveys (e.g.,
the MIS) on their own.

The analysis in this document permits several other practical
recommendations to be made. First, if a multi-country analysis is to be
conducted, a dedicated team is needed. For example, in addition to
partners that would actually conduct surveys, manage DSSs, etc., such an
evaluation team should have a minimum of an epidemiologist, a data
manager, and a part-time statistical consultant. Furthermore, as
mentioned in section IV (Dissemination of results), the work of this
team should be thoroughly transparent and well documented. Second, the
PMI should re-consider the programmatic targets of 85% coverage for
malaria control interventions. The reasons are that it will be difficult
to achieve (e.g., the highest published estimate of ITN coverage was
82.5% \[Lindblade, 2004\] and that was in a research setting) and that a
lower target (e.g., 70%) will probably be adequate for achieving the
impact target of 50% reduction in malaria mortality. It would be
unfortunate if the PMI could not claim "complete" success because,
despite reaching impact targets, intervention coverage targets were
missed. Third, in the future, quantitative goals of large-scale public
health initiatives should be stated in measurable terms that are
appropriate for the health issue to be addressed. For diseases like
malaria, for which disease-specific mortality is essentially impossible
to measure in the places where the burden is greatest, policy-makers
could acknowledge the problem and in simple terms describe alternative
targets that are measurable.

Finally, although it would be ideal to have an evaluation plan that,
once developed, requires no additional modifications, it is recommended
that some flexibility be maintained and that operations research be
conducted to fill knowledge gaps that currently exist for the evaluation
methods (Box 2). We should learn as we go, and we should not hesitate to
modify the evaluation plan based on new knowledge. Measuring change in
the burden of malaria in places with a high burden is difficult to do in
the best of circumstances; but as the PMI will be implemented in
countries that generally have a weak infrastructure, such measurements
become even more challenging.

# VI. References

Adult Morbidity and Mortality Project. The Policy Implications of Adult
Morbidity and Mortality End of Phase 1 Report. August 1997. Pages 125
and 134. Internet address:
<http://www.ncl.ac.uk/ammp/ammp_rep/ammp_rpt.pdf>. Accessed February 3,
2003.

Alonso P, Lindsay S, Armstrong J, et al. The effect of
insecticide-treated bednets on mortality of Gambian children. Lancet
1991;337:1499--1502.

Amin R. Immunization coverage and child mortality in two rural districts
of Sierra Leone. Social Science and Medicine 1996;42:1599--1604.

Anker M, Black RE, Coldham C, et al. A standard verbal autopsy method
for investigation causes of death in infants and children. Geneva: World
Health Organization, 1999:1--78.

Arifeen SE, Blum LS, Hoque DME, Chowdury EK, Khan R, Black RE, et al.
Integrated Management of Childhood Illness (IMCI) in Bangladesh: early
findings from a cluster-randomized study. Lancet 2004;
364**:**1595--1602.

Armstrong Schellenberg JRM, Abdulla S, Nathan R, Mukasa O, Marchant TJ,
Kikumbih N, Mushi AK, Mponda H, Minja H, Mshinda H, Tanner M, Lengeler
C. Effect of large-scale social marketing of insecticide-treated nets on
child survival in rural Tanzania. Lancet 2001;357:1241--47.

Attaran A. An immeasurable crisis? A criticism of the Millennium
Development Goals and why they cannot be measured. PLoS Medicine
2005;2:e318.

Baqui AH, Black RE, Arifeen SE, Hill K, Mitra SN, Sabir AA. Causes of
childhood deaths in Bangladesh: results of a nationwide verbal autopsy
study. Bulletin of the WHO 1998;76:161--71.

Baqui AH, Sabir AA, Begum N, Arifeen SE, Mitra SN, Black RE. Causes of
childhood deaths in Bangladesh: an update. Acta Paediatr
2001;90:682--90.

Binka FN, Kubaje A, Adjuik M,Williams LA, Lengeler C, Maude GH, et al.
Impact of permethrin impregnated bednets on child mortality in
Kassena-Nankana district, Ghana: a randomized controlled trial. Tropical
Medicine & International Health 1996;1:147--54.

Bradley DJ. Morbidity and mortality at Pare-Taveta, Kenya and Tanzania,
1954--66: the effects of a period of malaria control. In: *Disease and
mortality in Sub-Saharan Africa*, 1991, Eds. Richard G. Feachem, Dean T.
Jamison. The World Bank. Oxford University Press. (Mkomazi Valley
results, Figure 17-8. Note that the source for the figure was: Draper
CC. Studies of malaria in man in Africa. M.D. thesis, Oxford University,
1962).

Bryce J, Boschi-Pinto C, Shibuya K, Black RE, and the WHO Child Health
Epidemiology Reference Group. WHO estimates of the causes of death in
children. Lancet 2005;365:1147--52.

Byass P, Berhane Y, Emmelin A, et al. The role of demographic
surveillance systems (DSS) in assessing the health of communities: an
example from rural Ethiopia. Public Health 2002;116:145--50.

Chandramohan D, Maude GH, Rodrigues LC, Hayes, R. Verbal autopsies for
adult deaths: their development and validation in a multicentre study.
Tropical Medicine & International Health 1998;3:436--46.

Chongsuphajaisiddhi T (1988). Malaria in paediatric practice. In
*Malaria, Principles and Practice of Malariology*. (ed. Wernsdorfer WH,
McGregor Sir I) New York: Churchill Livingstone, 889--902.

Cohen JE. Estimating the effects of successful malaria control
programmes on mortality. Pop Bull U.N. 1988;25:6--26.

D\'Alessandro U, Olaleye B, McGuire W, Langerock P, Bennett S, Aikins
MK, et al.Mortality and morbidity from malaria in Gambian children after
introduction of an impregnated bednet programme. Lancet
1995;345:479--83.

De Savigny D, Binka F. Monitoring future impact on malaria burden in
sub-Saharan Africa. Am J Trop Med Hyg 2004;71:224--231.

De Savigny D, Mwageni E, Masanja H, Mbuya C, Kasale H, Reid G, Mkilindi
Y, Mkikima S, Kamuzora P. Health systems interventions and child
mortality in Rufiji, Tanzania. INDEPTH Network Annual Scientific
Conference, Durban, South Africa, May 23-27, 2005.

Egan A, Crawley J, Schellenberg D, on behalf of the IPTi Consortium.
Intermittent preventive treatment for malaria control in infants: moving
towards evidence-based policy and public health action. Tropical
Medicine & International Health 2005;10:815--7.

Garenne M, Fontaine O. Chapter 8: Assessing probable causes of death
using a standard questionnaire: a study in rural Senegal. In: Vallin J,
S DS, Palloni A, eds. *Measurement and analysis of mortality - new
approaches*. Oxford: Clarendon Press, 1990:123--142.

Gimnig JE, Kolczak MS, Hightower AW, Vulule JM, Schoute E, Kamau L,
Phillips-Howard PA, Ter Kuile FO, Nahlen BL, Hawley WA. Effect of
permethrin-treated bed nets on the spatial distribution of malaria
vectors in western Kenya. Am J Trop Med Hyg 2003;68 Suppl(4):115--20.

Global Fund to Fight AIDS, Tuberculosis and Malaria. Monitoring and
evaluation toolkit: HIV/AIDS, tuberculosis and malaria. Geneva: Global
Fund to Fight AIDS, Tuberculosis and Malaria, 2004
(<http://www.theglobalfund.org/pdf/guidelines/pp_me_toolkit_en.pdf>
accessed November 1, 2005).

Global Fund to Fight AIDS, Tuberculosis, and Malaria.
<http://www.theglobalfund.org/en/> accessed November 2, 2005.

Guyatt HL, Snow RW. Impact of malaria during pregnancy on low birth
weight in sub-Saharan Africa. Clinical Microbiology Reviews
2004;17:760--767.

Habicht J-P, Victora CG, Vaughan JP. Evaluation designs for adequacy,
plausibility and probability of public health programme performance and
impact. Int J Epidemiol 1999;28:10--18.

HawleyWA, Phillips-Howard PA, ter Kuile FO, Terlouw DJ, Vulule JM, Ombok
M, et al. Community-wide effects of permethrin-treated bed nets on child
mortality and malaria morbidity in western Kenya. Am J Trop Med Hyg
2003;68 Suppl (4):121--7.

Hay SI, Guerra CA, Tatem AJ, Atkinson PM, Snow RW. Urbanization, malaria
transmission and disease burden in Africa. Nature Microbiology Review
2005;3:84--93.

Number range CHAPTERHill AG, Aguirre A. Childhood mortality estimates
using the preceding birth technique: some applications and extensions.
Population Studies 1990;44:317--40.

Hill AG, David PH. Monitoring changes in child mortality: new methods
for use in developing countries. Health Policy and Planning
1988;3:214--26.

Hill K, Pande R,Mahy M, Jones G. Trends in child mortality in the
developing world: 1960--1996. New York: UNICEF, 1999.

INDEPTH Monograph Series: Demographic Surveillance Systems for Assessing
Populations and their Health in Developing countries, Volume 1:
Population, Health and Survival in INDEPTH Sites. Ottawa: IDRC/CRDI,
2002.

INDEPTH. INDEPTH Demographic Surveillance Sites. Internet address:
<http://indepth-network.org/dss_site_profiles/dss_sites.htm> accessed
September 19, 2005.

Joint United Nations Programme on HIV/AIDS (UNAIDS). The "Three Ones" in
action: where we are and where we go from here. Geneva, Switzerland:
UNAIDS, 2005. Document UNAIDS/05.08E.

Jones G, Steketee RW, Black RE, et al. How many child deaths can we
prevent this year? The Lancet 2003;362:65--71.

Kidane G, Morrow RH. Teaching mothers to provide home treatment of
malaria in Tigray, Ethiopia: a randomized trial. Lancet 2000;
356:550--555.

Korenromp EL, Arnold F, Williams BG, Nahlen BL, Snow RW. Monitoring
trends in under-5 mortality rates through national birth history
surveys. Int J Epidemiol 2004:33;1293--1301.

Korenromp EL, Armstrong-Schellenberg JRM, Williams BG, Nahlen BL, Snow
RW. Impact of malaria control on childhood anemia in Africa -- a
quantitative review. Tropical Medicine & International Health
2004;9:1050--65.

Lengeler C. Insecticide-treated bed nets and curtains for preventing
malaria. The Cochrane Database of Systematic Reviews 2004, Issue 2. Art.
No.: CD000363.pub2.

Lindblade KA, Eisele TP, Gimnig JE, Alaii JA, Odhiambo F, ter Kuile FO,
et al. Sustainability of reductions in malaria transmission and infant
mortality in Western Kenya with use of insecticide-treated bednets: 4 to
6 years of follow-up. JAMA 2004;291:2571--80.

Lusinde R, Masanja H, Mwageni Eleuther, Craig M, de Savigny D. Obtaining
rainfall estimates from satellite remote sensing: a step-by-step
technical guide for INDEPTH DSS sites. INDEPTH Technical Notes Series
Number 1. INDEPTH Network, August 2003. Available at:
[http://indepth-network.net](http://indepth-network.net/).

Mathers CD, Ma Fat D, Inoue M, Rao C, Lopez AD. Counting the dead and
what they died from: an assessment of the global status of cause of
death data. Bulletin of WHO 2005;83:171--7.

McGregor IA. Measles and child mortality in the Gambia. West Afr Med J
1964;13:251--7.

Millennium Development Goals. Internet address:
<http://www.developmentgoals.org/> accessed November 2, 2005.

Mobley CC, Boerma JT, Titus S, Lohrke B, Shangula K, Black RE.
Validation study of a verbal autopsy method for causes of childhood
mortality in Namibia. Journal of Tropical Pediatrics 1996; 42:365--369.

Molineaux L. The impact of parasitic diseases and their control, with an
emphasis on malaria and Africa. In: *Health policy, social policy and
mortality prospects*. Vallin J and Lopez AD, eds. Liege: Ordina
Editions, 1985.

Molineaux L. Chapter 35: The epidemiology of human malaria as an
explanation of its distribution, including some implications for its
control. In: Wernsdorfer WH, McGregor I, eds. *Malaria. Principles and
practice of malariology*. Volume 2. Edinburgh: Churchill Livingstone,
1988: 978--980.

Molineaux L. Malaria and mortality: some epidemiological considerations.
Annals of Tropical Medicine & Parasitology 1997;91:811--25.

Molineaux L, Gramiccia G. *The Garki Project: Research on the
Epidemiology and Control of Malaria in the Sudan Savanna of West
Africa*. Geneva: World Health Organization, 1980.

Morel CM, Lauer JA, Evans DB. Cost effectiveness analysis of strategies
to combat malaria in developing countries. BMJ 2005;331:1299.

Morgenstern H. Uses of ecologic analysis in epidemiologic research.
American Journal of Public Health 1982;72:1336--44.

Murray CJL, Lopez AD. Mortality by cause for eight regions of the world:
Global Burden of Disease Study. Lancet 1997;349:1269--1276.

Murray CJL, Lopez AD, Mathers CD, Stein C. The Global Burden of Disease
2000 project: aims, methods and data sources. Global Programme on
Evidence for Health Policy Discussion Paper No. 36. World Health
Organization, November 2001 (revised).

Nabarro DN, Taylor EM. The "Roll Back Malaria" campaign. Science
1998;280:2067--68.

Nevill CG, Some ES, Mung\'ala VO, Mutemi W, New L, Marsh K,et al.
Insecticide-treated bednets reduce mortality and severe morbidity from
malaria among children on the Kenyan coast. Tropical Medicine &
International Health 1996;1:139--46.

ORC Macro, MEASURE DHS+. Demographic and Health Surveys (DHS). ORC
Macro, Calverton, MD, USA. Internet address:
[http://www.measuredhs.com](http://www.measuredhs.com/) accessed
September 13, 2005.

Program for Appropriate Technology in Health (PATH). The Malaria Control
and Evaluation Partnership (MACEPA).
<http://www.path.org/projects/malaria_control_partnership.php> accessed
November 2, 2005.

Payne D, Grab B, Fontain RE, Hempel JHG. Impact of control measures on
malaria transmission and general mortality. Bulletin of the WHO
1976;54:369--377.

Phillips-Howard PA, Nahlen BL, Kolczak MS, Hightower AW, ter Kuile FO,
Alaii JA, et al. Efficacy of permethrin-treated bed nets in the
prevention of mortality in young children in an area of high perennial
malaria transmission in western Kenya. Am J Trop Med Hyg 2003;68
Suppl(4):23--9.

Quigley MA, Armstrong Schellenberg JRM, Snow RW. Algorithms for verbal
autopsies: a validation study in Kenyan children. Bulletin of the WHO
1996;74:147--54.

Roll Back Malaria website.
<http://rbm.who.int/cgi-bin/rbm/rbmportal/custom/rbm/home.do> accessed
November 2, 2005.

RBM-MERG Anemia Task Force. Minutes of meeting on October 27--28, 2003.
<http://rbm.who.int/partnership/wg/wg_monitoring/docs/MERG_Anaemia_tfm1_minutes.doc>
accessed November 2, 2005.

RBM-MERG. Malaria Indicator Survey. <http://rbm.who.int/merg> accessed
November 3, 2005.

Rommelmann V, Setel PW, Hemed Y, Angeles G, Mponezya H, Whiting D,
Boerma T. Cost and results of information systems for health and poverty
indicators in the United Republic of Tanzania. Bulletin of the WHO
2005;83:569--77.

Rosner R. Fundamentals of biostatistics. Fourth edition. Belmont:
Duxbury Press, 1995:182--3.

Rowe AK, Rowe SY, Snow RW, et al. For the Child Health Epidemiology
Reference Group. Estimates of the burden of mortality directly
attributable to malaria for children under 5 years of age in Africa for
the year 2000. Final report. 2005.

Rowe AK, Rowe SY, Snow RW, Korenromp EL, Armstrong Schellenberg JRM,
Stein C, Nahlen B, Bryce J, Black RE, Steketee RW. The burden of malaria
mortality among African children in the year 2000. International Journal
of Epidemiology (in press)

Rowe AK. Should verbal autopsy results for malaria be adjusted to
improve validity? International Journal of Epidemiology
2005;34:712--713.

Rudan I, Lawn J, Cousens S, Rowe AK, Boschi-Pinto C, Tomašković L,
Mendoza W, Lanata CF, Roca-Feltrer A, Carneiro I, Schellenberg JA,
Polašek O, Weber M, Bryce J, Morris SS, Black RE, Campbell H. Gaps in
policy-relevant information on burden of disease in children: a
systematic review. Lancet 2005;365:2031--2040.

Schellenberg D, Menendez C, Aponte J, Guinovart C, Mshinda H, Tanner M,
Alonso P. The changing epidemiology of malaria in Ifakara Town, southern
Tanzania. Tropical Medicine & International Health 2004;9:68--76.

Setel PW, Sankoh O, Chalapati R, Velkoff VA, Mathers C, Gonghuan Y,
Hemed Y, Jha P, Lopez AD. Sample registration of vital events with
verbal autopsy: a renewed commitment to measuring and monitoring vital
statistics. Bulletin of the WHO 2005;83:611--7.

Snow RW, Omumbo JA, Lowe B, Molyneux CS, Obiero JO, Palmer A, Weber MW,
Pinder M, Nahlen B, Obonyo C, Newbold C, Gupta S, Marsh K. Relation
between severe malaria morbidity in children and level of *Plasmodium
falciparum* transmission in Africa. Lancet 1997;349:1650--54.

Snow RW, Craig M, Deichmann U, Marsh K. Estimating mortality, morbidity,
and disability due to malaria among Africa\'s non-pregnant population.
Bulletin of the WHO 1999;77:624--40.

Snow RW, Craig MH, Newton CRJC, Steketee RW. The public health burden of
Plasmodium falciparum malaria in Africa: Deriving the number. Working
Paper No. 11, Disease Control Priorities Project. Bethesda, Maryland:
Fogarty International Center, National Institutes of Health. August
2003. Also available on the worldwide web at
[www.fic.nigh.gov/dcpp](http://www.fic.nigh.gov/dcpp).

Snow RW, Korenromp EL, Gouws E. Pediatric mortality in Africa:
*Plasmodium falciparum* malaria as a cause or risk? Am J Trop Med Hyg
2004;71(Suppl 2):16--24.

Steketee RW, Nahlen BL, Parise ME, Menendez C. The burden of malaria in
pregnancy in malaria-endemic areas. Am J Trop Med Hyg 2001;64(Suppl
1,2):28--35.

Tanser F, Hosegood V, Benzler J, Solarsh G. New approaches to spatially
analyse primary health care usage patterns in rural South Africa.
Tropical Medicine & International Health 2001;6:826--38.

Thompson R, Owusu-Agyei S, Diallo D, Abdulla S. Malaria Transmission
Intensity and Mortality Burden Across Africa (MTIMBA). Abstract
presented at the Third Multilateral Initiative on Malaria (MIM)
Pan-African Malaria Conference, Arusha, Tanzania, November 17, 2002.

The United Nations Children's Fund (UNICEF). The Multiple Indicator
Cluster Survey (MICS2) Manual. Internet address:
<http://www.childinfo.org/MICS/finman/M2finM.htm> accessed on August 26,
2003.

UNICEF. Final report to CIDA. Accelerating Child Survival and
Development. A results-based approach in high under-5 mortality areas.
May 2005.

UNICEF. UNICEF Statistics. Internet address:
<http://www.childinfo.org/areas/childmortality/> accessed on September
13, 2005.

United States Department of Health and Human Services. Technical
Appendix from Vital statistics of United States 1995: Mortality.
National Center for Health Statistics, Centers for Disease Control and
Prevention, Public Health Service, U.S. Department of Health and Human
Services. Hyattsville, Maryland: April 1999. Page 22. Available at
Internet address: <http://www.cdc.gov/nchs/data/statab/techap95.pdf>

Victora CG, Habicht J-P, Bryce J. Evidence-based public health: moving
beyond randomized trials. Am J Public Health 2004;94:400--405.

Watt C, Dye C (2000 July) Indicators to measure the impact of malaria
control. Report number WHO/CDS/RBM/2000.22. Geneva: World Health
Organization. Available at Internet address:

http:⁄⁄whqlibdoc.who.int/hq/2000/WHO_CDS_RBM_2000.22.pdf accessed Sept
17, 2005

World Bank Malaria Booster Program.
<http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTHEALTHNUTRITIONANDPOPULATION/EXTMALARIA/0,,contentMDK:20461038~menuPK:377615~pagePK:148956~piPK:216618~theSitePK:377598,00.html>
accessed November 2, 2005.

World Bank. Rolling Back Malaria: The World Bank global strategy and
booster program. Washington, DC: The World Bank, 2005. Available at
Internet address:
<http://siteresources.worldbank.org/INTMALARIA/Resources/377501-1114188195065/WBMalaria-GlobalStrategyandBoosterProgram-June2005.pdf>

World Health Organization. International statistical classification of
diseases and related health problems. Tenth revision. Volume 2:
Instruction manual. Geneva: World Health Organization, 1993.

World Health Organization Regional Office for Africa. Roll Back Malaria
initiative in the WHO African Region: monitoring and evaluation
guidelines. Brazzaville: World Health Organization Regional Office for
Africa, 2004.

World Health Organization and UNICEF. World Malaria Report 2005. Geneva:
World Health Organization. Report no. WHO/HTM/MAL/2005.1102. Available
at Internet address: <http://rbm.who.int/wmr2005/>

World Health Surveys. Internet address:
<http://www.who.int/healthinfo/survey/en/index.html> accessed September
14, 2005.

Zambia Ministry of Health. National Malaria Prevention and Control
Monitoring and Evaluation Plan 2006--2011. Draft version from December
2005. National Malaria Control Centre, Zambia Ministry of Health. Pages
21--23.

**Annex 1. A statistical method for determining whether a minimum
coverage level for a malaria control intervention has been achieved**

## 

If a programmatic goal is to achieve a minimum coverage level (MCL) for
a malaria control intervention (e.g., MCL = 70%), then the statistical
method for determining whether the level has been achieved must account
for the sampling error of the surveys used to measure intervention
coverage. Furthermore, if the most basic question that the evaluation is
to answer is in terms of success or failure (i.e., did the program
achieve the MCL---yes or no?), then a determination of how much the
program might have exceeded the MCL is of secondary importance (although
clearly this question could be answered). Therefore, the recommended
method is to construct a 1-sided 95% confidence interval (CI) for the
coverage estimate. If the lower bound of the 1-sided 95% CI is
[\>]{.underline} MCL, then one can conclude with high confidence (95%
confidence) that the MCL was reached and the program succeeded. If the
lower bound of the 1-sided 95% CI is \< MCL, then for the purposes of
this analysis, one can conclude that the program failed.

To estimate the lower bound of a 1-sided 95% CI, the following formula
can be used (adapted from Rosner \[Rosner, 1995\]):

*p* -- (z~1-α~ x SE\[*p*\])

where *p* is the coverage that was measured in the survey, SE\[*p*\] is
the standard error of *p* (also measured in the survey), z~1-α~ is the
critical point that defines (1 -- α)% of the area under the curve of a
normal distribution, and α is the probability of making a type I error
(i.e., concluding the program was a success when, in truth, the program
failed). For a 1-sided 95% CI, α is 0.05, and z~1-α~ is approximately
1.645. The upper bound of a 1-sided 95% CI is always 1.0 (i.e., 100%
coverage). The interpretation of a 1-sided 95% CI is that one can say
with high confidence (95% confidence) that the true prevalence of the
characteristic in the population is within the CI.

For example, if the MCL is 70%, and an end-of-project survey (a simple
random survey of 100 children) finds that 80% of children slept under an
ITN the previous night, then *p* = 0.80, and SE\[*p*\] = 0.04 (i.e.,
square-root of {0.8 x \[1 -- 0.8\] / 100}). Thus, the lower bound of the
1-sided 95% CI is 0.8 -- (1.645 x 0.04), or 0.734. The 1-sided 95% CI
is: 73.4% -- 100%. One can conclude with high confidence that the true
prevalence of ITN use was between 73.4% and 100%, and therefore the
program achieved its goal.

[Note 1.]{.underline} With this method, if a survey measures a coverage
level that exactly equals the MCL, then one would have to conclude the
project failed. The reason is that the lower bound of the 1-sided 95% CI
will be lower than *p* (unless, theoretically, one performed a census
and SE\[*p*\] = 0).

[Note 2.]{.underline} With this method, one can derive an expression for
the minimum sample size (n) of an end-of-project survey for measuring
the coverage of malaria control interventions. If *p* is the expected
coverage value that one will measure in the survey (expressed as a
decimal such as 0.70 \[not 70%\]), and if one assumes that the survey
will use complex sampling (and thus the design effect, DE, is likely to
be greater than 1), then

n = DE x {\[1.645^2^ x *p* x (1 -- *p*)\] / (*p* -- MCL)^2^}.

The following table shows sample sizes for various values of MCL and
*p*. The DE is assumed to be 2 (a typical assumption for household
surveys).

  -------------------- --------------------- ---------------- ------------
  Minimum coverage     Intervention coverage Design effect,   Sample size,
  level of the         level expected to be  DE               n
  intervention to      measured in the                        
  conclude "success",  survey, *p*                            
  MCL                                                         

  0.70                 0.80                  2                87

  0.70                 0.75                  2                406

  0.70                 0.72                  2                2,728

  0.85                 0.95                  2                26

  0.85                 0.90                  2                195

  0.85                 0.87                  2                1,530
  -------------------- --------------------- ---------------- ------------

[Note 3.]{.underline} In the situation in which the lower bound of the
95% CI was slightly less than the MCL, one could reasonably be flexible
(or generous) in drawing conclusions about the success of the program.
For example, consider the situation in which the MCL is 70%, and a
survey measures a coverage of 76% and a 1-sided 95% CI of 68% -- 100%.
Strictly speaking, the program failed because 68% is less than the MCL
of 70%. However, one could reasonably argue that the results are close
enough to be acceptable. To quantify the phrase "close enough," one
could calculate the confidence level at which the lower bound of the
1-sided CI is exactly 70%. To continue with the above example, imagine
the SE\[*p*\] = 0.0427. One could calculate the critical value, z~1-α~,
with the expression z~1-α~ = (*p* -- MCL) / SE\[*p*\] = (0.76 -- 0.70) /
0.0427 = 1.405. One could then use a table of critical values of a
normal distribution and find that a value of 1.405 corresponds to an α
of 0.08. As the confidence level of a CI equals 1 -- α, one could
conclude that with 92% confidence (i.e., a "relatively high" or
"moderately high" level of confidence), the program achieved its goal
and succeeded. The justifications for this secondary approach (for it
should never be the primary analytic approach) are: 1) the use of the
95% confidence level is somewhat arbitrary and rooted in the culture of
statisticians and the public health research community; and 2) unlike a
study evaluating a new intervention (e.g., a new vaccine) that is to be
used in the future, a type I error in a program evaluation is probably
not going to have any serious public health impact (although it might
upset donors, which could affect future funding decisions about public
health interventions).

One should be careful and thoroughly transparent about using alternate
levels of confidence. To avoid misleading the audience, one should
[always]{.underline} present results with the 95% confidence level.
Depending on the circumstances, one could then cautiously add other
interpretations (with a very clear explanation) using another level of
confidence. As a somewhat arbitrary guideline, the lowest level of
confidence that should be reported in a secondary analysis is 90%. This
value is based on the fact that while 90% CIs have been in the
scientific literature occasionally, confidence levels less than 90% have
not been used (to the best of our knowledge).

**Annex 2. Estimates of the reduction in all-cause mortality for
children 0--59 months old if malaria mortality is reduced by 50%**

## Assumptions about transmission intensity

As the PMI and other large malaria initiatives generally target
countries with a high malaria burden, the analysis is only for the 26
"high malaria burden" countries in sub-Saharan Africa (defined as
countries in which [\>]{.underline} 70% of the population lives in areas
with high-intensity transmission \[defined as areas with a Mapping
Malaria Risk in Africa (MARA) climate suitability index
[\>]{.underline}0.75 (Snow, 1999)\]: Benin, Burkina Faso, Cameroon,
Central African Republic, Chad, Congo, Ivory Coast, Democratic Republic
of Congo, Equatorial Guinea, Gabon, The Gambia, Ghana, Guinea,
Guinea-Bissau, Liberia, Malawi, Mali, Mozambique, Niger, Nigeria,
Senegal, Sierra Leone, Tanzania, Togo, Uganda, and Zambia) \[Annex 12 of
Rowe-CHERG, 2005\]. Altogether in these 26 countries, in the year 2000,
3.0 million deaths occurred among children under 5 years old, and
672,000 deaths (or 22.4%) were directly attributable to malaria
(precision estimate based on summing the lower and upper limits of
country-specific 95% confidence intervals \[CIs\]: 575,000--770,000, or
19.1--25.6%) \[Annex 12 of Rowe-CHERG, 2005\]. Note that this assumption
is not meant to imply that these 26 countries are the only countries
that have a high malaria burden. For example, countries such as Kenya,
in which a smaller proportion (about one-fifth) of the population lives
in areas with high-intensity transmission, still have many malaria
deaths (indeed, more deaths than small countries in which the entire
population lives in areas with high-intensity transmission).

## Assumptions about the burden of indirect malaria mortality

The burden of indirect malaria mortality, the burden could be as high as
that of direct malaria mortality. Three types of evidence support this
statement. First, in the 1950s--1970s, studies in Kenya, Tanzania, and
Nigeria found that when malaria transmission was drastically reduced
with insecticide residual spraying, which would have little impact on
non-malaria causes of child mortality, all-cause child mortality dropped
by 40--50% \[Figure 17-8 in Feachem and Jamison, 1991; Payne, 1976;
Molineaux and Gramiccia,1980\]. If the direct effects of malaria caused
about 22% of deaths (see assumption 1), then the indirect effects of
malaria would have caused the remaining 18--28% of deaths. In other
words, the direct malaria mortality and indirect malaria mortality are
roughly similar. Second, a trial of ITNs in The Gambia from the early
1990s found that for children 1--9 years old, ITNs reduced ACM by 2.2
deaths per 1000 children per year (from Table 2, the ACM rate was
10.32/1000/year \[235 deaths/22,692 person-years\] in untreated villages
and 8.14/1000/year \[162 deaths/19,907 person-years\] in treated
villages), direct malaria mortality by 0.5 deaths/1000/year (from Table
4, malaria mortality was 3.7/1000/year in untreated villages and
3.2/1000/year in treated villages), and non-malaria-non-trauma mortality
by 1.2 deaths/1000/year (from Table 4, non-malaria-non-trauma mortality
was 4.2/1000/year in untreated villages and 3.0/1000/year in treated
villages) \[D'Alessandro, 1995\]. This study had a relatively high
proportion of deaths with unknown causes (verbal autopsy questionnaires
were obtained from parents for only 79% of deaths, and a cause of death
might not have been determined for every verbal autopsy \[number of
verbal autopsies with an undetermined cause of death was not reported\])
and the verbal autopsy method may lose positive predictive value for
identifying malaria deaths when used in populations exposed to malaria
control interventions, so it is difficult to make a firm estimate of the
burden of indirect malaria mortality relative to the burden of direct
malaria mortality. However, these results do suggest that the burden of
indirect malaria mortality might be as large (or greater) as the burden
of direct malaria mortality. Third, a recent meta-analysis of the
relationship between the prevalence of *Plasmodium falciparum*
parasitemia and all-cause under-5-year mortality in African demographic
surveillance systems found that as parasite prevalence decreased from
95% to 0%, the predicted all-cause child mortality dropped from 44 to 8
deaths per 1000 children per year---a reduction of 82% \[i.e.,
(44--8)/44 = 0.82\] \[Snow et al. Am J Trop Med Hyg 2004;71(Suppl
2):16--24}\]. If about 22% of deaths results from the direct effect of
malaria (see assumption 1), then the burden of indirect malaria
mortality would be substantially greater than the burden of direct
malaria mortality. The authors, however, recognize that the associations
found in their statistical model might not be causal: the results might
"merely reflect the association of both malaria infection risk and
mortality with other underlying factors (such as poverty) that determine
both." In addition, the model did not include other factors that
influence child mortality (immunization coverage, nutritional status,
etc), and thus the magnitude of the association between parasite
prevalence and mortality is likely to have been overestimated.

Results of an ITN trial in The Gambia from the late 1980s \[Alonso,
1991\] and a trial from Ghana \[Binka, 1996\] suggest that the burden of
indirect malaria mortality may be more modest. The study from The Gambia
found that when ITNs and chemoprophylaxis were used to prevent malaria
in children, the all-cause under-5-year mortality rate decreased by 42%
\[Alonso, 1991\]. In the study setting, the percentage of deaths
directly attributable to malaria was 25% (75/296, based on results in
Table II for pre-intervention PHC villages, pre-intervention non-PHC
villages, and post-intervention non-PHC villages); so, a minimum
estimate of the burden of indirect malaria mortality (assuming ITNs plus
chemoprophylaxis prevented all direct malaria mortality, an assumption
that leads to a conservative estimate of indirect malaria mortality) is
that indirect malaria mortality might be about two-thirds
(\[42--25\]/25, or 0.68) of direct malaria mortality. The Ghana trial
found that ITNs reduced direct malaria mortality by 2.1 deaths per 1000
per year (from Table 2, malaria mortality was 9.9/1000/year in control
clusters and 7.8/1000/year in treated clusters), and non-malaria
mortality by 1.0/1000/year (from Table 2, non-malaria mortality was
13.1/1000/year in control clusters and 12.1/1000/year in treated
clusters) \[Binka, 1996\]. This study had a moderate proportion of
deaths with unknown causes (e.g., \[62+72\]/\[396+461\], or 16%) and the
verbal autopsy method may lose positive predictive value for identifying
malaria deaths when used in populations exposed to malaria control
interventions, so it is difficult to make a firm estimate of the burden
of indirect malaria mortality relative to the burden of direct malaria
mortality; however, these results do suggest that the burden of indirect
malaria mortality might be about half the burden of direct malaria
mortality.

A fundamental problem with using the above evidence to estimate how much
indirect malaria mortality would be prevented if the coverage of malaria
control interventions were increased to a high level is that indirect
malaria deaths can be prevented in at least two ways: prevent the
malaria and prevent the other co-morbid disease(s). As there has been
renewed interest in preventing child deaths from all causes, it might be
prudent to assume that public health activities that impact non-malaria
causes of death will prevent some of the indirect malaria deaths.
Therefore, to be conservative, it is assumed that the burden of indirect
malaria mortality is half of the burden of malaria mortality (i.e., for
every 2 direct malaria deaths, there is 1 indirect malaria death). For
the sensitivity analysis, the lower limit of the burden of indirect
malaria mortality is arbitrarily chosen as one-quarter the burden of
direct malaria mortality (i.e., for every 4 direct malaria deaths, there
is 1 indirect malaria death), and the upper limit of the burden of
indirect malaria mortality is equal to the burden of direct malaria
mortality (i.e., for every 1 direct malaria death, there is 1 indirect
malaria death).

## Assumptions about the impact of malaria control interventions on non-malaria mortality

Assume that malaria control interventions have no impact on non-malaria
mortality. In fact, malaria interventions that strengthen health systems
might reduce non-malaria mortality. For example, improving drug supplies
and health worker performance for the case management of malaria might
lead to improvements in the care-seeking for and management of other
childhood illnesses. For simplicity, however, no impact will be assumed.

*Results*

If at baseline 22.4% of all under-5-year deaths are direct malaria
deaths and 11.2% (half of 22.4%) are indirect malaria deaths, then 33.6%
(22.4% + 11.2%) of all deaths are malaria deaths. If interventions are
implemented and malaria mortality (direct and indirect) is reduced by
50% and non-malaria mortality is unchanged, then in the
post-intervention period, relative to the baseline period, ACM would
decrease by 16.8 percentage points (0.5 x 33.6%). The limits of a rough
precision estimate for this ACM reduction (based on the precision limits
of the estimate of all deaths directly attributable to malaria shown in
assumption 1, 19.1--25.6%) are 14.3 percentage points (i.e., \[19.1 +
{19.1 / 2}\] / 2) and 19.2 percentage points (i.e., \[25.6 + {25.6 /
2}\] / 2).

For the lower limit of the sensitivity analysis, assume that indirect
malaria mortality is one-quarter of direct malaria mortality. The
estimated reduction in all-cause under-5-year mortality is 14.0
percentage points (i.e., \[22.4 + {22.4 / 4}\] / 2). The limits of a
rough precision estimate for this ACM reduction are 11.9 percentage
points (i.e., \[19.1 + {19.1 / 4}\] / 2) and 16.0 percentage points
(i.e., \[25.6 + {25.6 / 4}\] / 2). For the upper limit of the
sensitivity analysis, assume that indirect malaria mortality equals
direct malaria mortality. The estimated reduction in all-cause
under-5-year mortality is 22.4 percentage points (i.e., \[22.4 + 22.4\]
/ 2). The limits of a rough precision estimate for this ACM reduction
are 19.1 percentage points (i.e., \[19.1 + 19.1\] / 2) and 25.6
percentage points (i.e., \[25.6 + 25.6\] / 2).

**Annex 3. Estimates of the reduction in all-cause mortality for
children 0--59 months old if the coverage of interventions to prevent
and control malaria were increased from a very low level (\<5%) to a
"reasonably high" level (70%)**

## Assumption about the population and malaria burden

See assumption 1 in Annex 2. The following analysis is only for the 26
"high malaria burden" countries in sub-Saharan Africa. Altogether in
these 26 countries, in the year 2000, 3.0 million deaths occurred among
children under 5 years old, and 672,000 deaths (or 22.4%) were directly
attributable to malaria (precision estimate: 575,000--770,000, or
19.1--25.6%) \[Annex 12 of Rowe-CHERG, 2005\]. Also assume that the
burden of indirect malaria mortality is half the burden of direct
malaria mortality (see assumption 2 in Annex 2). Thus, 33.6% (22.4% +
\[22.4% / 2\]) of all deaths are attributable to the direct and indirect
effects of malaria, or 1.0 million malaria deaths. Lower and upper
limits of the precision estimate for this mortality proportion are 28.7%
(19.1% + \[19.1% / 2\]) and 38.4% (25.6% + \[25.6% / 2\]), respectively,
or 861,000 malaria deaths (28.7% x 3.0 million deaths) and 1,152,000
malaria deaths (38.4% x 3.0 million deaths), respectively.

## Contribution of insecticide-treated bednets (ITNs)

First, assume that if the prevalence of ITN use reaches about 70%, then
ACM among children 1--59 months old will decrease by 17% (95% CI:
10--23%) \[Lengeler, 2004\]. Second, assume that ITN use by neonates
does not reduce neonatal mortality, as there is no high-quality evidence
that ITN use is protective for this age group. Third, assume that 74% of
deaths among children 0--59 months old in Africa occur in the 1--59
month age range (based on an estimate that neonatal deaths account for
26% of all deaths among children 0--59 months old \[Bryce, 2005\]).
Thus, if ITN use increases to the "reasonably high" level of about 70%,
then ACM among children 0--59 months old will decrease by 17% (ITN's
protective efficacy) x 74%, or 12.6%. Lower and upper limits of the 95%
confidence interval for this estimate are 7.4% (10% \[ITN's protective
efficacy\] x 74%) and 17.0% (23% \[ITN's protective efficacy\] x 74%),
respectively.

*Contribution of intermittent preventive treatment for pregnant women
(IPTp)*

First, assume that if IPTp coverage reaches 100%, then ACM among
children 0--11 months old will decrease by 3%. This estimate is the
lower limit of a range of estimates (3--8%, see Table 5 of Steketee *et
al.*, 2001) that were based on a review of the effect of two trials of
malaria chemoprophylaxis on the prevalence of low birth weight and the
impact of low birth weight on infant mortality \[Steketee, 2001\].
Details of the derivation of the estimates were not published; so, to be
conservative, the lower estimate of 3% was assumed. Second, assume that
60% of deaths among children 0--59 months old in Africa occurs in the
first year of life (based on UNICEF estimates for sub-Saharan Africa in
2002: the infant mortality rate \[104 deaths by age 12 months per 1000
live births\] divided by the under-5-year mortality rate \[174 deaths by
age 5 years per 1000 live births\] \[UNICEF website,
<http://www.childinfo.org/cmr/revis/db2.htm>, accessed August 31,
2005\]). Third, assume that IPTp coverage increases to a "reasonably
high" level of about 70%. Thus, if IPTp coverage increases to about 70%,
then ACM among children 0--59 months old would decrease by 3% (IPTp's
protective efficacy) x 60% (percent of under-5-year deaths occurring in
the first year of life) x 70% (coverage), or 1.3% (no precision
estimate). For a sensitivity analysis to estimate the maximum impact,
assuming that IPTp's protective efficacy is 8% (the upper limit
estimated by Steketee *et al.* \[Steketee, 2001\]), the ACM among
children 0--59 months old would decrease by 8% x 60% x 70%, or 3.4% (no
precision estimate).

*Contribution of IPTp [and]{.underline} ITNs by children under 5 years
old*

Assume that IPTp provides protection from malaria that is independent of
the protection provided by ITNs. The justification is that IPTp prevents
deaths primarily by preventing low birth weight and that any child (even
those not born with a low birth weight) is at risk for malaria infection
and would benefit from ITN use. Thus, if the coverage of ITN use and IPT
use both reached 70%, then mortality among children 0--59 months old
would decrease by 12.6 percentage points (ITN's protective efficacy) +
1.3 percentage points (IPTp's protective efficacy), or 13.9%. Lower and
upper limits of the 95% confidence interval for this estimate are 8.7%
(7.4% + 1.3%) and 18.3% (17.0% + 1.3%), respectively. For a sensitivity
analysis to estimate the maximum impact, assuming IPT's protective
efficacy is 3.4%, the combination of ITN use and IPTp use would decrease
mortality among children 0--59 months old by 12.6% (ITN's protective
efficacy) + 3.4% (IPTp's protective efficacy), or 16.0% (95% CI:
10.8--20.4%).

*Contribution of prompt and effective case management (PECM) in addition
to IPTp and ITNs*

The following estimates were difficult to make because of the lack of
data. At least one study exists that demonstrates the potentially large
impact of PECM in the absence of IPTp and ITNs \[Kidane, 2000\], however
we were unable to find any data on the added benefit of PECM in a
population with reasonably high coverage (about 70%) of IPTp
[and]{.underline} ITNs (or ITNs alone). Thus the following estimates are
an educated guess.

First, PECM is defined as the correct treatment of true malaria
infections with a highly efficacious antimalarial drug within 24 hours
of the onset of symptoms. A programmatically relevant definition of
"correct treatment" is assumed in which (for inpatients) the child
receives all necessary drug doses or (for outpatients) the child's
caretaker has the drug in hand and the knowledge of how to administer it
correctly at home. Second, assume that PECM does not affect mortality
among neonates because neonates rarely die from the direct effects of
malaria \[Chongsuphajaisiddhi, 1988\]. Third, assume that PECM coverage
is 70% ("reasonably high" coverage), ITN coverage is 70%, and PECM
coverage and ITN coverage are independent (admittedly a rough
assumption). Thus, among children 1--59 months old, 49% (i.e., 70% x
70%) are protected by ITNs [and]{.underline} PECM, 21% (i.e., 70% --
49%) are protected by ITNs only, 21% (i.e., 70% -- 49%) are protected by
PECM only, and 9% are protected by neither ITNs nor PECM. Fourth, as a
conservative assumption for estimating the impact of PECM, assume that
if a child is protected by an ITN, then the child is completely
protected from malaria and thus PECM provides no additional protection
against mortality. Fifth, assume 1.0 million malaria deaths occur in
this population each year (see assumptions at the beginning of this
Annex). Sixth, assume the protective efficacy of PECM for preventing
direct and indirect malaria mortality is 80%. This estimate is a guess;
it is based on the notion that highly efficacious antimalarials almost
always (\>90%) prevent malaria deaths when administered under ideal
conditions, but that antimalarials will not protect against death in 20%
of cases because the drug is not correctly administered at home (despite
correct knowledge), the child does not absorb an adequate amount of the
drug (e.g., because of diarrhea), or because the child presents with
unusually severe complications and dies very soon thereafter. Finally,
assume that the protective benefit of IPT is independent from that of
PECM (see previous section of this Annex).

Given the above assumptions, PECM could provide additional benefit to
the 21% of children who are covered by PECM only (i.e., no ITN). If one
further assumes that 21% of the malaria burden would occur in this
sub-population in the absence of PECM, then one can estimate that PECM
would prevent 80% (PECM's protective efficacy) of 210,000 malaria deaths
(21% of the 1.0 million malaria deaths occurring during a "baseline" or
pre-intervention year in the 26 countries with high malaria burden), or
168,000 malaria deaths. If the 168,000 malaria deaths prevented by PECM
is divided by the total number of deaths among children 0--59 months old
(3.0 million deaths), then in a population with 70% coverage of ITNs,
PECM is estimated to reduce all-cause under-5-year mortality by an
additional 5.6 percentage points (168,000 / 3.0 million). Lower and
upper limits of the precision estimate for this figure are 4.8%
(\[861,000 malaria deaths x 21% x 80%\] / 3.0 million total deaths) and
6.5% (\[1,152,000 malaria deaths x 21% x 80%\] / 3.0 million total
deaths), respectively.

In summary, with reasonably high coverage (70%) of ITNs, IPT, and PECM,
ACM among children 0--59 months old is estimated to decrease by 19.5%
(12.6% \[from ITNs\] + 1.3% \[from IPT\] + 5.6% \[from PECM, given high
coverage of ITNs\]). Lower and upper limits of the precision estimate
for this figure are 13.5% (7.4% \[ITNs\] + 1.3% \[IPT\] + 4.8% \[PECM\])
and 24.8% (17.0% \[ITNs\] + 1.3% \[IPT\] + 6.5% \[PECM\]), respectively.
For a sensitivity analysis to estimate the maximum impact, assuming
IPT's protective efficacy is 3.4%, the combination of ITN use, IPT use,
and PECM would decrease mortality among children 0--59 months old by
12.6% (ITN's protective efficacy) + 3.4% (IPT's protective efficacy) +
5.6% (PECM's protective efficacy), or 21.6%. Lower and upper limits of
the precision estimate for this figure are 15.6% (7.4% \[ITNs\] + 3.4%
\[IPT\] + 4.8% \[PECM\]) and 26.9% (17.0% \[ITNs\] + 3.4% \[IPT\] + 6.5%
\[PECM\]), respectively.

Other estimates exist for the number of under-5-year deaths that would
be prevented by malaria interventions. Jones et al. \[Jones et al.,
2003\] estimated child deaths that could be prevented in the 42
countries with 90% of worldwide child deaths in 2000---a group of
countries that includes India, China, and Pakistan, as well as 23
countries from sub-Saharan Africa. They estimated that, if scaled up to
99% coverage, ITNs could prevent 691,000 deaths, IPT could prevent
22,000 deaths, and antimalarials could prevent 467,000 deaths. Given
that it is impossible to separate out how many of these deaths were
estimated for sub-Saharan Africa and given a lack of documentation about
the model used to make the estimates, it is difficult to compare these
figures to the estimates presented above. In a review of the burden of
malaria during pregnancy in Africa, Guyatt and Snow \[Guyatt, 2004\]
estimated that IPT could prevent 80,000 child deaths (no precision
estimate given). This estimate assumed that about 100,000 infant deaths
occur each year due to malaria in pregnancy causing low birth weight
(17,765,000 live births/year in areas of stable malaria risk in Africa x
17% risk of being born with low birth weight in areas with endemic
malaria x 19% of low birth weight children having the low birth weight
caused by malaria in pregnancy x 16.2% of low birth weight children
dying during the first year of life), and that IPT's efficacy for
preventing placental malaria infection was 80%. For comparison purposes,
based on the estimates derived in this document, if coverage was 100%,
IPT could prevent 54,000 child deaths (i.e., 3% \[IPT's protective
efficacy\] x 60% \[% of under-5-year deaths occurring in the first year
of life\] x 100% \[coverage\] x 3 million under-5-year deaths), or
144,000 deaths (if one assumes the upper estimate of IPT's protective
efficacy: 8%). Thus, Guyatt and Snow's estimate of 80,000 is almost
halfway in between the range of estimates presented in this document
(54,000 and 144,000).

**Annex 4. Estimate of the time needed for a demographic surveillance
system (DSS) to detect a 50% reduction in malaria mortality**

## Assumptions

First, assume the DSS is in a rural area and has a constant population
of 60,000 (a medium-sized DSS), of which 9600 (16%) are under 5 years
old. Second, assume the malaria mortality rate (i.e., the mortality rate
for deaths directly attributable to malaria) for children under 5 years
old is 11.4 deaths per 1000 children per year (the rate estimated for
rural populations in areas with high malaria transmission in the year
2000 \[Rowe-CHERG, 2005\]) at baseline, when coverage of interventions
is low (\<5%). Thus at baseline (e.g., the year 2005) there are 109
malaria deaths per year among children, and the goal is to reduce
malaria mortality by 50% to a rate of 5.7/1000/year. Third, 95% CIs for
rates are estimated using the standard error of a Poisson distribution:
rate +/- (1.96 x \[square-root{deaths} / person-time\]) \[U.S. DHHS,
1999\]. Fourth, assume it takes one year to scale up interventions (the
year 2006), and that malaria mortality has been reduced by 50% the
following year (2007).

## Results

At baseline, the child malaria mortality rate is 11.4/1000/year, with a
95% CI of 9.2--13.5. For simplicity, ignore results for the one year in
which interventions are scaled up. In the first follow-up year in which
interventions are completely scaled up, the child malaria mortality rate
is 5.7/1000/year, with a 95% CI of 4.2--7.2. As the 95% CIs for the
baseline and follow-up year do not overlap, one can conclude that a
statistically significant reduction in the malaria mortality rate could
be detected after only one year of follow-up. Even if one assumes a
lower baseline malaria mortality rate (e.g., 8.3/1000/year, the average
rate estimated for urban populations in areas with high malaria
transmission in the year 2000 \[Rowe-CHERG, 2005\]), the 95% CI for the
baseline (6.5--10.2) and follow-up year (rate = 4.2/1000/year, 95% CI:
2.9--5.5) still do not overlap, indicating statistical significance.

# Annex 5. A multi-faceted M&E proposal for PMI in a generic country (see explanation provided in the Discussion, section V)

+----------------------------------+---+----+---+---+---+---+----+
| Activity (M&E, operations        | Y |    |   |   |   |   |    |
| research, capacity building, or  | e |    |   |   |   |   |    |
| programmatic activity)           | a |    |   |   |   |   |    |
|                                  | r |    |   |   |   |   |    |
|                                  | a |    |   |   |   |   |    |
|                                  | n |    |   |   |   |   |    |
|                                  | d |    |   |   |   |   |    |
|                                  | e |    |   |   |   |   |    |
|                                  | s |    |   |   |   |   |    |
|                                  | t |    |   |   |   |   |    |
|                                  | i |    |   |   |   |   |    |
|                                  | m |    |   |   |   |   |    |
|                                  | a |    |   |   |   |   |    |
|                                  | t |    |   |   |   |   |    |
|                                  | e |    |   |   |   |   |    |
|                                  | d |    |   |   |   |   |    |
|                                  | c |    |   |   |   |   |    |
|                                  | o |    |   |   |   |   |    |
|                                  | s |    |   |   |   |   |    |
|                                  | t |    |   |   |   |   |    |
|                                  | ^ |    |   |   |   |   |    |
|                                  | a |    |   |   |   |   |    |
|                                  | ^ |    |   |   |   |   |    |
|                                  | ( |    |   |   |   |   |    |
|                                  | g |    |   |   |   |   |    |
|                                  | r |    |   |   |   |   |    |
|                                  | a |    |   |   |   |   |    |
|                                  | y |    |   |   |   |   |    |
|                                  | b |    |   |   |   |   |    |
|                                  | o |    |   |   |   |   |    |
|                                  | x |    |   |   |   |   |    |
|                                  | i |    |   |   |   |   |    |
|                                  | n |    |   |   |   |   |    |
|                                  | d |    |   |   |   |   |    |
|                                  | i |    |   |   |   |   |    |
|                                  | c |    |   |   |   |   |    |
|                                  | a |    |   |   |   |   |    |
|                                  | t |    |   |   |   |   |    |
|                                  | e |    |   |   |   |   |    |
|                                  | s |    |   |   |   |   |    |
|                                  | y |    |   |   |   |   |    |
|                                  | e |    |   |   |   |   |    |
|                                  | a |    |   |   |   |   |    |
|                                  | r |    |   |   |   |   |    |
|                                  | o |    |   |   |   |   |    |
|                                  | f |    |   |   |   |   |    |
|                                  | a |    |   |   |   |   |    |
|                                  | c |    |   |   |   |   |    |
|                                  | t |    |   |   |   |   |    |
|                                  | i |    |   |   |   |   |    |
|                                  | v |    |   |   |   |   |    |
|                                  | i |    |   |   |   |   |    |
|                                  | t |    |   |   |   |   |    |
|                                  | y |    |   |   |   |   |    |
|                                  | ) |    |   |   |   |   |    |
+----------------------------------+---+----+---+---+---+---+----+
|                                  | 2 | 20 | 2 | 2 | 2 | 2 | 20 |
|                                  | 0 | 06 | 0 | 0 | 0 | 0 | 11 |
|                                  | 0 |    | 0 | 0 | 0 | 1 |    |
|                                  | 1 |    | 7 | 8 | 9 | 0 |    |
|                                  | - |    |   |   |   |   |    |
|                                  | - |    |   |   |   |   |    |
|                                  | 5 |    |   |   |   |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| "Phase" of programmatic          | B | Sc |   | F |   |   |    |
| activities                       | a | al |   | u |   |   |    |
|                                  | s | e- |   | l |   |   |    |
|                                  | e | up |   | l |   |   |    |
|                                  | l |    |   | i |   |   |    |
|                                  | i |    |   | m |   |   |    |
|                                  | n |    |   | p |   |   |    |
|                                  | e |    |   | l |   |   |    |
|                                  |   |    |   | e |   |   |    |
|                                  |   |    |   | m |   |   |    |
|                                  |   |    |   | e |   |   |    |
|                                  |   |    |   | n |   |   |    |
|                                  |   |    |   | t |   |   |    |
|                                  |   |    |   | a |   |   |    |
|                                  |   |    |   | t |   |   |    |
|                                  |   |    |   | i |   |   |    |
|                                  |   |    |   | o |   |   |    |
|                                  |   |    |   | n |   |   |    |
|                                  |   |    |   | ( |   |   |    |
|                                  |   |    |   | i |   |   |    |
|                                  |   |    |   | . |   |   |    |
|                                  |   |    |   | e |   |   |    |
|                                  |   |    |   | . |   |   |    |
|                                  |   |    |   | , |   |   |    |
|                                  |   |    |   | m |   |   |    |
|                                  |   |    |   | a |   |   |    |
|                                  |   |    |   | l |   |   |    |
|                                  |   |    |   | a |   |   |    |
|                                  |   |    |   | r |   |   |    |
|                                  |   |    |   | i |   |   |    |
|                                  |   |    |   | a |   |   |    |
|                                  |   |    |   | m |   |   |    |
|                                  |   |    |   | o |   |   |    |
|                                  |   |    |   | r |   |   |    |
|                                  |   |    |   | t |   |   |    |
|                                  |   |    |   | a |   |   |    |
|                                  |   |    |   | l |   |   |    |
|                                  |   |    |   | i |   |   |    |
|                                  |   |    |   | t |   |   |    |
|                                  |   |    |   | y |   |   |    |
|                                  |   |    |   | s |   |   |    |
|                                  |   |    |   | h |   |   |    |
|                                  |   |    |   | o |   |   |    |
|                                  |   |    |   | u |   |   |    |
|                                  |   |    |   | l |   |   |    |
|                                  |   |    |   | d |   |   |    |
|                                  |   |    |   | b |   |   |    |
|                                  |   |    |   | e |   |   |    |
|                                  |   |    |   | r |   |   |    |
|                                  |   |    |   | e |   |   |    |
|                                  |   |    |   | d |   |   |    |
|                                  |   |    |   | u |   |   |    |
|                                  |   |    |   | c |   |   |    |
|                                  |   |    |   | e |   |   |    |
|                                  |   |    |   | d |   |   |    |
|                                  |   |    |   | b |   |   |    |
|                                  |   |    |   | y |   |   |    |
|                                  |   |    |   | 5 |   |   |    |
|                                  |   |    |   | 0 |   |   |    |
|                                  |   |    |   | % |   |   |    |
|                                  |   |    |   | f |   |   |    |
|                                  |   |    |   | o |   |   |    |
|                                  |   |    |   | r |   |   |    |
|                                  |   |    |   | t |   |   |    |
|                                  |   |    |   | h |   |   |    |
|                                  |   |    |   | e |   |   |    |
|                                  |   |    |   | 3 |   |   |    |
|                                  |   |    |   | - |   |   |    |
|                                  |   |    |   | y |   |   |    |
|                                  |   |    |   | e |   |   |    |
|                                  |   |    |   | a |   |   |    |
|                                  |   |    |   | r |   |   |    |
|                                  |   |    |   | p |   |   |    |
|                                  |   |    |   | e |   |   |    |
|                                  |   |    |   | r |   |   |    |
|                                  |   |    |   | i |   |   |    |
|                                  |   |    |   | o |   |   |    |
|                                  |   |    |   | d |   |   |    |
|                                  |   |    |   | ) |   |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| DHS with "regular" sample size   | S |    |   |   |   |   |    |
| (10,400 women), i.e., to         | e |    |   |   |   |   |    |
| estimate ~5~q~0~ for the 5 years | e |    |   |   |   |   |    |
| before survey                    | f |    |   |   |   |   |    |
|                                  | o |    |   |   |   |   |    |
|                                  | o |    |   |   |   |   |    |
|                                  | t |    |   |   |   |   |    |
|                                  | n |    |   |   |   |   |    |
|                                  | o |    |   |   |   |   |    |
|                                  | t |    |   |   |   |   |    |
|                                  | e |    |   |   |   |   |    |
|                                  | b |    |   |   |   |   |    |
|                                  | . |    |   |   |   |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| DHS with extra sample (17,326    |   |    |   |   |   |   | \$ |
| women) to estimate ~5~q~0~ for   |   |    |   |   |   |   | 35 |
| the 3 years before survey^b^     |   |    |   |   |   |   | 0, |
|                                  |   |    |   |   |   |   | 00 |
|                                  |   |    |   |   |   |   | 0^ |
|                                  |   |    |   |   |   |   | b^ |
+----------------------------------+---+----+---+---+---+---+----+
| Teams to conduct VAs after the   | \ |    |   |   |   |   | \  |
| survey (if necessary)            | $ |    |   |   |   |   | $1 |
|                                  | 1 |    |   |   |   |   | 00 |
|                                  | 0 |    |   |   |   |   | ,0 |
|                                  | 0 |    |   |   |   |   | 00 |
|                                  | , |    |   |   |   |   |    |
|                                  | 0 |    |   |   |   |   |    |
|                                  | 0 |    |   |   |   |   |    |
|                                  | 0 |    |   |   |   |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| DSS with a population size of    |   | \  | \ | \ | \ | \ |    |
| 60,000 to measure all-cause and  |   | $2 | $ | $ | $ | $ |    |
| malaria mortality^c^             |   | 80 | 1 | 1 | 1 | 1 |    |
|                                  |   | ,0 | 8 | 8 | 8 | 8 |    |
|                                  |   | 00 | 0 | 0 | 0 | 0 |    |
|                                  |   |    | , | , | , | , |    |
|                                  |   |    | 0 | 0 | 0 | 0 |    |
|                                  |   |    | 0 | 0 | 0 | 0 |    |
|                                  |   |    | 0 | 0 | 0 | 0 |    |
+----------------------------------+---+----+---+---+---+---+----+
| PBT to measure all-cause child   |   | \  | \ | \ | \ | \ | \$ |
| mortality ~2~q~0~ (if            |   | $1 | $ | $ | $ | $ | 50 |
| appropriate conditions exist)^d^ |   | 00 | 5 | 5 | 5 | 5 | ,0 |
|                                  |   | ,0 | 0 | 0 | 0 | 0 | 00 |
|                                  |   | 00 | , | , | , | , |    |
|                                  |   |    | 0 | 0 | 0 | 0 |    |
|                                  |   |    | 0 | 0 | 0 | 0 |    |
|                                  |   |    | 0 | 0 | 0 | 0 |    |
+----------------------------------+---+----+---+---+---+---+----+
| Study to estimate sensitivity    |   | \$ |   |   | \ |   |    |
| and specificity of VA's ability  |   | 40 |   |   | $ |   |    |
| to identify malaria deaths^e^    |   | ,0 |   |   | 4 |   |    |
|                                  |   | 00 |   |   | 0 |   |    |
|                                  |   |    |   |   | , |   |    |
|                                  |   |    |   |   | 0 |   |    |
|                                  |   |    |   |   | 0 |   |    |
|                                  |   |    |   |   | 0 |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| National MIS with anemia and     |   | \  |   | \ | \ |   |    |
| parasitemia^e^ at end of rainy   |   | $3 |   | $ | $ |   |    |
| season (plus info for wealth     |   | 96 |   | 3 | 3 |   |    |
| index and factors related to     |   | ,0 |   | 9 | 9 |   |    |
| all-cause mortality)^a^          |   | 00 |   | 6 | 6 |   |    |
|                                  |   |    |   | , | , |   |    |
|                                  |   |    |   | 0 | 0 |   |    |
|                                  |   |    |   | 0 | 0 |   |    |
|                                  |   |    |   | 0 | 0 |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| Capacity building to establish   |   | \$ | \ | \ | \ |   |    |
| an in-country team that can      |   | 30 | $ | $ | $ |   |    |
| conduct and analyze MIS (and     |   | ,0 | 1 | 1 | 1 |   |    |
| hopefully other surveys) on      |   | 00 | 5 | 5 | 5 |   |    |
| their own.                       |   |    | , | , | , |   |    |
|                                  |   |    | 0 | 0 | 0 |   |    |
|                                  |   |    | 0 | 0 | 0 |   |    |
|                                  |   |    | 0 | 0 | 0 |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| National health facility survey  |   | \$ |   | \ |   | \ |    |
| in outpatient sick child clinic  |   | 60 |   | $ |   | $ |    |
| and antenatal clinic to evaluate |   | ,0 |   | 6 |   | 6 |    |
| malaria case management, ACT     |   | 00 |   | 0 |   | 0 |    |
| stocks, IPT use, and other       |   |    |   | , |   | , |    |
| malaria-related health worker    |   |    |   | 0 |   | 0 |    |
| functions                        |   |    |   | 0 |   | 0 |    |
|                                  |   |    |   | 0 |   | 0 |    |
+----------------------------------+---+----+---+---+---+---+----+
| Collect supervision/quality      |   | \$ | \ | \ | \ | \ |    |
| improvement data, after          |   | 80 | $ | $ | $ | $ |    |
| establishing a                   |   | ,0 | 5 | 5 | 5 | 5 |    |
| quality-improvement "group" to   |   | 00 | 0 | 0 | 0 | 0 |    |
| monitor and improve              |   |    | , | , | , | , |    |
| malaria-related activities at    |   |    | 0 | 0 | 0 | 0 |    |
| outpatient sick child clinics    |   |    | 0 | 0 | 0 | 0 |    |
| and antenatal clinics^e^         |   |    | 0 | 0 | 0 | 0 |    |
+----------------------------------+---+----+---+---+---+---+----+
| Collect HIS data and hospital    |   | \$ | \ | \ | \ | \ | \$ |
| data on inpatient child malaria  |   | 30 | $ | $ | $ | $ | 20 |
| and severe anemia deaths,        |   | ,0 | 2 | 2 | 2 | 2 | ,0 |
| hospital utilization, ITNs       |   | 00 | 0 | 0 | 0 | 0 | 00 |
| distributed (regular, LLINs),    |   |    | , | , | , | , |    |
| ITNs retreated, and ACTs used    |   |    | 0 | 0 | 0 | 0 |    |
| for children^e^                  |   |    | 0 | 0 | 0 | 0 |    |
|                                  |   |    | 0 | 0 | 0 | 0 |    |
+----------------------------------+---+----+---+---+---+---+----+
| Data collection for              |   | \$ | \ | \ | \ | \ | \$ |
| "confounders" (rainfall,         |   | 50 | $ | $ | $ | $ | 50 |
| urbanization, wealth, etc)       |   | 00 | 5 | 5 | 5 | 5 | 00 |
|                                  |   |    | 0 | 0 | 0 | 0 |    |
|                                  |   |    | 0 | 0 | 0 | 0 |    |
|                                  |   |    | 0 | 0 | 0 | 0 |    |
+----------------------------------+---+----+---+---+---+---+----+
| Estimated total for each year    | \ | \  | \ | \ | \ | \ | \  |
| that PMI would have to cover.    | $ | $1 | $ | $ | $ | $ | $4 |
| Assumes: 1) other malaria        | 0 | ,0 | 3 | 7 | 7 | 3 | 25 |
| initiatives do not contribute to |   | 21 | 2 | 7 | 5 | 6 | ,0 |
| the plan, 2) a PBT system is     |   | ,0 | 0 | 6 | 6 | 5 | 00 |
| implemented, 3) a DSS is         |   | 00 | , | , | , | , |    |
| implemented, 4) VA teams are     |   |    | 0 | 0 | 0 | 0 |    |
| [not]{.underline} needed, and 5) |   |    | 0 | 0 | 0 | 0 |    |
| MIS cost is \$396,000.           |   |    | 0 | 0 | 0 | 0 |    |
|                                  |   |    |   |   |   |   |    |
| **Estimated grand total for all  |   |    |   |   |   |   |    |
| years = \$3,663,000**            |   |    |   |   |   |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| Estimated total for each year    | \ | \  | \ | \ | \ | \ | \  |
| that PMI would have to cover.    | $ | $8 | $ | $ | $ | $ | $4 |
| Assumes: 1) other malaria        | 0 | 25 | 3 | 5 | 5 | 3 | 25 |
| initiatives do not contribute to |   | ,0 | 2 | 8 | 6 | 6 | ,0 |
| the plan, 2) a PBT system is     |   | 00 | 0 | 0 | 0 | 5 | 00 |
| implemented, 3) a DSS is         |   |    | , | , | , | , |    |
| implemented, 4) VA teams are     |   |    | 0 | 0 | 0 | 0 |    |
| [not]{.underline} needed, and 5) |   |    | 0 | 0 | 0 | 0 |    |
| MIS cost is \$200,000.           |   |    | 0 | 0 | 0 | 0 |    |
|                                  |   |    |   |   |   |   |    |
| **Estimated grand total for all  |   |    |   |   |   |   |    |
| years = \$3,075,000**            |   |    |   |   |   |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| Estimated total for each year    | \ | \  | \ | \ | \ | \ | \  |
| that PMI would have to cover.    | $ | $9 | $ | $ | $ | $ | $3 |
| Assumes: 1) other malaria        | 0 | 21 | 2 | 7 | 7 | 3 | 75 |
| initiatives do not contribute to |   | ,0 | 7 | 2 | 0 | 1 | ,0 |
| the plan, 2) a PBT system is     |   | 00 | 0 | 6 | 6 | 5 | 00 |
| [not]{.underline} implemented,   |   |    | , | , | , | , |    |
| 3) a DSS is implemented, 4) VA   |   |    | 0 | 0 | 0 | 0 |    |
| teams are [not]{.underline}      |   |    | 0 | 0 | 0 | 0 |    |
| needed, and 5) MIS cost is       |   |    | 0 | 0 | 0 | 0 |    |
| \$396,000.                       |   |    |   |   |   |   |    |
|                                  |   |    |   |   |   |   |    |
| **Estimated grand total for all  |   |    |   |   |   |   |    |
| years = \$3,313,000**            |   |    |   |   |   |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| Estimated total for each year    | \ | \  | \ | \ | \ | \ | \  |
| that PMI would have to cover.    | $ | $7 | $ | $ | $ | $ | $3 |
| Assumes: 1) other malaria        | 0 | 25 | 2 | 5 | 5 | 3 | 75 |
| initiatives do not contribute to |   | ,0 | 7 | 3 | 1 | 1 | ,0 |
| the plan, 2) a PBT system is     |   | 00 | 0 | 0 | 0 | 5 | 00 |
| [not]{.underline} implemented,   |   |    | , | , | , | , |    |
| 3) a DSS is implemented, 4) VA   |   |    | 0 | 0 | 0 | 0 |    |
| teams are [not]{.underline}      |   |    | 0 | 0 | 0 | 0 |    |
| needed, and 5) MIS cost is       |   |    | 0 | 0 | 0 | 0 |    |
| \$200,000.                       |   |    |   |   |   |   |    |
|                                  |   |    |   |   |   |   |    |
| **Estimated grand total for all  |   |    |   |   |   |   |    |
| years = \$2,725,000**            |   |    |   |   |   |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| Estimated total for each year    | \ | \  | \ | \ | \ | \ | \  |
| that PMI would have to cover.    | $ | $6 | $ | $ | $ | $ | $3 |
| Assumes: 1) other malaria        | 0 | 01 | 9 | 5 | 4 | 1 | 75 |
| initiatives do not contribute to |   | ,0 | 0 | 4 | 8 | 3 | ,0 |
| the plan, 2) a PBT system is     |   | 00 | , | 6 | 6 | 5 | 00 |
| [not]{.underline} implemented,   |   |    | 0 | , | , | , |    |
| 3) a DSS is [not]{.underline}    |   |    | 0 | 0 | 0 | 0 |    |
| implemented (and therefore no    |   |    | 0 | 0 | 0 | 0 |    |
| need to study VA sensitivity and |   |    |   | 0 | 0 | 0 |    |
| specificity), 4) VA teams are    |   |    |   |   |   |   |    |
| [not]{.underline} needed, and 5) |   |    |   |   |   |   |    |
| MIS cost is \$396,000.           |   |    |   |   |   |   |    |
|                                  |   |    |   |   |   |   |    |
| # Estimated grand t              |   |    |   |   |   |   |    |
| otal for all years = \$2,233,000 |   |    |   |   |   |   |    |
+----------------------------------+---+----+---+---+---+---+----+
| Estimated total for each year    | \ | \  | \ | \ | \ | \ | \  |
| that PMI would have to cover.    | $ | $4 | $ | $ | $ | $ | $3 |
| Assumes: 1) other malaria        | 0 | 05 | 9 | 3 | 2 | 1 | 75 |
| initiatives do not contribute to |   | ,0 | 0 | 5 | 9 | 3 | ,0 |
| the plan, 2) a PBT system is     |   | 00 | , | 0 | 0 | 5 | 00 |
| [not]{.underline} implemented,   |   |    | 0 | , | , | , |    |
| 3) a DSS is [not]{.underline}    |   |    | 0 | 0 | 0 | 0 |    |
| implemented (and therefore no    |   |    | 0 | 0 | 0 | 0 |    |
| need to study VA sensitivity and |   |    |   | 0 | 0 | 0 |    |
| specificity), 4) VA teams are    |   |    |   |   |   |   |    |
| [not]{.underline} needed, and 5) |   |    |   |   |   |   |    |
| MIS cost is \$200,000.           |   |    |   |   |   |   |    |
|                                  |   |    |   |   |   |   |    |
| # Estimated grand t              |   |    |   |   |   |   |    |
| otal for all years = \$1,645,000 |   |    |   |   |   |   |    |
+----------------------------------+---+----+---+---+---+---+----+

[Footnotes:]{.underline}

Abbreviations: ACT = artemisinin-based combination therapy, DHS =
Demographic and Health Survey, DSS = demographic surveillance system,
HIS = health information system, ITN = insecticide-treated net, LLIN =
long-lasting insecticide-treated net, M&E = monitoring and evaluation,
PBT = preceding birth technique, PMI = President's Malaria Initiative,
VA = verbal autopsy.

^a^ Many costs are very rough estimates. Except for DHSs, cost estimates
exclude indirect costs that might be needed for organizations to conduct
the activities. If the capacity-building component of the plan is
successful, MIS costs might decrease over time, as less external
technical assistance is required. Also, it is possible that the MIS
would cost less than the \$396,000 estimate shown on the table (e.g.,
perhaps \$200,000).

^b^ For the DHS in 2005, it is assumed that the malaria program would
not pay for any of the estimated \$2.1 million cost because the DHS is
usually funded from other sources. Similarly, for the 2011 DHS, it is
assumed that the malaria program would not pay for any of the "regular"
part of the survey (i.e., the estimated \$2.35 million for data
collected from the regular sample of 10,400 women). However, the DHS in
2011 is assumed to have extra sample (6926 women) especially for the
malaria program; so the malaria program would pay for the extra sample
(estimated as \$350,000). To reduce survey costs, for this extra sample,
a shorter questionnaire would be used that only collects information
needed to estimate child mortality. In summary, the total estimated cost
of the DHS in 2011 is \$2.7 million (i.e., \$2.35 million for the
regular part of the survey + \$350,000 for the malaria program's extra
sample). If a DHS was not going to be conducted in a year when the
malaria program wanted an estimate of child mortality for the 5 years
before the survey, a DHS-type survey could be conducted that only
measured malaria indicators and child mortality. Such a survey would
cost about \$800,000 (cost could vary from \$600,000 to \$1 million).

^c^ Based on an annual cost of \$3 per person under surveillance per
year, plus \$100,000 during the first year for start-up costs, such as
vehicles and computers.

^d^ Necessary conditions are: 1) a high proportion of births occur at
health facilities, 2) a high fertility rate, and 3) low use of
contraceptives.

^e^ This is an operations research component of the evaluation. The idea
is that the PMI might be a unique opportunity to collect large amounts
of monitoring and evaluation data at one time; thus, for a few targeted
data types (e.g., parasite prevalence, hospital-based surveillance), the
M&E plan proposes to collect the data and then evaluate its validity,
utility, and cost.

[^1]: This latter method would be important if there was a clear, large
    downward or upward trend in the baseline (i.e., pre-project) period.
    For example, imagine a country in southern Africa that has good
    malaria surveillance and that had a clear upward malaria mortality
    trend of 25% per year during the baseline period (e.g., 1995--2000).
    If there were 200 malaria deaths in the year 2000, one could
    estimate that if conditions did not change, 1863 deaths would occur
    in 2010 (i.e., 200 x 1.25^10^). If one used the first method
    described above, with 200 deaths in the baseline year, then the 50%
    reduction goal would only be achieved if in 2010 there were 100 or
    fewer malaria deaths. But, given the clear upward trend (e.g.,
    caused by unavoidable migrations from low transmission areas to high
    transmission areas), it might be impossible to reduce malaria
    mortality to 100 deaths by 2010---even with well-implemented malaria
    control strategies (it would require a 95% mortality reduction
    relative to the predicted 1863 deaths, \[1863--100\]/1863). In this
    case, the second method seems more appropriate. If the country in
    the above example had 931 malaria deaths in 2010 (i.e., 50% of the
    predicted 1863 deaths), one could reasonably judge that the goal of
    50% reduction had been achieved. Most African countries with a high
    malaria burden will not have reliable trend data for malaria
    mortality. Perhaps in the future, if trend data are available and
    large, clear trends exist, the second method could be attempted.
    Also, if an evaluation option is used that is based on trends in
    all-cause mortality (option 1, see section III.B, below), perhaps
    the second method might be useful.

[^2]: Potential sources of information on rainfall, other climate data,
    and how health outcomes might be adjusted to account for climate
    change are: a) Grover et al. Malaria Journal 2005;4:6; b) Thomson et
    al. AJTMH 2005;73:214; c) Internet web site
    [www.iri.columbia.edu](http://www.iri.columbia.edu/); d) Over et al.
    AJTMH 2004;71:214; and e) the INDEPTH Network \[Lusinde et al,
    2003\].

[^3]: See Annex 3 for a justification of minimum coverage level of 70%,
    and see Annex 1 for a description of the statistical method
    recommended for determining whether a minimum coverage level has
    been achieved.

[^4]: The 5 categories are: 1) Type I -- No observed change in morbidity
    or mortality; 2) Type II -- Observed change in morbidity or
    mortality, not clearly attributable to intervention; 3) Type III --
    Observed change in morbidity or mortality, probably attributable to
    intervention, magnitude unquantifiable; 4) Type IV -- Observed
    change in morbidity or mortality, probably attributable to
    intervention, magnitude quantifiable; and 5) Type V -- Observed
    change in morbidity or mortality, almost certainly attributable to
    intervention, magnitude quantifiable.

[^5]: Note that the use of the term "child mortality" in this section
    means any reasonable measure of mortality among young children.
    Practically speaking, the measure that would be used most frequently
    is ~5~q~0~ from birth history surveys (e.g., DHS), however, other
    measures could be used (e.g., 0--4 year mortality rate per 1000
    children per year directly measured in a demographic surveillance
    system, or ~2~q~0~ from the preceding birth technique).

[^6]: There are 3 reasons why the rates may have been overestimated.
    First, malaria mortality rates for children 5--14 years old in areas
    with high malaria transmission were so high that the median value in
    the analysis (1.58 malaria deaths per 1000 children per year)
    represented 52.2% of all deaths in this age group---a rather high
    percentage \[Snow-DCPP, 2003 (page 11)\]. Second, some studies
    contributing to the malaria mortality rate for the 15+ year-old
    group were from before 1950 \[Snow-DCPP, 2003 (page 11); and
    personal communication from R.W. Snow on November 1, 2005)\], which
    might not be applicable to the year 2000. Finally, two adult studies
    with relatively high malaria mortality rates \[AMMP, 1997 (for Dar
    es Salaam and Morogoro)\] were based on results for "acute febrile
    illness deaths"---many of which are not caused by malaria (e.g., one
    multicenter study found only 85 \[36.0%\] of 236 acute febrile
    illness deaths were caused by malaria \[Chandramohan, 1998 (Table
    2)\]).

[^7]: Among under 5s, the all-cause mortality rate was 36.8 per 1000
    person years (95% CI: 33.2--40.7) in 1999, and 16.5 per 1000 person
    years (95% CI: 14.5--18.5) in 2003. Among individuals from 5--59
    years old, the all-cause mortality rate was 6.2 per 1000 person
    years (95% CI: 5.5--7.0) in 1999, and 4.6 per 1000 person years (95%
    CI: 4.1--5.2) in 2003 \[personal communication, D. de Savigny,
    February 5, 2006\].

[^8]: Although a less satisfying outcome, one could also make a
    plausibility argument that malaria control activities were
    "successful" if mortality does not significantly change when the
    coverage of malaria control interventions increased to high levels
    [and]{.underline} anemia prevalence decreased [and]{.underline}
    other factors that might influence child mortality changed
    dramatically in a direction that would increase child mortality. The
    idea is that mortality reductions caused by the malaria control
    interventions could be offset or negated by mortality increases
    caused by other factors---e.g., if vaccination coverage dropped
    substantially and a large measles epidemic occurred.

[^9]: For example, over one year, if ITN coverage (measured at the
    population level) increases and measles immunization coverage
    (measured at the population level) remains unchanged, all-cause
    mortality might decrease because, at the individual level, ITNs were
    distributed to children who (by chance) were not destined to be
    bitten by an infected mosquito during the year while measles
    immunization doses (a number just large enough to equal the number
    of immunized 4-year olds who turned 5 and left the under-5-year
    cohort) were administered to children who (by chance) were destined
    to be exposed to measles cases during the year. So, although more
    children slept under ITNs, the extra ITNs did not prevent malaria
    deaths during the year. All-cause mortality decreased because
    children at greatest risk for dying from measles were protected from
    measles. From the population perspective, however, it would be
    impossible to know this situation existed; all that would be
    apparent is that ITN coverage increased, measles immunization
    coverage remained unchanged, and all-cause mortality decreased.

[^10]: The DSSs (population size under surveillance in parentheses)
    are: 1) Bandafassi, Senegal (10,500); 2) Bandim, Guinea-Bissau
    (100,000); 3) Butajira, Ethiopia (40,000); 4) Farafenni, The Gambia
    (16,883); 5) Ifakara, Tanzania (65,000); 6) Iganda/Mayuge, Uganda
    (70,000); 7) Karonga, Malawi (20,000); 8) Kilifi, Kenya (56,000); 9)
    Kintampo, Ghana (140,000); 10) Kisumu, Kenya (135,500); 11) Magu,
    Tanzania (30,000); 12) Manhiça, Mozambique (36,000); 13) Mlomp,
    Senegal (6200); 14) Navrongo, Ghana (140,000); 15) Niakhar, Senegal
    (29,000); 16) Nouna, Burkina Faso (55,000); 17) Ouagadougou, Burkina
    Faso (5000); 18) Oubritenga, Burkina Faso (99,705); 19) Rakai,
    Uganda (12,000); 20) Rufiji, Tanzania (85,000); and 21) Gwembe,
    Zambia (15,000). Note that Agincourt, South Africa (66,800);
    Dikgale, South Africa (7900); Hlabisa, South Africa (90,000); and
    Nairobi, Kenya (population size not found) are not counted because
    the burden of malaria mortality is very low.

[^11]: For example, a death that was truly caused by bacterial sepsis
    might be attributed to malaria because the terminal illness was
    characterized by fever, loss of consciousness, seizures, and death
    (i.e., clinical signs that are common to both bacterial sepsis and
    cerebral malaria).

[^12]: An approach for developing these categories is to examine
    datasets from ITN trials to see which verbal autopsy categories were
    impacted by ITN use---such categories are likely to reflect direct
    or indirect malaria mortality.

[^13]: Numerous anecdotes suggest that it may be a common practice in
    sub-Saharan Africa for families to take patients out of the hospital
    if the patient appears to be at the brink of death.

[^14]: Snow et al. write: \"The most plausible explanation for the
    patterns of severe malaria we have described is that a given amount
    of exposure is required for effective clinical immunity to develop.
    When infection rates are high exposure early in life furnishes a
    child with an acquired resistance to the consequences of infection.
    During this window of time other mechanisms operate to modulate
    disease severity, particularly during the first six months of life
    when the infant is protected by maternally acquired antibody. The
    combined effects of an attenuated disease-risk period and reduced
    disease susceptibility early in life is likely to account for the
    reduced cumulative risk of disease by the end of childhood. In areas
    of very-low-intensity transmission the lack of development of
    immunity is balanced by the low frequency of infection, so disease
    rates will be low.\"

[^15]: Individuals never infected with malaria, individuals with fever
    from malaria, individuals infected with malaria but without fever,
    and individuals previously but not currently infected with malaria.

[^16]: For example, if intervention coverage is expected to be slightly
    under program targets, an end-of-project mortality survey could be
    modified to have a larger sample size to have adequate statistical
    power to identify a smaller mortality reduction. Alternatively, if
    intervention coverage is expected to miss program targets by a large
    margin, an end-of-project mortality survey could be cancelled or
    replaced with a smaller (less expensive) coverage survey. One could
    assume that the mortality goal was not achieved.