![](media/image1.wmf){width="1.198611111111111in"
height="0.9923611111111111in"}

Verification & Validation Report for ArboNET/Plague Surveillance System

NASA GSFC / GIMMS in partnership with the Centers for Disease Control
and Prevention

September 2006

Acknowledgments

This work was supported by NASA Science Mission Directorate, Earth
Science Division under the Applied Sciences Program, Public Health
Program Element.

+-----------------------------------+-----------------------------------+
| ArboNET/PSS Team                  |                                   |
+-----------------------------------+-----------------------------------+
| *Compton J. Tucker--Team Leader*  |                                   |
+-----------------------------------+-----------------------------------+
| [NASA/GIMMS/SSAI]{.underline}     | [CDC]{.underline}                 |
|                                   |                                   |
| Jorge E. Pinzon                   | Kenneth L. Gage                   |
|                                   |                                   |
| Joseph Nigro                      | Russell E. Enscore                |
|                                   |                                   |
|                                   | Rebecca J. Eisen                  |
+-----------------------------------+-----------------------------------+

**© National Aeronautics and Space Administration**

This document is not a formal publication of NASA, and all rights are
reserved by the Administration. The document may, however, be freely
reviewed, abstracted, reproduced and translated, in part or in whole,
but not for sale nor for use in conjunction with commercial purposes.
The views expressed in documents by named authors are solely the
responsibility of those authors. The mention of specific companies or
specific manufacturers\' products does no imply that they are endorsed
or recommended by the National Aeronautics and Space Administration in
preference to others of a similar nature that are not mentioned.

# Table of Contents {#table-of-contents .Heading-NoTOC}

**Table of Contents [iii](#table-of-contents)**

Figures iii

Tables [iv](#tables)

Executive Summary [v](#executive-summary)

1\. Introduction [1](#introduction)

1.1. Epidemiology and Distribution of Plague
[1](#epidemiology-and-distribution-of-plague)

1.2. The Surveillance/Observational Data
[4](#the-surveillanceobservational-data)

1.3. Integrated System Solution: ESD Inputs against ArboNET/PSS
Requirements [6](#__RefHeading___Toc22189583)

2\. Methods and Data. [8](#__RefHeading___Toc22189584)

2.1. Systems Engineering Approach [8](#systems-engineering-approach)

2.2. ArboNET/PSS and ESD data and models
[9](#arbonetpss-essd-data-and-models)

2.3. V&V measures [10](#vv-measures)

3\. Results [11](#results)

3.2. NASA ESD Models [12](#nasa-esd-models)

3.3. Communicating Risk [16](#communicating-risk)

4\. Conclusions [18](#conclusions)

4.1. Findings [18](#findings)

4.2. Recommendations [18](#recommendations)

4.3. Next Steps [18](#next-steps)

5\. References [19](#references)

Figures

Figure 1. World distribution of plague. Plague remains endemic in
natural foci around the world: large parts of Africa, Central and South
Asia, South America and Southwest US, particularly in the Four Corners
states. Source WHO. [1](#__RefHeading___Toc22189514)

Figure 2. Climate/Plague linkages. [5](#__RefHeading___Toc22189515)

Figure 3. Integrated system solution diagram for incorporation of Earth
science research results to enhance the Plague Surveillance System
component of CDC's ArboNET. [8](#__RefHeading___Toc22189516)

Figure 4. DEM and water bodies of the Four Corners states where most of
the plague cases have been reported. Source USGS EROS Data Center.
[10](#__RefHeading___Toc22189517)

Figure 5. Wild and domestic plague risk derived from control
environmental fake conditions: reported plague outbreaks, land cover,
NDVI, precipitation and elevation. [13](#__RefHeading___Toc22189518)

Figure 6. Correspondence between domestic risk patterns and temporal
plague outbreak patterns. Empirical mode decomposition shows a 5-year
cycle correspondence (green and red.) Reported cases by year are
normalized by 45. [15](#__RefHeading___Toc22189519)

Figure 7. NDVI and normalized anomaly assuming a t\^2 distribution. This
data is used to issue risk maps with one month of precedence.
[15](#__RefHeading___Toc22189520)

Figure 8. Domestic risk associated to the environmental conditions given
in Figure 7. Colors proportional to risk (0.5-blue  1-red) as in Figure
5. [17](#__RefHeading___Toc22189521)

# Tables {#tables .Heading-NoTOC}

Table 1. Integrated Plague Surveillance System. [3](#_Ref125201698)

Table 2**.** ArboNET/PSS requirements. [6](#__RefHeading___Toc22189531)

Table 3. ArboNET/PSS requirements and ESD data and models match.
[9](#__RefHeading___Toc22189532)

Table 4. Risk disaggregated by month. [14](#__RefHeading___Toc22189533)

Table 5. Interpretation of risk and uncertainty of environmental
conditions associated to plague risk in terms of common words used to
express degrees of danger. [17](#__RefHeading___Toc22189534)

#  {#section .Heading-NoTOC .unnumbered}

# Executive Summary {#executive-summary .Heading-TOC}

Members of the Global Inventory Modeling and Mapping Studies (GIMMS)
group at NASA Goddard Space Flight Center and CDC National Center for
Infectious Diseases, Division of Vector-borne Infectious Diseases in
Fort Collins, have conducted a follow up validation and verification
study of the integration of the NASA Earth Science Division (ESD)
results and models into the CDC\'s ArboNET/Plague Surveillance System
(PSS). This work was performed under NASA's Verification and Validation
(V&V) Program as a systematic review of the ESD suite that are more
readily available for integration. This V&V covers all datasets
recommended in the evaluation report of the entire ESD suite that may be
integrated as system inputs for addressing ArboNET/PSS requirements and
may improve its accuracy, quality, or efficiency. Many of these matches
are products derived from the Moderate Resolution Imaging
Spectroradiometer (MODIS) instrument flown on the Terra and Aqua
satellites of the EOS and from the Advanced Very High resolution
Radiometer (AVHRR) processed by the Global Inventory Modeling and
Mapping Studies (GIMMS) group at NASA Goddard Space Flight Center.

This is a first attempt to respond to the need for a better
understanding of climate/disease linkages with continuous spatial and
temporal environmental data that would complement "traditional
surveillance and response" systems with "prediction and prevention"
strategies[^1]. The study is focused on the risk areas of the four
corners states (New Mexico, Arizona, Utah and Colorado) identified in
the ArboNET/PSS evaluation report, and on the resulting new NASA monthly
environmental risk maps of plague for the region. The environmental risk
maps of plague are the result of the NASA/GSFC/GIMMS disease-ecoclimatic
monitoring system that integrates relevant ESD data[^2], i.e. 16-day
NDVI and 8-day temperature 1-km data from MODIS sensors, land cover
information from Landsat TM data, elevation data from Shuttle Radar
Topography Mission (SRTM), monthly precipitation data from Tropical
Rainfall Measuring Mission (TRMM), and 15-day NDVI 8-km data processed
by the GIMMS group from NOAA-Advanced Very High Resolution Radiometer
(AVHRR)[^3].

Effective plague prevention and control programs require up-to-date
information on the incidence and distribution of the disease. The best
means of gathering this information is through a surveillance program
that collects, analyses, and interprets clinical, epidemiological, and
epizootiological data on plague. A surveillance system is a key element
of decision support within the Centers for Disease Control and
Prevention (CDC). Surveillance should identify cases and epizootics as
quickly as possible so that steps can be taken to control disease
spread. The ArboNET/PSS is defined by guidelines issued by the World
Health Organization in collaboration with the CDC. The primary goal of
the ArboNET/PSS is to promptly identify changes in incidence either in
the form of an acute outbreak or in a change in long-term trends. At
present, plague is one of only three infectious diseases subject to the
International Health Regulations, which stipulate that all confirmed
cases of human plague be investigated and reported through appropriate
authorities to the World Health Organization. Whenever clinical symptoms
or laboratory results suggest that a patient is infected with *Yersinia
pestis*, the suspect case should be reported immediately. This will
allow public health authorities to: (1) advise on treatment and
management of human plague cases; (2) initiate efforts to identify the
source of infection; (3) determine the extent of any epizootic activity;
(4) assess the potential for additional human cases; (5) disseminate
information on plague to health care personnel; and (6) implement
emergency prevention and control measures. Thus, ArboNET/PSS uses in
essence traditional "surveillance and response" strategies.

CDC defines the success of a surveillance system in terms of a number of
factors including simplicity, flexibility, data quality, acceptability,
sensitivity, predictive value positive, timeliness, representativeness,
and stability. Clearly, sensitivity and predictive value are important
to measure the usefulness of the system. The other factors are also
critical since they can affect the performance of the ArboNET/PSS by
data reporting cost, level of integration with other systems, legal
assurance of privacy and confidentiality, staff training, or legal
requirements for data collection. The ArboNET/PSS includes human,
rodent, and vector surveillance since the appearance of early cases of
plague in a population is the most commonly used tool to prevent further
human infections. These "surveillance and response" approaches provide
fairly high predictive certainty of plague occurrence but often leave
public health authorities with little advance notice for mobilizing
actions to prevent further spread of the disease. In contrast,
ecological observations and climate forecasts can potentially be used in
efforts to predict the appearance of a pathogen and thus open the
opportunity to minimize its transmission. The ArboNET/PSS relies on
meteorological data, such as precipitation and temperature, to feed the
various modeling routines that are central to the surveillance system.
Much of this data was collected from systems implemented long before
NASA-ESD missions.

The results of this V&V study are part of the overall NASA/GIMMS
ArboNET/PSS enhancement effort that follows a rigorous NASA systems
engineering approach to ensure that the products selected for
implementation meet ArboNET/PSS requirements. It builds heavily from the
previous evaluation phase. The evaluation underpinned the assumption
made by both NASA and CDC agencies when entering into this partnership
-- that NASA ESD systems have the potential to enhance and complement
the plague "surveillance and response" system used by the CDC with
"prediction and prevention" strategies. Specifically, this study
undertook the following three tasks:

1.  Conduct an in-depth, critical review of the results of the
    NASA/GIMMS environmental risk maps of plague derived from the
    linkages between temporal and spatial variations of climate and the
    plague events in the four corners states;

2.  Examine the potential for establishing useful plague-oriented
    climate early-warning and surveillance systems, and for developing
    new benchmark metrics communicating and dealing with uncertainty in
    quantitative risk and policy analysis;

3.  Identify future metrics and data that could further clarify and
    quantify the effective use of the plague-oriented climate
    early-warning and surveillance system for societal responses to any
    such early warnings.

There are many substantial challenges associated with establishing a
useful plague-oriented climate early-warning and surveillance system.
For instance, the current strategies for controlling plague, or more
generally infectious diseases, depend largely on surveillance for new
outbreaks followed by a rapid response to control the epidemic. For
these "surveillance and response" approaches, risk analysis does not
need to deal with uncertainty and therefore it does not need to develop
correspondent metrics. For "prediction and prevention" approaches a key
limiting factor is the uncertainty associated to predictions. This
divergence in metrics for assessing uncertainty is usually overlooked
and limits the application of early warning systems. In this study, we
have considered carefully how users (decision makers) in public health
policy and response might use uncertainties in analyzing risk and have
implemented quantitative and qualitative tools to communicate
uncertainty[^4]. Recognition of the importance of uncertainties for risk
analysis can also improve communication by more clearly distinguishing
between uncertainty in whether, or how frequently, certain events or
outcomes might occur and uncertainty in their consequences. This
perspective shows that when faced with uncertainty in validating and
verifying an early warning system, it is not sufficient to identify only
the most likely outcome and compare its result, but to communicate it in
a common standardized language.

The results of this V&V demonstrate that NASA ESD assets could make
important contributions to ArboNET/PSS. The GIMMS group, using ESD
results, has implemented a progressive "prediction and prevention"
approach that could be used to issue an alert that environmental
conditions are conducive to plague outbreak which in turn can trigger
surveillance efforts for the area in question. This watch/warning
approach is analogous to the multi-stage early warning system used by
the U.S. National Weather Service. Although, it is likely to have much
lower predictive value, a benefit of this multi-stage approach is that
response plans can gradually ramp up as forecast certainty increases and
as surveillance data confirm earlier projections. The active
participation of the system's end users helps ensure that forecast
information is provided in a useful manner and that effective response
measures are developed. The input of CDC's stakeholders will ensure that
data and products produced for the NASA science community meet the data
delivery and continuity requirements of ArboNET/PSS, so that response
plans can be developed with realistic expectations. Finally, a recent
paper showed that the concerns about data continuity were mitigated in
the long-term by focusing integration efforts on the GIMMS-AVHRR NDVI
data set [^5] that is consistent with the ESD systems that have an
operational follow-on (such as the National Polar-orbiting Operational
Environmental Satellite System (NPOESS)) and planned system continuity
(such as the Global Precipitation Measurement (GPM) mission).

# Introduction

In this section, we described the characteristics of the ArboNET/Plague
surveillance system and its relationship with ESD data and models.

## Epidemiology and Distribution of Plague

Plague is one of three epidemic diseases still subject to the
International Health Regulations and notifiable to the World Health
Organization (WHO). The pathogen causing the disease *Yersinia pestis*
circulates in animal reservoirs, particularly in rodents, in the natural
foci of infection found on all continents except Australia. The natural
foci of plague are situated in a broad belt in the tropical and
sub--tropical latitudes and the warmer parts of the temperate latitudes
around the globe between the parallels 55 degrees North and 40 degrees
South (Error: Reference source not found). Plague remains endemic in
natural foci around the world: large parts of Africa, Central and South
Asia, South America, and the Soutwest US, particularly in the Four
Corners states. However, within these limits many areas are free of
natural foci of plague, such as desert areas with few or no rodents and
large areas of continuous forest, particularly in the tropics and high
glacier--covered mountain ranges.
![](media/image2.wmf){width="5.998611111111111in" height="4.325in"}

[]{#__RefHeading___Toc22189514 .anchor}Figure 1. World distribution of
plague. Plague remains endemic in natural foci around the world: large
parts of Africa, Central and South Asia, South America and the Soutwest
US, particularly in the Four Corners states. Source WHO.

Plague is transmitted between rodents and to other animals via wild
rodent fleas, cannibalism or (possibly) contaminated soil. *Wild plague*
exists in its natural foci independent of human populations and their
activity. *Domestic plague* is intimately associated with rodents living
with humans and can produce epidemics in both human and animal
populations. Humans are extremely susceptible to plague and may be
infected either directly or indirectly. Indirect transmission through
the bite of a flea is the most common route of transmission between
plague--infected rodents and humans. Human infection most frequently
occurs when an epizootic develops among rats in centers of human
population, following contact with infected wild rodents. Commensal rat
fleas, including plague--infected fleas, leave the bodies of rats killed
by plague seeking a blood meal from another host and may bite human
beings. Humans who contract the disease may subsequently become
infective to other people. However, plague is only occasionally
transmitted between humans, either through the bites of human fleas
(*Pulex irritans*) infected after biting patients in the septicemic
stage, or through direct contact between a healthy person with an
infected person. When primary bubonic plague develops into secondary
pneumonic plague, airborne transmission of the infective agent may take
place via the respiratory route, leading to primary pneumonic plague
among close contacts. Infection through direct contact with objects
contaminated with sputum from pneumonic plague patients can lead to the
development of bubonic plague. Effective treatment methods enable almost
all plague patients to be cured if diagnosed in time. The use of these
measures has led to a sharp reduction in the number of cases of plague
throughout the world. Today the distribution of plague coincides with
the geographical distribution of its natural foci. It is unlikely that
all the primary foci of plague have been discovered. Accordingly, close
examination should be given to any rural area in which repeated cases of
human plague occur. The prolonged absence of human cases in the vicinity
of a natural focus does not in itself mean that plague has disappeared.
If there is no evidence that plague has come from outside sources, the
disease must be sought among local wild rodents. On every continent,
primary natural foci of plague are connected with particular types of
landscape in which climatic conditions are favorable for a high and
stable number of rodent reservoirs and flea vectors of *Y. pestis*. Most
natural foci of plague, including the mountains, are found where there
is low annual precipitation, or where dry seasons inhibit the growth of
thick woody vegetation and lead to the formation of deserts,
semi--deserts and steppes (savannas, prairies, pampas and so on). Human
cases of plague are relatively sparse in natural foci. Cases occur among
people who come in contact with wild rodents in the course of their
work, hunting, or camping. The risk of human infection increases
significantly when plague penetrates populations of domestic rodents,
particularly rats of the genus *Rattus*.

[]{#_Ref125201698 .anchor}WHO recommends that health facilities use the
following surveillance case definition for reporting suspected cases of
plague - any person with painful swelling under the arms or in the groin
area; or in an area known to have plague, any person with cough, chest
pain and fever. For plague, a single suspected case is a suspected
outbreak and the case-based information should be reported immediately
to the district following the standard forms. Summary reports are due
monthly, even in those periods where zero cases were confirmed. Error:
Reference source not found summarizes the surveillance case definition
for reporting suspected/confirmed cases of plague.

Table 1. Integrated Plague Surveillance System.

+-------------+--------------------------------------------------------+
| **B         | -   Zoonotic systemic bacterial infection caused by    |
| ackground** |     *Yersinia pestis* (plague bacillus) usually        |
|             |     transmitted to humans by rodents and their fleas.  |
|             |                                                        |
|             | -   Main disease forms: bubonic, pneumonic, and        |
|             |     septicemic; large-scale epidemics may occur in     |
|             |     urban or rural settings.                           |
|             |                                                        |
|             | -   Incubation period is 1 to 7 days.                  |
|             |                                                        |
|             | -   Case fatality rate (CFR) may exceed 50-60% in      |
|             |     untreated bubonic plague and approaches 100% in    |
|             |     untreated pneumonic or septicemic plague, but is   |
|             |     usually \<1% with appropriate treatment.           |
|             |                                                        |
|             | -   Risk factor: rural residence. Exposure to infected |
|             |     populations of wild or domestic rodents and their  |
|             |     fleas.                                             |
+-------------+--------------------------------------------------------+
| **S         | -   Detect outbreaks of plague promptly. Verify        |
| urveillance |     etiology of all suspected non-outbreak-related     |
| Goal**      |     cases and the first 5 to 10 outbreak-related       |
|             |     cases.                                             |
+-------------+--------------------------------------------------------+
| **          | ***Suspected case***:                                  |
| Recommended |                                                        |
| case        | > Any person with sudden onset of fever, chills,       |
| d           | > headache, severe malaise, prostration and very       |
| efinition** | > painful swelling of lymph nodes, or cough with blood |
|             | > stained sputum, chest pain, and difficulty in        |
|             | > breathing.                                           |
|             |                                                        |
|             | ***Confirmed case***:                                  |
|             |                                                        |
|             | > Suspected case confirmed by isolation of *Yersinia   |
|             | > pestis* from blood or aspiration of buboes, or       |
|             | > epidemiologic link to confirmed cases or outbreak.   |
+-------------+--------------------------------------------------------+
| **Respond   | **If a single case is suspected:**                     |
| to alert    |                                                        |
| threshold   | -   Report case-based information to the next level.   |
| for**       |                                                        |
|             | -   Collect specimen for confirming the case.          |
| **epi       |                                                        |
| demic-prone | -   Investigate the case.                              |
| diseases**  |                                                        |
|             | -   Treat the patient with streptomycin, gentamicin or |
|             |     chloramphenicol, and administer chemoprophylaxis   |
|             |     of close contacts with tetracycline for seven days |
|             |     from time of last exposure.                        |
+-------------+--------------------------------------------------------+
| **Respond   | **If the suspected case is confirmed:**                |
| to action** |                                                        |
|             | -   Isolate patients and contacts of pneumonic plague  |
| **threshold |     with precaution against airborne spread (wear      |
| for**       |     masks, for example) until at least after 48 hours  |
|             |     of appropriate antibiotic therapy.                 |
| **epide     |                                                        |
| mic-prone** | -   Mobilize community to enable rapid case detection  |
|             |     and treatment, and to recognize mass rodent        |
| *           |     die-off as a sign of possible impending epidemic.  |
| *diseases** |                                                        |
|             | -   Identify high risk population groups through       |
|             |     person, place, and time analysis.                  |
|             |                                                        |
|             | -   Reduce sporadic and outbreak-related cases via     |
|             |     improved control or rodent populations (remove     |
|             |     trash, food sources, and rat harborages) and       |
|             |     protect against fleas with insect repellent on     |
|             |     skin and clothing and environmental flea control   |
|             |     (especially in homes and seaports and airports).   |
+-------------+--------------------------------------------------------+
| **Analyze   | **Time:** Graph monthly trends in cases and deaths.    |
| and**       | Construct epidemic curve for outbreak cases.           |
|             |                                                        |
| **interpret | **Place:** Plot the location of case households.       |
| data**      |                                                        |
|             | **Person:** Immediate case-based reporting of cases    |
|             | and deaths for routine surveillance. Count weekly      |
|             | cases and deaths for outbreaks. Analyze age            |
|             | distribution and assess risk factors to improve        |
|             | control of sporadic disease and outbreaks.             |
+-------------+--------------------------------------------------------+
| **          | Gage,1999                                              |
| Reference** |                                                        |
+-------------+--------------------------------------------------------+

As in most of the current public health systems, this surveillance
relies primarily on "surveillance and response" approaches to
controlling infections of plague. In contrast, recent progress in
climate forecasting and the availability of remote-sensing technologies
can potentially be used in efforts to provide early warnings of
conditions conducive to disease outbreak. In the next two sections, the
elements of such a system are described  ESD data and models, and their
integration into the current "surveillance and response" approach for
plague.

## The Surveillance/Observational Data

Since surveillance should identify cases and epizootics as quickly as
possible so that steps can be taken to control disease spread, early
warnings of a potential outbreak derived from observational climatic
conditions could be used for early detection of outbreaks, and analysis
of trends. As human cases of plague are relatively sparse in natural
foci, an enviro-climatic early warning system should differentiate
between *wild* and *domestic plague risk*. Implicit in an early warning
system is the uncertainty associated to the prediction and the
quantification of this risk. Continuous surveillance of climatic
conditions, rodent and vector populations, and their proximity to human
populations are important even during periods when no human cases are
reported since many years may elapse between the occurrence of isolated
cases or epidemics. In these cases, the ESD system integrated as part of
a systematic collection of surveillance information over many years will
provide information that can be used to:

1.  Predict areas where future human cases and rodent epizootics may
    occur;

2.  Identify the most common zoonotic sources of human infection;

3.  Identify particular environments and climatic conditions that are
    optimal for plague outbreaks;

4.  Assess the effectiveness of plague prevention and control measures;

5.  Identify local ecological factors or human activities that may
    result in increased plague exposure risks for humans; and

6.  Detect trends in the epidemiology and epizootology of plague in a
    given region.

A national update paper of modeling relationships between climate and
frequency of human plague cases in the Southwestern U.S found that
precipitation, temperature, humidity, land cover, vegetation, and
altitude are important factors to be analyzed in an early warning system
(Enscore et al., 2002.) Disease plague agents and their vectors each
have particular environments that are optimal for growth, survival,
transport, and dissemination (Figure 2.)

![](media/image3.jpeg){width="6.330555555555556in" height="8.0in"}

[]{#__RefHeading___Toc22189515 .anchor}Figure 2. Climate/Plague
linkages.

## Integrated System Solution: ESD Inputs against ArboNET/PSS Requirements 

It might seem a simple task to use ESD observations and models to create
predictive disease models. Unfortunately, the mathematical relationships
between climate and disease are neither so obvious nor so simple. In
particular, with the current Trophic Cascade Hypothesis (TCH) developed
to explain changing levels of human risk for zoonotic diseases
associated with climate variability in the US Southwest, the role of
ecological/environmental mechanisms by which human health is impacted by
climate variability is not well understood. In fact, infectious disease
patterns are affected by many factors other than climate, and the
relationship between climatic variations and disease outbreaks is often
mediated by ecological, biological, or societal changes. When the TCH
modeling system is applied to forecast plague outbreak conditions, two
different triggering mechanisms are observed. Depending on the region, a
bimodal (May and July) or unimodal (August) peak dynamic is found.
Rodents' movements also play an important role in plague dynamics,
complicating the system. Rodents can move down slopes, from higher
elevations to lower elevations; plague does the same. A summary of these
new requirements is given in Table 2.

  -------------- ---------------- ---------------- ----------------------------
  ARBONET/PSS                     Requirement      Drivers
  Property                                         

  Data Frequency Monthly          10 day, with     Need to establish trends
  (time step)    reporting        15-day summaries within 1-month cycle
                                                   (seasonal signature of
                                                   incidences)

                 Emergency        2 day            Time requirements of
                                                   emergency response planners

  Spatial        Monthly          Synoptic         Surveillance of
  Resolution     reporting        analysis: 8 km   enviro-climatic conditions
                                  or better        with better resolution than
                                                   stations

                                  Field-level      For high risk areas
                                  analysis: 30 m   
                                  or better        

                 Emergency        1 km or better   Breeding/resting/feeding
                                                   habitats

  General        Cost             Moderate         Budget constraints by region
  Properties                      resolution       
                                  product should   
                                  be free          

                 Format           Simple (web      Easy reporting
                                  based)           

                 Accessibility    Web accessible   Continue greater public
                                                   benefit leveraging being
                                                   demonstrated

                 Data validation  Validated and    Quality assurance
                                  science team in  
                                  place            

                 Data redundancy  Multiple sources Data continuity, convergence
                                                   of evidence

  Parameters     Precipitation    1 degree or      Cumulative precipitation of
                                  better           stand-alone interest and
                                                   assessment of vegetation

                 Min and max      1 degree or      Temperature changes affect
                 temperature      better           vectors and vegetation

                 Vegetation index 8 km or better   Needed for relative
                 numbers                           vegetation condition
                                                   assessments

                 Bare earth       5 km or better   Needed for rodent movement
                 elevation                         model

                 Soil moisture    Unknown at this  soil moisture of stand-alone
                                  time             interest in regard to flea
                                                   survival
  -------------- ---------------- ---------------- ----------------------------

[]{#__RefHeading___Toc22189531 .anchor}Table 2**.** ArboNET/PSS
requirements.

The GIMMS group has implemented a progressive "prediction and
prevention" approach that is being used to issue an alert that
environmental conditions are conducive to a plague outbreak which in
turn can trigger surveillance efforts for the area in question (Pinzon
et al., 2005a,b.) A reverse engineering paradigm was used to identify
the ESD data and models that match requirements and specifications of
the ArboNET/PSS given in Table 2. Figure 3 shows the GIMMS' Integrated
System Solution (ISS) for integrating ESD results into ArboNET/PSS. Part
of this system is the integration of the epidemiological surveillance
data provided by CDC/NCID, such as proximity of infected rodents and
fleas to human dwellings or workplaces and estimated number of people
involved in activities that place them at high risk of plague infection.
Reports of these investigations include: 1) a complete history of the
patients\' activities and travel during the incubation period of the
infection; 2) results of field studies to determine which animal and
flea species are likely sources of infection or pose a continuing threat
to humans (surveillance techniques for rodents and fleas can be found in
Gage, 1999); 3) proximity of infected rodents and fleas to human
dwellings or workplaces; 4) estimated number of people involved in
activities that place them at high risk of plague infection; and 5)
information on possible exposure to *Y. pestis* infection from patient
contacts (especially important for pneumonic plague cases).

In the United States, rapid sub--urbanization has resulted in increasing
numbers of people living in or near active plague foci. During each
successive decade from 1944 to 1993, the number of states reporting
plague cases increased: from three in 1944--1953 to thirteen in
1984--1993. Surveillance of plague in rodent and rodent--consuming
carnivore populations indicates that plague spread eastward in the 1990s
to areas believed to have been disease--free since extensive animal
surveillance began in the 1930s. Two hundred and forty seven human
plague cases were reported in the United States during 1980--2000, the
highest of any 20--year period since the epidemic years in the early
part of the century. Thirty seven of these patients died \-- a case
fatality rate of 15. The number of cases by year during this 20--year
period ranged from one indigenous case in 1990 to highs of 40 in 1983
and 31 in 1984.

The NASA/GSFC/GIMMS disease-ecoclimatic monitoring system was customized
according to the PSS operations to be more in-line with their CDC
modeling systems. The MODIS-AVHRR NDVI datasets integration
(complemented by other remote sensing observation products) are
highlighted as part of the system. Thus, the epidemiological data
combined with products derived from Earth observing instruments,
technology, and science information that consider not only vegetation
dynamics, but also elevation, slope, aspect, and land cover diversity in
their models will provide information on plague vector habitats. This
will enable ArboNET to better characterize wild and domestic plague
outbreak conditions and will enhance the plague surveillance system.

![](media/image4.png){width="6.479166666666667in"
height="5.135416666666667in"}

[]{#__RefHeading___Toc22189516 .anchor}Figure 3. Integrated system
solution diagram for incorporation of Earth science research results to
enhance the Plague Surveillance System component of CDC's ArboNET.

# Methods and Data.

## Systems Engineering Approach

This validation and verification (V&V) phase is part of the systems
engineering approach followed by the GIMMS group for the ArboNET/PSS
which entails three phases: evaluation, verification and validation
(V&V), and benchmarking. In terms of this approach, the purpose of this
report is to verify and validate that the NASA ESD data and models
integrated into the ArboNET/PSS provide the early warning capabilities
needed to enhance the system.

As expected, this process has not been strictly linear. It uses a
Reverse Engineering Paradigm to calibrate NASA ESD measurements and
models to ensure that all factors that affect the success of the
system - sensitivity, predictive value positive, timeliness,
acceptability, simplicity, flexibility, data quality, stability,
continuity, and representativeness -- are for improvement of
socioeconomic benefits and adaptation or adoption by the CDC. This
process inherently requires interdisciplinary collaboration and must
progress in a coordinated fashion with an active participation of the
ArboNET/PSS system's end users.

## ArboNET/PSS & ESSD data and models 

Table 3 summarizes all ESD inputs used in the GIMMS' Integrated System
Solution (ISS) approach and the validation and verification steps
followed in this report. The NDVI-AVHRR dataset, created by the GIMMS
group, is of particular interest because it is the first AVHRR 8-km NDVI
time series that covers the period from July 1981 to the present and it
is compatible with the NDVI from more modern instruments, e.g. MODIS and
SPOT Vegetation (Pinzon et al. 2005c, Fensholt et al. 2006.) Since most
of the plague activity in the USA has occurred in the 1980s and 1990s,
the NDVI-AVHRR data set is an essential part of the GIMMS' ISS to study
the environmental factors that often set the stage for optimal
conditions for the spread of plague and are important in determining its
transmission, intensity and duration.

[]{#__RefHeading___Toc22189532 .anchor}Table 3. ArboNET/PSS requirements
and ESD data and models match.

+-----------------+--------------+------------------------------------+
| PSS Required    | ESD data and | Validation and Verification        |
| Observation &   | models       |                                    |
| Predictions     |              |                                    |
+-----------------+--------------+------------------------------------+
| > High/Moderate | AVHRR &MODIS | -   15-day global coverage → AVHRR |
| > Resolution    |              |     (global coverage)              |
| > Visible-IR    |              |                                    |
| > Remote        |              | -   8-km spatial resolution →      |
| > Sensing       |              |     MODIS 250 m to 1 km            |
| > Information & |              |                                    |
| > Products      |              | > → AVHRR 8 km                     |
+-----------------+--------------+------------------------------------+
| a.  Moderate    | -   MODIS    | -   Continuity from 1981 data      |
|     Resolution  |     NDVI     |     (AVHRR)                        |
|     Vegetation  |              |                                    |
|     Indices     | -   AVHRR    | -   Temporal Domain: The           |
|                 |              |     NASA/GIMMS cross-validation    |
|                 | NDVI/GIMMS   |     work among the various AVHRR   |
|                 |              |     sensors and MODIS has given    |
|                 |              |     confidence in the long-term    |
|                 |              |     NDVI data quality and          |
|                 |              |     important as a transition to   |
|                 |              |     the next generation moderate   |
|                 |              |     resolution sensor. It favors   |
|                 |              |     efforts on the understanding   |
|                 |              |     of disease/climatic linkages   |
|                 |              |     and their application to early |
|                 |              |     warning systems.               |
|                 |              |                                    |
|                 |              | -   Spatial Domain: MODIS          |
|                 |              |     characterization of current    |
|                 |              |     enviro-climatic conditions.    |
+-----------------+--------------+------------------------------------+
| a.  Temperature | -   MODIS    | -   Measured land surface          |
|     (Min and    |     Land     |     temperature as opposed to      |
|     Max)        |     Surface  |     current interpolated           |
|                 |              |     temperatures                   |
|                 |  Temperature |                                    |
+-----------------+--------------+------------------------------------+
| a.  Topography  | -   SRTM     | -   SRTM30 for North America is    |
|                 |              |     available since 2003.          |
+-----------------+--------------+------------------------------------+
| a.              | -   TRMM     | -   Precipitation reaching the     |
|   Precipitation |              |     ground measured by radar as    |
|                 |              |     opposed to current             |
|                 |              |     interpolated WMO stations.     |
+-----------------+--------------+------------------------------------+
| a.  Land        | -   AVHRR    | -   Useful to ArboNET/PSS          |
|     Surface     |     Land     |     identification of rodent       |
|     type        |     Cover    |     habitat, Regions with high     |
|                 |              |     risk use Landsat for better    |
|                 |              |     interpretation of land cover   |
|                 |              |     mixing.                        |
+-----------------+--------------+------------------------------------+

## The GIMMS group also developed a t^2^-normalized anomaly product based on the AVHRR-NDVI data set to better characterize the disease/climatic linkages. The integration of this data set with the higher resolution and better quality NDVI data from MODIS is used to display current climatic/environmental conditions. The NDVI AVHRR/GIMMS and MODIS data sets are also important as a transition to the next generation moderate resolution sensors: the Visible Infrared Imaging Radiometer Suite (VIIRS) that will be part of the National Polar-orbiting Operational Environmental Satellite System (NPOESS) and the NPOESS Preparatory Project (NPP) missions.

##  V&V measures

This study reports V&V measures for the Four Corners region (figure 4.)

[]{#__RefHeading___Toc22189517
.anchor}![](media/image5.jpeg){width="6.0in"
height="4.405555555555556in"}Figure 4. DEM and water bodies of the Four
Corners states where most of the plague cases have been reported. Source
USGS EROS Data Center.

The V&V measures are: 1) correlations between plague risk warnings
derived from the ISS system and reported outbreak cases; 2) time series
analysis of risk maps; 3) correspondence between risk and plague
seasonality; 4) correlation between wild and domestic plague risk; 5)
correspondence between risk maps and regions with reported plague; and
6) assessment of the uncertainty associated to the prediction of risk
maps at current enviro-climatic conditions and characterization of
predominant vegetation types. This includes the amount of local land
surface covered by each vegetation type, roads, railways, airports, and
seaports, land use patterns (agricultural, residential, industrial,
other), types of dwellings present, and whether these dwellings and
associated food storage areas or other man-made sites provide food and
harborage for rodents.

# Results

1.  **ESD Data Sets**

#### NDVI AVHRR

As discussed earlier, the NASA/GIMMS NDVI archive is a vital data stream
for ArboNET/PSS. The NASA/GIMMS cross-validation work among the various
AVHRR sensors has given confidence in the long-term data quality and has
allowed the group to focus more efforts on the disease/climate linkages.
In a recent evaluation of the 8 km NASA/GIMMS NDVI data set, Rasmus et
al. (Rasmus, 2006) showed that overall the accuracy of the data set is
consistent to the SPOT-4 VGT, a more accurate and recent sensor,
explaining most of the SPOT-4 VGT variance. They concluded that this
data set can be considered the most accurate long time AVHRR data
record, and should be used for analyses of long-term trends in regional
or continental scale NDVI. Pinzon et al. (Pinzon, 2005a,b) used the
t^2^-normalized anomaly product based on this AVHRR-NDVI data set to
better characterize the disease/climatic linkages. In particular, their
results showed high consistency with plague cases in the Four Corners
region (Pinzon, 2005b.)

#### MODIS standard products (NDVI, temperature) 

The MODIS Standard Products (NDVI and land surface temperature) were
used to characterize current conditions of regions of high risk at 1km
resolution based on the risk maps derived from NASA/GIMMS AVHRR-NDVI.
The data is integrated, presented, and accessible in secure web
environments and GIS tools. The NDVI vegetation index and the surface
temperature products met ArboNET/PSS requirements (8km) and therefore
were an upgrade to the data quality and continuity (radiometric and
spatial) that were derived from the surface weather stations.

#### TRMM precipitation

Currently ArboNET/PSS's precipitation information comes from WMO weather
stations or from precipitation estimates. The weather station
information is limited to point estimates, which are sparse in some
regions.

TRMM calibrates its passive microwave sensor, the TRMM Microwave Imager
(TMI) with simultaneous radar: the Precipitation Radar (PR). The radar
can determine if precipitation is reaching the ground; in fact, it gives
three-dimensional information about rain cloud structure. It is not
feasible to achieve broad coverage with radar, but it can serve as a
calibration source for the passive microwave sensor. This calibrated
data can then be cross-calibrated with passive microwave sensors on
other platforms to create a global precipitation measurement
constellation. This constellation is the concept of the Global
Precipitation Measurement (GPM) mission for which TRMM is a precursor. A
research group at NASA/GSFC has developed a cross-calibration of the
TRMM TMI data with other microwave sensors, namely the DMSP SSM/I
sensors to create a level 3 TRMM product (3B42RT), the "TRMM Real-Time
Multi-Satellite Precipitation Analysis", with some completeness and
consistency. This product is being delivered online at
<http://daac.gsfc.nasa.gov/CAMPAIGN_DOCS/hydrology/TRMM_analysis.html>

The TRMM is used to complement current climatic conditions and improved
risk prediction from the NASA/GIMMS AVHRR-NDVI.

#### Land Cover Characterization

The system uses the AVHRR 8km global land cover data set to characterize
the land cover of natural foci around the world where plague remains
endemic and to facilitate the extension worldwide of the prototype being
developed for the Four Corners states of the U.S. Landsat is used to
provide a high resolution data set for better interpretation of the land
cover map at identified (predicted) high risk areas.

#### Bare Earth Elevation

The new global dataset is based primarily on SRTM30 data. The SRTM30 is
a near-global digital elevation model (DEM) comprising a combination of
data from the SRTM mission and the U.S. Geological Survey's GTOPO30 data
set. It can be considered to be either an SRTM data set enhanced with
GTOPO30, or as an upgrade to GTOPO30. It is used to characterized
movement of rodents (and thus plague wave.) It provides (static)
environmental conditions useful for a better characterization of rodent
niche (plague foci.)

## NASA ESD Models

The NASA/GSFC/GIMMS disease-ecoclimatic monitoring system uses the land
cover map, the SRTM30 data, precipitation profiles, and NDVI profiles to
characterize the regions more susceptible to plague outbreaks. For
security reasons, we use fake data in this report for this
characterization due to the International Health Regulations that plague
is still subject. Wild and domestic risk are derived from this
characterization (Figure 5.) We repeat the same computations with actual
data with similar results. Thus, the results of this V&V measure applied
to the control fake data is a genuine representation of what the disease
ecoclimatic monitoring system would predict when applied to actual data.

The risk is derived according to the similarity between the (fake)
environmental conditions at any point and the temporal and spatial
environmental characteristics of the regions with (fake) outbreaks
reports. The V&V measures summarize temporal and spatial correspondence
between plague outbreak and predicted plague risk regions. The domestic
risk is derived from the predicted wild risk map by computing (fake)
distance to populated areas. About 27% percent of continental USA has
environmental conditions similar to outbreak areas. However, less than
35% percent of those regions have "optimal conditions" for plague, i.e.
they have high risk. In the disease-ecoclimatic monitoring system, the
conditions are also evaluated by month. Integrating the risk associated
to each point by month we can estimate the risk of plague by month.
Table 4 show this risk disaggregated by month.

![](media/image6.jpeg){width="5.25in"
height="4.0in"}![](media/image7.jpeg){width="5.083333333333333in"
height="4.0in"}

[]{#__RefHeading___Toc22189518 .anchor}Figure 5. Wild and domestic
plague risk derived from control environmental fake conditions: reported
plague outbreaks, land cover, NDVI, precipitation and elevation.

[]{#__RefHeading___Toc22189533 .anchor}Table 4. Risk disaggregated by
month.

  -------------- ------------- ----------------- -------------- -------------
  **Month**      **Wild**                        **Domestic**   

                 **Percent**   **Avg. risk**     **Percent**    **Avg. risk**

  January        2             48                0              26

  February       3             48                1              24

  March          5             46                2              25

  April          19            52                6              26

  May            38            52                21             29

  June           39            53                21             30

  July           41            54                15             30

  August         43            55                20             30

  September      44            54                23             30

  October        38            54                22             31

  November       12            55                5              29

  December       4             52                0              22
  -------------- ------------- ----------------- -------------- -------------

Wild and domestic plague are related to each other by identifying the
months of May through October as the period with higher percentage risk.
However, summing up all individual risk to obtain average risks, wild
and domestic risks have a notable difference. While domestic risk still
identifies May-November, wild risk extends from April to December. The
correlation between the percent area of domestic risk and the average
domestic risk is 0.99. This relation allows the use of percent or
average domestic risk indifferently. Figure 6 shows a comparison between
the predicted average domestic risk lagged by one month and the average
behavior of reported plague outbreaks. Their similarity in profiles and
in the time series analysis show that environmental and climatic
features can be successfully used as an early warning system of plague
risk with one month of precedence. Here, empirical mode decomposition
(Pinzon et al. 2005c) is applied to the domestic risk average to
identify trends in the time series. A 5-year cycle in plague outbreaks
and predicted domestic risk is apparent from the two EMD modes shown.
This reaffirms the use of the NASA/GSFC/GIMMS disease-ecoclimatic
monitoring system as an accurate early warning system for the
ArboNET/PSS.

The NASA/GSFC/GIMMS disease-ecoclimatic monitoring system is also
applied to generate monthly warnings. It uses NDVI from AVHRR data at
8km resolution and the derived normalized anomaly (Figure 7). The AVHRR
data is used for continuity purposes since it covers 26 years
(1981-present) and it is global. The risk maps are transformed as
dynamic GIS layers and they are combined with other ESD data (MODIS
NDVI-higher resolution, temperature-higher resolution, SRTM, and other
ancillary data) for better interpretation of the risk and, therefore,
better supporting tools for decision making.

![](media/image8.jpeg){width="6.0in" height="4.3875in"}

[]{#__RefHeading___Toc22189519 .anchor}Figure 6. Correspondence between
domestic risk patterns and temporal plague outbreak patterns. Empirical
mode decomposition shows a 5-year cycle correspondence (green and red.)
Reported cases by year are normalized by 45.

![](media/image9.jpeg){width="5.75in" height="3.2875in"}

[]{#__RefHeading___Toc22189520 .anchor}Figure 7. NDVI and normalized
anomaly assuming a t\^2 distribution. This data is used to issue risk
maps with one month of precedence.

## Communicating Risk

In general settings, danger and risk are near synonyms. However, these
two words have two different meanings: "danger" is the qualitative
description of an effect that can cause harm or injury, while "risk" is
a quantitative measure of the probability of a situation involving
exposure to danger. In general, it is possible to reduce risk when
appropriate actions are taken. People tend to be particularly resistant
to the idea that they are at risk from any particular hazard. They tend
to feel infallible; they feel they are in less danger than the average
with a lower than average likelihood of suffering the effects of the
exposure to danger. This influences people's response to risk. The
message "this includes you" is more difficult to get across than "many
will die." Thus, perception is an important factor to be taken into
account when communicating risks. Research by sociologists and
anthropologists has shown that risk perception and its acceptance are
rooted in cultural and social factors. It is not different in the case
of communicating plague risk.

It is important to transmit the environmental information along other
information relevant to risk management about the magnitude of the
plague risk so that people may become aware of this connection.
Information about personal vulnerability is important for the transition
from awareness to the decision to act. However, the decision to act is
not the same thing as acting.

There are many substantial challenges associated with establishing a
useful plague-oriented climate early-warning and surveillance system.
The communication of early plague risk based on environmental conditions
is an important factor for establishing such a system. Another important
factor is communication of uncertainty associated to the environmental
risk. For instance, the current strategies for controlling plague, or
more generally infectious diseases, depend largely on surveillance for
new outbreaks followed by a rapid response to control the epidemic. For
these "surveillance and response" approaches, risk analysis does not
need to deal with uncertainty and therefore it does not need to develop
correspondent metrics. For "prediction and prevention" approaches a key
limiting factor is the uncertainty associated to predictions. This
divergence in metrics for assessing uncertainty is usually overlooked
and limits the application of early warning systems. In this study, we
have considered carefully how users (decision makers) in public health
policy and response might use uncertainties in analyzing risk and have
implemented quantitative and qualitative tools to communicate
uncertainty[^6]. Recognition of the importance of uncertainties for risk
analysis can also improve communication by more clearly distinguishing
between uncertainty in whether, or how frequently, certain events or
outcomes might occur and uncertainty in their consequences. This
perspective shows that when faced with uncertainty in validating and
verifying an early warning system, it is not sufficient to identify only
the most likely outcome and compare its result, but to communicate it in
a common standardized language (Figure 8 and Table 5.)

![](media/image10.jpeg){width="6.0in" height="3.4375in"}

[]{#__RefHeading___Toc22189521 .anchor}Figure 8. Domestic risk
associated to the environmental conditions given in Figure 7. Colors
proportional to risk (0.5-blue  1-red) as in Figure 5.

Studies in risk communication during the past 20 years have recognized
that their evolution has passed through seven chronological phases
(Fischhoff, 1987, 1995.) The successive stages build on one another, but
they do not replace one another:

i.  All we have to do is get the numbers right (Figures 4-8.)

ii. All we have to do is tell them the numbers.

iii. All we have to do is explain what we mean by the numbers (Table 5.)

iv. All we have to do is show them that they've accepted similar risks
    in the past.

v.  All we have to do is show them that it's a good deal.

vi. All we have to do is make them our partners.

vii. All of the above.

[]{#__RefHeading___Toc22189534 .anchor}Table 5. Interpretation of risk
and uncertainty of environmental conditions associated to plague risk in
terms of common words used to express degrees of danger.

  ---------------------------- ------------------------------------------
  Risk range                   Quality description

  \>0.9                        Probable

  0.85-0.9                     Good Chance

  0.7-0.84                     Possible

  0.5-0.7                      Unlikely

  \<0.5                        Improbable
  ---------------------------- ------------------------------------------

This V&V shows that the early-warning system is in phase (iii). In the
benchmark, it should obtain phases (iv) through (vii.)

# Conclusions

A systematic validation and verification of the ESD suite of
measurements against the ArboNET/PSS requirements showed that they add
value as input requirements of the ArboNET/PSS. However, there are still
many substantial challenges associated with establishing a useful
plague-oriented climate early-warning and surveillance system. In
particular, tools from risk communication research should be included in
the following benchmark phase from the System Engineering approach
followed in this project.

## Findings

The results of this V&V demonstrate that NASA ESD assets make important
contributions to ArboNET/PSS. The GIMMS group, using ESD results, has
implemented a progressive "prediction and prevention" approach that is
being used to issue an alert that environmental conditions are conducive
to plague outbreak which in turn can trigger surveillance efforts for
the area in question. This alert has been translated in terms of
qualitative descriptions that are used in common language to facilitate
the communication of plague risk. These alerts should be used and
integrated along with other information into the set of tools available
for the decision-making process of plague risk management, policy
establishment, and hazard identification.

## Recommendations

This V&V report reinforces the findings of the previous evaluation
report which underpins the assumption that ESD systems have the
potential to enhance and complement the plague "surveillance and
response" system used by the CDC with "prediction and prevention"
strategies. The overall ArboNET/PSS enhancement effort should continue
to be managed through a systems engineering approach so that verified
and validated ESD products result in improvements to the ArboNET/PSS
system and so they can be effectively benchmarked. The active
participation of CDC ArboNET stakeholders should continue so that data
and products produced by the NASA science community meet the data
delivery and continuity requirements of ArboNET/PSS. Still, a prudent
strategy to reduce people's overall vulnerability is to continue with
strong public health measures such as epidemiological surveillance and
prevention measures.

## Next Steps

Having verified and validated ESD products and models against the
ArboNET/PSS requirements \-- making sure the quantitative risk numbers
are correct and then translating them into a common language used for
interpretation of risk \-- the next part of the systems engineering
process is benchmarking. In the benchmarking phase, special attention
should be taken to the integration of these environmental alerts of
plague risk along other information relevant to risk management so that
people may become aware of the connection to personal vulnerability.
This is important for the transition from awareness to the decision to
act. However, it should be noted that the decision to act is not the
same thing as acting.

Work will continue with the active participation of CDC researchers to
benchmark the integration of NASA products into ArboNET/PSS.

# References

Enscore, Rusell E, BJ Biggerstaff, TL Brown, RF Fulgham, PJ Reynolds, DM
Engelthaler, CE Levy, RR Parmenter, JA Montenieri, JE Cheek, RK
Grinnell, PL Ettestad, and KL Gage,. 2002. Modeling relationships
between climate and the frequency of human plague cases in the
Southwestern United States, 1960-1997, *American Journal of Tropical
Medicine and Hygiene*, 66(2): 186-196.

Fensholt, Rasmus, TT Nielsen, and S Stisen, 2006. Evaluation of AVHRR
PAL and GIMMS 10-day composite NDVI time series products using SPOT-4
vegetation data for the African continent. *International Journal of
Remote Sensing,* 27(13): 2719-2733.

Fischhoff, B. 1987. Treating the public with risk communication: A
public health perspective. *Science, Technology & Human Values*, 12 (3-
4), 13- 19.

Fischhoff B. 1995. Risk perception and communication unplugged: Twenty
years of progress. *Risk Analysis*, 15(2):137-145.

Gage, Kenneth L, 1999. Plague Surveillance, *Plague Manual:
Epidemiology, Distribution, Surveillance and Control.*
WHO/CDS/CSR/EDC/99.2, Geneva, 1999,135-170.

Morgan, MG, and M Henrion (1990). *Uncertainty: A guide to Dealing with
Uncertainty in Quantitative Risk and Policy Analysis*, Cambridge
University Press, Cambridge, UK.

Pinzon, Jorge E., JM Wilson, and CJ Tucker, 2005a. Climate-based health
monitoring systems for eco-climatic conditions associated with
infectious diseases. *Bulletin de la Societe de Pathologie Exotique,*
98(3): 239-243.

Pinzon, Jorge E, CJ Tucker, KL Gage, RE Enscore, and RJ Eisen, 2005b.
Integration of NASA data into ArboNET/Plague Surveillance System. *U.S.
Climate Change Science Program (CCSP) Workshop*, "Climate Science in
Support of Decision Making," November 14-16, 2005,
<http://www.climatescience.gov/workshop2005/posters/P-AQ1.10_Pinzon.pdf>

Pinzon, Jorge E, ME Brown, and CJ Tucker, 2005c. EMD Correction of
Orbital Drift Artifacts in Satellite Data Stream, Chapter 8, Part II in
*Hilbert-Huang Transform and and its Applications* Ed. N. E. Huang and
S. S. Shen, World Scientific, Singapore.

Pinzon, Jorge E, CJ Tucker, KL Gage, RE Enscore, and RJ Eisen, 2006.
Evaluation Report for ArboNET/Plague Surveillance System. NASA, January
2006.

Tucker, CJ, JE Pinzon, ME Brown, D Slayback, EW Pak, R Mahoney, E
Vermote and N El Saleous. 2005. An Extended AVHRR 8-km NDVI Data Set
Compatible with MODIS and SPOT Vegetation NDVI Data. *International
Journal of Remote Sensing*, Vol. 26 (20), 4485-4498.

[^1]: Pinzon, JE, CJ Tucker, KL Gage, RE Enscore, and RJ Eisen (2006).
    ArboNET/Plague surveillance system: Evaluation report, Jan/2006.

[^2]: Pinzon JE, Wilson JM & Tucker CJ Climate-based health monitoring
    systems for eco-climatic conditions associated with infectious
    diseases/Systèmes de surveillance de santé des maladies
    infectieuses, basés sur les conditions climatiques*.* *Bulletin de
    la Societe de Pathologie Exotique*, T.98 (3), 2005.

[^3]: Pinzon, JE, ME Brown, and CJ Tucker, 2005. EMD Correction of
    Orbital Drift Artifacts in Satellite Data Stream, Chapter 8, Part II
    in *Hilbert-Huang Transform and and its Applications* Ed. N. E.
    Huang and S. S. Shen, World Scientific, Singapore.

    Tucker, CJ, JE Pinzon, ME Brown, D Slayback, EW Pak, R Mahoney, E
    Vermote and N El Saleous. 2005. An Extended AVHRR 8-km NDVI Data Set
    Compatible with MODIS and SPOT Vegetation NDVI Data. *International
    Journal of Remote Sensing*, Vol. 26 (20), 4485-4498.

[^4]: Morgan, MG, and M Henrion (1990). *Uncertainty: A guide to Dealing
    with Uncertainty in Quantitative Risk and Policy Analysis*,
    Cambridge University Press, Cambridge, UK.

[^5]: Fensholt, Rasmus, TT Nielsen, and S Stisen, 2006. Evaluation of
    AVHRR PAL and GIMMS 10-day composite NDVI time series products using
    SPOT-4 vegetation data for the African continent. *International
    Journal of Remote Sensing,* 27(13): 2719-2733.

[^6]: Morgan, MG, and M Henrion (1990). *Uncertainty: A guide to Dealing
    with Uncertainty in Quantitative Risk and Policy Analysis*,
    Cambridge University Press, Cambridge, UK.