United States Government Accountability Office

GAO Report to Congressional Requesters

DARFUR CRISIS

Death Estimates 

Demonstrate Severity 

of Crisis, but Their 

Accuracy and 

Credibility Could Be 

Enhanced 

November 2006 

 

GAO-07-24 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

November 2006

DARFUR CRISIS

Death Estimates Demonstrate Severity of 

Crisis, but Their Accuracy and Credibility 

Could Be Enhanced 

Highlights of GAO-07-24, a report to 

congressional requesters 

In 2003, violent conflict in Darfur, 

Sudan, broke out between rebel 

groups and government troops and 

government-supported Arab 

militias. While few would dispute

that many thousands of Darfur

civilians have died, less consensus 

exists about the total number of 

deaths attributable to the crisis. 

Estimates by the Department of 

State (State) and other parties 

report death tolls up to about

400,000 for varying populations and

periods of time between February

2003 and August 2005. Based on the

views of experts convened by GAO 

and the National Academy of 

Sciences, interviews with estimate 

authors, and a review of relevant

literature, this report (1) evaluates 

six Darfur death estimates, (2)

identifies general challenges to 

estimating deaths in such crises, 

and (3) discusses measures to 

improve death estimates. 

What GAO Recommends

To safeguard the U.S. government’s 

credibility as a source of reliable

death estimates, GAO recommends 

ensuring greater transparency

regarding the data and methods 

used for such estimates. GAO also 

recommends that the U.S. 

government consider other

measures suggested by the experts 

to help address gaps in data and

improve the quality of any future

death estimates. State and the U.S.

Agency for International 

Development agreed with GAO’s 

recommendations. 

The experts GAO consulted did not consistently rate any Darfur death 

estimate as having a high level of accuracy; moreover, they noted that all the

studies had methodological strengths and shortcomings. Most of the experts 

had the highest overall confidence in estimates by the Centre for Research 

on the Epidemiology of Disasters (CRED) and had a slightly lower level of 

confidence in State’s estimate. Many experts believed State’s lower-end 

estimate was too low. Additionally, the published documents describing 

State’s estimate lacked sufficient information about its data and methods to 

allow it to be replicated and verified by external parties. 

Estimating deaths in a humanitarian crisis such as that in Darfur involves 

numerous challenges. For example, in Darfur, difficulties in collecting 

mortality data, such as lack of access to particular geographical regions, 

impacted the data’s quality and led to data gaps. Because of such data gaps, 

some Darfur death estimates relied on potentially risky assumptions and 

limited contextual information. Further, limitations in estimates of Darfur’s 

population before and during the crisis may have led to over- or

underestimates of the death toll. Finally, varying use of baseline mortality 

rates—the rate of deaths that would have occurred without the crisis—may 

have led to overly high or low death estimates. 

The experts proposed and rated a wide range of measures that U.S. agencies 

could take to improve the quality and reliability of death estimates for Darfur 

and future humanitarian crises. Among these measures, the most highly

rated was ensuring that public documentation of the data and methods used 

contain sufficient information to enable external replication and verification 

of the estimates. Other very highly rated measures include collecting and 

maintaining data for specific periods of time and geographic areas and 

housing the responsibility for making estimates in a reputable independent 

body. 

Darfur Village Burning 

Source: United States Holocaust Museum. Photograph by Brian Steidle.

www.gao.gov/cgi-bin/getrpt?GAO-07-24.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact David Gootnick 

at (202) 512-3190 or gootnickd@gao.gov. 
Contents 

Letter 1

Results in Brief 3 

Background 5

Some Death Estimates Judged More Accurate and 

Methodologically Stronger, Despite Shortcomings 19 

Estimating Deaths in Humanitarian Crises Involves Many 

Challenges 27

Wide Range of Measures Could Improve Death Estimates for 

Darfur and Future Humanitarian Crises 34 

Conclusions 41

Recommendations for Executive Action 42 

Agency Comments and Our Evaluation 42 

Appendix I Objectives, Scope, and Methodology 44 

Appendix II List of Experts and Meeting Agenda 48 

Appendix III List of Death Estimates and Mortality Surveys 

Provided to Experts and Additional 

Bibliographical References 50 

Appendix IV Summary Description of Death Estimates 

Reviewed 56 

Appendix V Additional Follow-Up Survey Results 62 

Appendix VI Comments from the Department of State 64 

GAO Comments 67 

Appendix VII Comments from the U.S. Agency for International 

Development 68 

Page i GAO-07-24 Darfur
Appendix VIII Summary of Authors’ Comments 69 

Appendix IX GAO Contacts and Staff Acknowledgments 71 

Tables 

Table 1: Accuracy of Darfur Death Estimates Rated by Experts 20 

Table 2: Measures Rated by Experts as Likely to Greatly Improve 

Death Estimates for Future Crises, in Order of Ranking 

and Number of Endorsements 35 

Figures 

Figure 1: Map of Sudan 6 

Figure 2: Map of Destroyed and Damaged Darfur Villages, as of 

February 2005 8 

Figure 3: Example of Generation of a Death Estimate 14 

Figure 4: Death Estimates Based on Reported Figures and Time 

Included 17

Page ii GAO-07-24 Darfur
Abbreviations 

CDC Centers for Disease Control and Prevention 

CE-DAT Complex Emergency Database 

CIJ Coalition for International Justice 

CMR crude mortality rate 

CRED Centre for Research on the Epidemiology of Disasters 

IDP internally displaced persons 

JEM Justice and Equality Movement 

NGO nongovernmental organization 

SLM/A Sudan Liberation Movement/Army 

SMART Standardized Monitoring and Assessment of Relief and 

 Transitions 

USAID U.S. Agency for International Development 

UN United Nations 

WHO World Health Organization 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 

Page iii GAO-07-24 Darfur
United States Government Accountability Office

Washington, DC 20548 

November 9, 2006 

The Honorable Tom Lantos 

Ranking Minority Member 

Committee on International Relations 

House of Representatives 

The Honorable Mike DeWine 

The Honorable Richard J. Durbin 

United States Senate 

In 2003, violent conflict broke out in the Darfur region of western Sudan 

when rebel groups, believing that the region was marginalized by the 

Sudanese government, led attacks against the government. In response, 

the government armed and supported local Arab and tribal militias, 

commonly known as the Janjaweed, to defeat the rebels. Attacks on the 

civilian population by these militias, sometimes in conjunction with the 

Sudanese army, have resulted in widespread death and disruption. The 

crisis has affected an estimated 3.76 million people in Darfur, including 

approximately 1.85 million—“internally displaced persons” (IDPs)—who 

now live in camps. To aid these populations, the United States has 

provided more than $1 billion in assistance for Darfur since fiscal year 

2004, largely through the U.S. Agency for International Development 

(USAID).1

While few would dispute that many civilians have died in Darfur owing to 

violence, disease, and malnutrition, less consensus exists about the total

number of deaths during, or directly resulting from, the crisis. The U.S. 

Department of State (State) reported that a total of 98,000 to 181,000 

people died between March 2003 and January 2005. Five other studies—

conducted for varying purposes by international institutions, academics, 

and individual researchers—have produced estimates ranging up to about 

 1

Simultaneously with the release of this report, we are issuing a separate report on U.S. 

humanitarian assistance to relieve the crisis in Darfur; challenges that have affected the

delivery of the assistance; the African Union’s efforts to fulfill a mandate to support peace

in Darfur; and factors that have affected its implementation of this mandate. See GAO, 

Darfur Crisis: Progress in Aid and Peace Monitoring Threatened by Ongoing Violence

and Operational Challenges, GAO-07-9 (Washington, D.C.: Nov. 9, 2006). 

Page 1 GAO-07-24 Darfur
400,000 deaths for various periods of time between February 2003 and 

August 2005.2

Because policymakers require an accurate estimate of the death toll in 

Darfur to understand the dimensions of the crisis and determine the U.S. 

response, we were asked to examine the estimates, the methods used to 

produce them, and their relative accuracy. This report (1) evaluates the 

relative accuracy and methodological strengths and shortcomings of the 

six death estimates for Darfur;3

 (2) identifies general challenges to 

estimating the total death toll in Darfur and similar humanitarian crises; 

and (3) discusses measures that the U.S. government could take to 

improve its death estimates for Darfur and any similar, future crisis. 

To evaluate the estimates, we reviewed and analyzed public information 

on the estimates and interviewed the estimate authors regarding their 

studies’ data, methods, and objectives. We provided this information and 

summaries of the interviews to a group of 12 experts in epidemiology, 

demography, statistics, and the Darfur crisis convened in April 2006 in 

collaboration with the National Academy of Sciences.4

 These experts 

discussed their review of this information and evaluation of the estimates 

during an all-day session and also assessed the estimates in a follow-up 

survey. State’s Bureau of Intelligence and Research, which conducted the 

department’s death estimate for Darfur, declined to speak with us or 

provide additional information, limiting the experts’ ability to fully 

understand State’s methods of analysis. However, despite this limitation, 

the experts were able to discuss State’s estimate in detail and assess its 

accuracy and methodologies. To identify challenges of estimating total 

deaths in Darfur, we asked the group of experts to highlight key problems, 

and we reviewed literature related to death estimates and mortality data in

crises such as the one in Darfur. To identify ways in which the U.S. 

government could improve death estimates for that and any future 

 2

The five estimates that were publicly available in March 2006 were conducted by, 

respectively, (1) Jan Coebergh, a medical doctor who has worked in Darfur; (2) the Centre 

for Research on the Epidemiology of Disasters (CRED) in Brussels, Belgium; (3) John 

Hagan, Wynnona Rymond-Richmond, and Patricia Parker (released by the Coalition of

International Justice); (4) Eric Reeves, an researcher and advocate for Sudan-related 

issues; and (5) the World Health Organization (WHO).

3

This report’s assessment of the estimates’ methodological strengths and shortcomings is

based on experts’ opinions of the estimates’ data, methods, objectivity, and sufficiency of 

reporting. 

4

Two additional experts participated by phone for parts of the day. 

Page 2 GAO-07-24 Darfur
humanitarian crisis, we asked the experts to provide suggestions during 

their discussion, and we solicited further opinions on these suggestions in

the follow-up survey. Additionally, to learn about current practices related

to the experts’ suggestions for improving death estimates, we spoke with 

officials from USAID, State, and the Centers for Disease Control and 

Prevention (CDC) in the U.S. Department of Health and Human Services. 

Appendix I provides additional details of our objectives, scope and 

methodology. We conducted our work from September 2005 to November 

2006 in accordance with generally accepted government auditing 

standards. 

The experts we consulted did not consistently rate any of the death 

estimates as having a high level of accuracy and noted that all of the 

studies had methodological strengths and shortcomings. Most of the 

experts had the highest overall confidence in the estimates by the Centre

for Research on the Epidemiology of Disasters (CRED), which relied 

primarily on a statistical analysis of about 30 mortality surveys, and they 

rated the CRED estimates’ accuracy and methodological strengths highest 

among the six.5

 The experts had a slightly lower level of confidence in the 

State estimate and gave it slightly lower ratings for accuracy and 

methodological strengths. Further, many experts believed that the lower 

end of State’s estimate was too low and found that published documents 

describing State’s estimate lacked sufficient information about its data and 

methods to allow it to be replicated and verified by external researchers. 

Most experts rated an estimate by the World Health Organization (WHO), 

which primarily extrapolated findings from its 2004 mortality survey, 

somewhat lower than the estimates by CRED and State. Finally, most of 

the experts expressed the least confidence in three estimates that reported 

the highest number of deaths. They cited several shortcomings in these 

estimates, such as a reliance on unrealistic assumptions regarding fixed 

levels of mortality for all populations and time periods included in the 

estimate.6

Results in Brief 

Estimating deaths in a humanitarian crisis such as that in Darfur involves 

numerous challenges. In Darfur, difficulties related to the collection of 

survey data, including a lack of access to particular geographical regions, 

 5

CRED conducted two death estimates: one for the period of September 2003-January 2005 

and another for the period of February-June 2005. 

6

The three highest estimates reviewed by the experts ranged from 253,573 to 396,563 deaths

over varying periods of time. 

Page 3 GAO-07-24 Darfur
challenging survey conditions, and limited resources and training for 

research staff, impacted the data’s quality and resulted in data gaps. 

Because of the lack of complete mortality data, some of the estimates rely 

on potentially risky assumptions and limited contextual information. For 

example, some estimates assume that mortality rates in accessible areas 

can be applied to inaccessible areas, without accounting for differences 

between the two due to factors such as patterns of violence and 

humanitarian relief efforts. In addition, limitations in estimates of Darfur’s 

population before and during the crisis may have led to over- or 

underestimations of the death toll. Finally, varying use of baseline 

mortality rates—the rate of deaths that would have occurred regardless of 

the crisis—may also have led to overly high or low death estimates. 

The group of experts proposed and rated a wide range of measures that 

U.S. agencies could take, directly or by supporting other organizations, to 

improve the quality and reliability of death estimates and relevant data for 

Darfur and any future such humanitarian crises. The most highly rated

measure was ensuring that publicly available documentation of U.S. 

government estimates contains sufficient information on data and 

methods, so that external researchers can replicate the estimates and 

verify their credibility and objectivity. Other very highly rated measures 

that the government could consider included collecting and maintaining 

temporal and spatial data (i.e., data covering specific periods of time and 

geographic areas); housing the responsibility for making estimates in a 

reputable independent body; improving the training of nongovernmental 

organization (NGO) staff who collect mortality survey data; and promoting 

an interdisciplinary approach to estimating mortality. Through various 

initiatives already under way, U.S. agencies are carrying out efforts that 

may address some of the experts’ suggestions, for example, providing 

guidance on the design and implementation of survey instruments and 

supporting the collection and maintenance of temporal and spatial data.

To safeguard State’s credibility as a source of accurate and reliable death 

estimates, we recommend that the Secretary of State provide, for this and 

any future estimates of mortality that State conducts, sufficiently detailed 

descriptions of its data and methodology to enable other parties to assess 

and replicate its findings.7

 Additionally, to enhance the U.S. government’s 

capacity to assess and respond to any future humanitarian crisis, we 

recommend that the Secretary of State and the Director of U.S. Foreign 

 7

We recognize that when such estimates draw from classified data, external researchers 

reviewing these data would require appropriate levels of security clearance.

Page 4 GAO-07-24 Darfur
Assistance and USAID Administrator consider the experts’ other 

suggestions to help address gaps in data and improve any future death 

estimates. 

We provided a draft of this report to the Department of State and USAID. 

State and USAID responded with formal comments, agreeing with our 

recommendations, and State provided additional perspectives on reporting 

and documentation regarding its death estimate. Reproductions of these 

letters, as well as our responses to the letters, can be found in appendixes 

VI and VII. We also provided a draft to the CDC for technical review, and 

we received technical comments from both the CDC and State, which we 

incorporated in the report as appropriate. Finally, we provided the 

authors of the other five estimates the portions of the report pertaining to 

their individual estimates. They provided comments, which we 

incorporated in the report as appropriate. These authors’ comments and

our responses are summarized in appendix VIII. 

Sudan is the largest country in Africa, with a population estimated at about 

40 million (see fig. 1). Darfur is in the western region of Sudan and 

comprises three states, with an estimated preconflict population of around 

6 million. 8

Background 

 8

Estimates for the preconflict population of Darfur range from about 4 million to close to 7 

million. Experts we interviewed have noted uncertainty regarding the population estimate 

for Darfur due to the lack of a current census and the fact that migration in this region

occurs even during nonconflict times.

Page 5 GAO-07-24 Darfur
Figure 1: Map of Sudan 

Page 6 GAO-07-24 Darfur

White Nile

Blue Nile

Nile River

Juba

Nyala

El

Geneina

El Fasher

North

Darfur

South

Darfur

West

Darfur

Wadi Halfa

Nimule

En Nahud

El Odaiya

El Obeid

Umm Durman

Wad

Medani

Port Sudan

Gedaref

Ed Dueim

Ye m e n

Uganda

Sudan

Somalia

Saudi

Arabia

Red

Sea

Libya

Kenya

Ethiopia

Eritrea

Egypt

Dijbouti

Democratic Republic of Congo

Chad

Central

Africa Republic

Source: GAO based on Map Resources (map).

Khartoum

0 350 mile 175 s
In early 2003, Darfur rebels attacked Sudanese police stations and the 

airport in El Fasher, the capital of North Darfur. In El Fasher, the rebels 

destroyed numerous military aircraft, killed many Sudanese soldiers, and 

kidnapped a Sudanese general. In response, the government armed and 

supported the local tribal and Arab militias (the Janjaweed). Fighting 

between the principal rebel groups—the Sudan Liberation 

Movement/Army (SLM/A) and the Justice and Equality Movement (JEM)—

and the Sudan military and Janjaweed intensified during late 2003. In 

addition to displacing approximately 1.85 million Darfurians in the region, 

attacks on civilians by the Sudan military and Janjaweed have forced an 

estimated 220,000 Darfur refugees to take shelter in Chad9

 and so affected 

approximately 1.91 million additional Darfur residents that they require 

humanitarian assistance. (Fig. 2 shows the locations of Darfur villages 

destroyed or damaged in the conflict.) 

Conflict in Darfur 

 9

U.S. law considers any person who is outside the country of such person’s nationality and

who is unable or unwilling to return to, and is unable or unwilling to avail himself or herself 

of the protection of, that country because of persecution or a well-founded fear of

persecution on account of race, religion, nationality, membership in a particular social 

group, or political opinion to be a refugee. 8 U.S.C. § 1101 (a) (42)A. 

Page 7 GAO-07-24 Darfur
Figure 2: Map of Destroyed and Damaged Darfur Villages, as of February 2005 

0 30 60 miles

North 

Darfur

S UDAN

CHAD

Central 

African 

Republic

South

Darfur

West 

Darfur

El Fasher

El

Geneina

Nyala

Village confirmed as

damaged

Village confirmed as

destroyed or ruined

State boundaries

IDP camp

State capital

Refugee camp in Chad

Chad

C.A.R.

Sudan

Ethiopia

Source: GAO based on a map from State’s Humanitarian Information Unit and Map Resources (map).

Page 8 GAO-07-24 Darfur
Peace negotiations under the mediation of the Chad government in 2003 

led to a September 2003 cease-fire agreement between the SLM/A and the 

Sudan government; however, the agreement collapsed in December of that 

year. A second cease-fire agreement was signed by the Sudanese 

government, the SLM/A, and the JEM in April 2004.10 At this point, the 

African Union was brought in to monitor compliance with the cease-fire 

agreement between the three parties, and peace negotiations continued 

under African Union auspices with Chadian participation.11 After a 

relatively calm 2005, cease-fire violations and violent incidents reportedly 

began to increase in the final months of the year and into 2006. 

On May 5, 2006, the government of Sudan and one faction of the SLM/A 

signed the Darfur Peace Agreement, which establishes agreements in key 

areas such as power sharing, wealth sharing, and security arrangements.12

The U.S. government and other parties support a transition of 

peacekeeping responsibilities from the African Union to the UN. In August 

2006, the UN Security Council adopted a resolution expanding the 

mandate of the UN Mission in the Sudan and calling for the mission’s 

deployment to Darfur.13 As of October 2006, the Sudanese president had 

rejected a UN transition but expressed support for a September offer by 

the UN Secretary-General to assist the African Union Mission in Sudan by

providing equipment and dedicated personnel. Meanwhile, the 

environment in Darfur remained insecure, with attacks and displacement 

continuing and, during some periods, worsening over time. 

 10“Agreement on Humanitarian Ceasefire on the Conflict in Darfur,” signed on April 8, 2004, 

in N’Djamena, Chad. 

11Additional interim agreements were also reached, including the July 5, 2005, “Declaration

of Principles for the Resolution of the Sudanese Conflict in Darfur” signed by the Sudanese

government, the SLM/A, and the JEM. This declaration contains 17 principles to guide

future deliberations, such as respect for the diversity of the people of the Sudan, 

democracy, a federal system of government, effective representation in all national 

government institutions, and equitable distribution of national wealth. 

12Darfur Peace Agreement" signed on May 5, 2006, at Abuja, Nigeria. 

13UN Security Council Resolution 1706, adopted August 31, 2006. The expanded UN Mission 

in the Sudan mandate includes, among other things, supporting the implementation of the

2006 Darfur Peace Agreement and the 2004 Humanitarian Ceasefire Agreement. 

Page 9 GAO-07-24 Darfur
The U.S. government has been active in addressing the conflict. On July 22, 

2004, the U.S. House and the Senate each passed separate resolutions 

citing events in Darfur as acts of genocide.14 Further, on September 9, 2004,

in testimony before the Senate Foreign Relations Committee, the U.S. 

Secretary of State announced that genocide had been committed in Darfur 

and that the Sudanese government had supported the Janjaweed, directly

and indirectly, as they carried out a “scorched earth” policy toward the 

rebels and the African civilian population in Darfur.15

U.S. Response to Darfur 

Crisis 

According to State officials, the administration’s declaration of genocide 

was influenced by findings from interviews with 1,136 Darfur refugees in 

eastern Chad in July and August 2004, which demonstrated a pattern of 

abuse against members of Darfur’s non-Arab communities. The interviews 

were conducted by an “Atrocities Documentation Team” assembled by 

State and consisting of officials from State and USAID, as well as members 

of the Coalition for International Justice16 and the American Bar 

Association. State reported that 61 percent of those interviewed said that 

they had directly witnessed the killing of a family member in addition to 

other crimes.17

For fiscal years 2004 through 2006, the United States obligated $996 

million in humanitarian assistance for Darfur.18 Although more than 68 

 14H. Con. Res. 467, 108th Cong. (2004); S. Con. Res. 133, 108th Cong. (2004). 

15The Convention on the Prevention and Punishment of the Crime of Genocide of 1948

defines genocide as any of the following acts committed with intent to destroy, in whole or 

in part, a national, ethnical, racial or religious group, as such: (a) killing members of the 

group, (b) causing serious bodily or mental harm to members of the group, (c) deliberately

inflicting on the group conditions of life calculated to bring about its physical destruction

in whole or in part,(d) imposing measures intended to prevent births within the group, or 

(e) forcibly transferring children of the group to another group. The U.S. government

ratified the convention in 1988. According to a State official, the key factor in the U.S. 

government’s genocide determination was the intent of the Sudanese government 

regarding its actions in Darfur (i.e., its intent to destroy, in whole or in part, a specific 

group of people); the number of deaths attributable to the crisis was not a critical factor. 

16The Coalition for International Justice (CIJ) was an international, nonprofit organization 

that supported international war crimes tribunals and justice initiatives. It closed its

operations in March 2006.

17Department of State, Documenting Atrocities in Darfur, Publication 11182 (Washington,

D.C.: Bureau of Democracy, Human Rights, and Labor and the Bureau of Intelligence and 

Research, September 2004). Available at http://www.state.gov/g/drl/rls/36028.htm. 

18In addition to the U.S. assistance provided for Darfur, an additional $197 million has 

supported Darfur refugees located in Eastern Chad for fiscal years 2004 through 2006. 

Page 10 GAO-07-24 Darfur
percent of this assistance has consisted of food aid, U.S. assistance to 

Darfur has also supported other vital needs, such as water and sanitation, 

shelter, and primary health care services. Partly as a result of U.S. 

assistance, NGOs and UN agencies have made significant progress in 

increasing the number of internally displaced persons and affected 

residents receiving aid. The number of international and national 

humanitarian aid workers in Darfur expanded from 202 in April 2004 to 

13,500 staff members of 84 NGOs and 13 UN agencies in July 2006. In 2005 

and early 2006, malnutrition and mortality rates in Darfur had dropped 

since 2004, a trend that U.S. and other officials attribute in part to 

international humanitarian assistance efforts. Further, according to State

sources, the U.S. government, via private contractors, provided about $280 

million, between June 2004 through September 2006, primarily to build 

and maintain facilities throughout Darfur to house African Union troops 

assigned to monitor compliance with the April 2004 cease-fire agreement.19

Data Sources and Methods 

for Darfur Death Estimates 

A key data source for the Darfur death estimates is health, nutrition, and 

mortality surveys conducted in the field by NGOs delivering humanitarian 

services (e.g., Médecins Sans Frontières, known in English as Doctors 

without Borders, and Save the Children)20 as well as by UN or 

governmental agencies (e.g., the WHO, the World Food Program, and the 

CDC). These surveys are discrete data collection exercises carried out at a 

specific time with a particular sample of the affected population, such as 

people in a certain IDP camp. 

Surveys that ask about mortality are often combined with those collecting 

health and nutrition data. “Retrospective” mortality data are collected by

asking a sample of respondents to recall the number of deaths that 

occurred in their household during an earlier defined period of time.21

Interviewers may also ask respondents questions that allow them to 

categorize the cause of death (e.g., deaths due to violence, disease, or 

malnutrition). Households surveyed may be located in a single area (e.g., 

displacement camp) or multiple areas (e.g., multiple camps or sites within 

a region.) Because an absolute number of deaths is difficult to interpret, 

 19See GAO-07-9. 

20 The estimates by Coebergh, Hagan, and Reeves used the Atrocities Documentation 

Team’s survey of Chad refugees as an additional key source of data. 

21Respondents are also asked to report the number of births and the numbers of people in

their household during the recall period. 

Page 11 GAO-07-24 Darfur
organizations conducting surveys calculate mortality rates for the 

population sampled and for the time included in the “recall period.” These 

mortality rates allow for comparison among different population groups 

and with different time periods, such as when no humanitarian crisis is 

occurring. In acute emergencies, when mortality may change significantly

during a short time interval, mortality rates are often expressed as the 

number of deaths per 10,000 people per day. Typically, a crude mortality 

rate—that is, the rate of death for the entire population, including both 

sexes and all ages22—is reported, as well as mortality rates for specific 

groups (such as those younger than 5 years or of a specific sex). 

Data from CRED’s Complex Emergency Database (CE-DAT) show that at 

least 68 surveys conducted with IDPs and affected residents reporting 

crude mortality rates were conducted in Darfur between 2004 and 2006.23

Two retrospective mortality studies used in most Darfur death estimates 

were conducted, respectively, by Médecins Sans Frontières and the WHO. 

• Médecins Sans Frontières Mortality Surveys in West Darfur. The 

organization conducted retrospective mortality surveys with 3,175 

 22According to the Sphere Project, the crude mortality rate can be expressed with different 

standard population denominators and for different time periods; however, the daily crude

mortality rate is the most specific and useful health indicator to monitor in a disaster 

situation. [See The Sphere Project, Humanitarian Charter and Minimum Standards in

Disaster Response (Geneva: Switzerland, 2004); also known as the Sphere Handbook.] 

Sphere, launched in 1997 by a group of humanitarian NGOs and the Red Cross, developed 

minimum standards to be attained in disaster assistance in five sectors: water supply and 

sanitation, nutrition, food aid, shelter, and health services. The Sphere Handbook reports

that a doubling of the baseline crude mortality rate indicates a significant public health 

emergency, requiring an immediate response. If the baseline rate is unknown, health 

agencies should aim to maintain the crude mortality rate at below 1.0 per 10,000 per day. 

According to data from the U.S. National Center for Health Statistics, the normal crude 

mortality rate for 2003 in the United States is about 0.23 deaths per 10,000 per day. 

23CE-DAT is a searchable database of complex emergencies that includes information on

health and mortality indicators. The database which receives funds from the State’s Bureau 

of Population, Refugees, and Migration, is housed in CRED, a WHO Collaborating Center

located within the School of Public Health of the Université Catholique de Louvain in 

Brussels. (See http://www.cred.be/cedat/index.htm.) According to CRED, the database 

contains 1,155 surveys covering 36 countries; half of these surveys are drawn from original 

reports, and the other half are drawn from secondary sources. Research staff from CE-DAT

we interviewed have told us that, because they rely on voluntary reporting from 

organizations conducting surveys to populate their database, not all surveys that have been 

conducted in Darfur may be included in their database. 

Page 12 GAO-07-24 Darfur
households in four sites in West Darfur.24 The surveys, conducted between 

April and June 2004, asked respondents about the number and cause of 

deaths in their household up to 6 months earlier,25 before they fled their 

villages, as well as after they arrived at their IDP camps or settlements. 

Médecins Sans Frontières reported crude mortality rates ranging from 5.9 

to 9.5 deaths per 10,000 per day during the period when respondents were 

in their villages and in flight and rates ranging from 1.2 to 5.6 deaths per 

10,000 per day during the camp period. Médecins Sans Frontières also 

reported that violence caused 68 to 93 percent of deaths during the village

and flight periods. 

• 2004 World Health Organization Retrospective Mortality Survey. The 

WHO surveyed 3,140 households in accessible IDP settlements in the three 

regions of Darfur to determine mortality rates between June and August of 

2004.

26 The organization reported crude mortality rates of 1.5 per 10,000 

per day for North Darfur; 2.9 per 10,000 per day in West Darfur; and 3.8 per 

10,000 per day for Kalma camp in South Darfur for the defined recall 

period. Diarrhea accounted for most of the reported deaths in all three 

regions, while violence or injury accounted for a smaller percentage of 

deaths, ranging from 10 to 21 percent.

Researchers estimating death tolls in humanitarian crises such as that in 

Darfur generally extrapolate mortality surveys’ various mortality rates to

longer time periods and larger populations. In addition, to estimate an 

“excess” number of deaths directly attributable to the conflict, some 

researchers subtract a baseline mortality rate—that is, an expected 

 24Across the four sites, Médecins Sans Frontières surveyed 3,175 households representing a 

total of about 17,500 people and drawn from an estimated IDP population of 215,400. See

Depoortere, Evelyn et al. “Violence and Mortality in West Darfur, Sudan (2003-2004): 

Epidemiological Evidence from Four Surveys.” Lancet, 364 (2004): 1,315-1,320. Médecins

Sans Frontières produced this study in collaboration with Epicentre, a nonprofit 

organization created in 1987 by Médecins Sans Frontières, which groups health 

professionals specialized in public health and epidemiology. (See 

http://www.epicentre.msf.org/.)

25The recall period varied in the surveys conducted across the four sites. The longest was 

183 days for the survey conducted at Zalingei, and the shortest was 39 days for the survey

at El Geneina. Médecins Sans Frontières reported using a calendar of locally important 

events to facilitate recall. 

26The WHO reported surveying a total of 3,140 households representing about 21,000 

people. The WHO surveyed IDPs from 43 locations in North Darfur and 43 locations in 

West Darfur, but due to security problems surveyed only IDPs residing in Kalma camp in

South Darfur. See World Health Organization. Retrospective Mortality Survey among the

Internally Displaced Population Greater Darfur, Sudan 2004, 2004. 

www.who.int/disasters/repo/14656.pdf

Page 13 GAO-07-24 Darfur
number of deaths that would have occurred absent the conflict—from the 

total number of deaths estimated for the time period and population. (Fig. 

3 illustrates the generation of a death estimate.) 

Figure 3: Example of Generation of a Death Estimate 

The death estimates that we and the group of experts reviewed were 

produced for varying purposes, according to their authors. 

Mortality surveys

• Conducted in the field 

by aid organizations. 

• Intended to collect 

data on health, 

nutrition, and mortality.

• Conducted with a

sample of the affected 

population in various

locations.

• Ask respondents to recall 

deaths that occurred in an 

earlier defined period of 

time and may ask about 

cause of death.

Data collection and

calculation performed in the field

Source: GAO.

Data aggregated to 

estimate total deaths

Calculated 

mortality rates

• Calculated based on 

analysis of mortality 

survey responses.

• Aggregated findings from 

multiple surveys to 

determine overall mortality 

rate for a region or location. 

• Often expressed in deaths

per 10,000 per day.

• Crude Mortality Rate (CMR) 

for all groups typically reported. 

Mortality rates for specific 

groups may also be reported 

(e.g., deaths of children younger 

than 5).

Death estimates

• Overall estimated 

number of deaths due to 

the crisis.

• Based on extrapolations

of CMRs from one or 

more surveys to a larger 

affected population over 

a longer period of time.

• May estimate both total 

deaths that have 

occurred during the 

crisis, as well as excess

deaths above an 

expected mortality rate 

absent the conflict.

Example

In 2004, the WHO conducted 

retrospective mortality 

surveys with 3,140 

households in 87 locations

across three regions of 

Darfur.

Example

The WHO reported a total of 

304 deaths and CMRs

ranging from 1.5 to 3.8 per 

10,000 per day for the 

regions of Darfur for the 

period of June-August 2004.

Example

The death estimate by 

Hagan and colleagues

extrapolated the WHO and 

another survey to estimate 

396,653 total deaths from 

February 2003 through 

March 2005 (26 months of 

the conflict).

Page 14 GAO-07-24 Darfur
• The authors of the CRED estimates27 said that their aim in conducting 

their estimate was to develop a method that, rather than extrapolating 

mortality from a single survey to the entire region and conflict period, 

took into account variations over time and space. 

• Dr. Coebergh28 noted that his estimate was intended as a political 

statement to increase public awareness of the crisis. He also stated 

that the estimate was produced as a response to earlier total death 

tolls quoted in the media, which he believed were too low and did not 

adequately capture violent deaths. 

• According to Dr. Hagan,29 his estimate in the spring of 2005 responded 

to earlier total death figures, quoted in the media, that he believed 

were too low and did not adequately capture violent deaths. He stated 

that one of his goals was to provide greater transparency about the 

available estimates. 

• Dr. Reeves30 said that he produced his first estimate of the Darfur death 

toll, in June 2004, because he believed that the figures being quoted by 

the UN significantly understated the Darfur death toll and were not 

supported by the data. 

 

• According to State, its purpose was to provide information for internal 

policymakers. 

 27CRED is a nonprofit research institution and a World Health Organization Collaborating 

Centre based in the School of Public Health of the Université Catholique de Louvain in

Brussels. 

28Jan Coebergh is a medical doctor in the Netherlands who has worked in Darfur. 

29John Hagan, a professor of sociology at Northwestern University, conducted his estimate 

with his colleagues, Wynnona Rymond-Richmond and Patricia Parker; the estimate was 

released by the Coalition of International Justice. Dr. Hagan stated that the 2005 estimate 

used an assumption of constant levels of mortality over time, because he believed other

estimates had relied on that assumption. He also noted that he subsequently modified his 

assumptions and estimating methods in his estimate with Alberto Palloni. [See John Hagan

and Alberto Palloni, “Social Science: Death in Darfur,” Science, vol. 313 (2006): 1,578-1,579.] 

30Eric Reeves is a professor of English language and literature at Smith College and has 

spent the past 7 years working full-time on research and advocacy issues related to Sudan. 

When the experts convened in April 2006, they reviewed and discussed Reeves’s latest 

available estimate of 375,000 through August 2005. At the end of April 2006, he reported 

that total excess mortality in Darfur was greater than 450,000 deaths. 

Page 15 GAO-07-24 Darfur
• A WHO31 official told us that the organization sought to assess the 

order of magnitude of deaths in Darfur to assist it in planning

humanitarian relief for IDPs in camps. 

The estimates that we and the experts reviewed vary quantitatively and in

the time that they cover—from 35,000 excess deaths or 45,000 total deaths

for IDPs in camps over 7 months of the conflict, to almost 400,000 total 

deaths for Darfur over 26 months of the conflict. (See fig. 4 for a summary

description of the estimates and their findings, and see app. IV for more 

details.) 

 31The WHO estimate was presented by David Nabarro, a senior WHO official. 

Page 16 GAO-07-24 Darfur
Figure 4: Death Estimates Based on Reported Figures and Time Included

J MAMJ JA F S OND J MAMJ JA F S OND F J MAMJ J A S OND J MAMJ J A F S OND

0

100

200

300

400

Estimated deaths

Deaths in thousands

Organization/ Reported deaths over time included in estimate

individual 2003 2004 2005 2006

Source: GAO analysis of estimates from above organizations.

98,000–181,000 total deaths/

63,000–146,000 excess deaths

(23 months)

218,449;

253,573 or 306,130 excess deaths

(21 months)

396,563 total deaths

(26 months)

Over 370,000 excess deaths

(31 months)

45,000–80,000 total deaths for IDPs in camps/

35,000–70,000 excess deaths for IDPs in camps

(7 months)

134,000 total deaths/

118,142 excess deaths

(17 months) 

36,237 total deaths/

23,658 excess deaths

(5 months)

State

CREDa

Coebergh

Hagan et al.b

Reeves

WHO

State

(Higher- and lower-end 

estimates for total and 

excess deaths)

CREDa

(Point estimates for total 

and excess deaths)

Coebergh

(Three different point 

estimates for excess

deaths)

Hagan et al.b

(Point estimate for total 

deaths)

Reeves

(Point estimate for excess

deaths)

WHO

(Higher- and lower-end 

estimates for total and 

excess deaths)

Baseline mortality

Point estimates for excess deaths

Point estimate for total deaths

a

CRED conducted two death estimates: one for the period of September 2003-January 2005 and 

another for the period of February-June 2005. The bar on the left for “estimated deaths” is the 

combined total of these two estimates. 

b

This estimate was conducted by John Hagan and colleagues and released by the CIJ. 

Page 17 GAO-07-24 Darfur
We and the group of experts also reviewed a preliminary version of a 

death estimate for West Darfur by John Hagan and Alberto Palloni; 

however, this estimate is not discussed in our report, because the version 

that the experts reviewed was preliminary and not publicly available when 

they convened in April 2006. In the estimate’s final version, which became 

publicly available in September 2006, Hagan and Palloni estimated that a 

range of 57,506 to 85,346 deaths had occurred in West Darfur over 31 

months. Using the same ratio of deaths to displacement, they estimated a 

range of 170,000 to 255,000 deaths in all three states of Darfur over the 

same time period. 32

In addition, the estimates differ methodologically, incorporating varying

source data, mortality rates and causes of death, assumptions regarding 

variable or fixed mortality rates, estimates of the affected population, 

consideration of baseline mortality rates, and reporting methods. (See app. 

IV for more information.) 

• Source data. The estimates’ respective sources of data range from about 

30 mortality surveys to a single mortality survey. The estimates also use 

nonsurvey or contextual information, such as the retrospective reporting 

of attacks or displacement patterns, to varying degrees. In addition, the 

amount of source data used varies according to the availability of such 

data when the estimates were conducted. For example, fewer data were 

available for estimates conducted in 200433 than for those conducted in 

2005. 

• Mortality rates and cause of death. The estimates differ in whether they 

applied mortality rates that include all causes of death or calculated and 

applied mortality rates due to violence and nonviolence separately. 

 32See John Hagan and Alberto Palloni, “Social Science: Death in Darfur,” Science, vol. 313 

(2006): 1578-1579. In addition, the experts did not review a study of the Darfur conflict by

Bloodhound, a Danish advocacy group, because it was published after the experts met in 

April 2006. The study estimates that deaths in Darfur ranged from 57,000 to 128,000 deaths 

between April 2003 and September 2005 due to attacks on villages throughout the region. 

[See Andreas Höfer Petersen and Lise-Lotte Tullin, The Scorched Earth of Darfur: Patterns

in Death and Destruction Reported by the People of Darfur. January 2001-September 

2005 (Copenhagen: Bloodhound, 2006). Available at 

http://www.bloodhound.se/rap_uk.html.] 

33The WHO conducted an initial estimate in July 2004 and a subsequent estimate in October 

2004. Eric Reeves began his estimates in June 2004 and has reported ongoing “mortality 

updates” since that time. As of October 2006, his last update was conducted at the end of

April 2006. 

Page 18 GAO-07-24 Darfur
However, most of the estimates, as well as the mortality surveys we 

reviewed, express mortality rates in deaths per 10,000 people per day. 

• Assumptions of variable or fixed mortality rates. The estimates include 

different assumptions regarding whether mortality rates varied over the 

period of the conflict or according to the affected region or remained 

constant for all populations in all areas over the entire period of conflict. 

• Affected population estimates. The estimates rely primarily on 

information from the UN Humanitarian Profiles in determining the total

affected population, but they differ in their assumptions about whether the 

affected population changed over time or remained constant. Additionally, 

one estimate also includes only IDPs in camps and excludes affected 

residents in Darfur and refugees in Chad. 

• Baseline mortality rates. The estimates varied in their use of a baseline

mortality rate—that is, the number of expected deaths absent the 

conflict—from using no baseline to using a baseline of 0.5 deaths per 

10,000 per day. 

• Reporting methods. Some estimates report a range of possible total 

deaths, others report point estimates — that is, single numbers. 

Additionally, some estimates are precise figures, such as 396,563, while 

others are rounded to fewer significant digits, such as 134,000. 

Although none of the death estimates was consistently considered 

accurate or methodologically strong, the experts we consulted rated some 

of the estimates more highly than others. Overall, the experts expressed 

the highest level of confidence in CRED’s estimates and slightly lower 

levels of confidence in State’s and the WHO’s estimates. They expressed 

the lowest level of confidence in the three estimates that report the highest 

number of deaths, citing multiple shortcomings, such as a reliance on

unrealistic assumptions about populations’ level of risk over periods of 

time.

Some Death 

Estimates Judged 

More Accurate and 

Methodologically 

Stronger, Despite 

Shortcomings 

Page 19 GAO-07-24 Darfur
Although the experts clearly had greater confidence in some estimates 

than in others, they did not consistently express a high level of confidence

in any of the estimates or consistently rate any of them as accurate.34

The experts expressed a slightly higher level of confidence in the CRED 

estimates than the State estimate. However, only 2 of the 12 experts 

expressed a high level of confidence, and most had a moderate level of 

confidence, in the CRED estimates. Experts had a moderate to low level of 

confidence in the WHO estimate. Almost all experts expressed a low level

of confidence in the estimates by Coebergh, Hagan, and Reeves.35 (See app. 

V for additional survey results.) 

Similarly, the experts did not consistently rate any of the estimates as 

accurate; the majority of experts rated almost all of the reported figures as

either too high or too low. Only CRED’s second estimate—36,000 total 

deaths for February-June 2005—was viewed by half of the experts as 

“about right.” (See table 1.) 

Table 1: Accuracy of Darfur Death Estimates Rated by Experts

No Estimate Consistently 

Considered Accurate 

Estimate (number of months in estimate) Too high About right Too low

Coebergh

218,449 excess deaths (21) 7 1 2

253,573 excess deaths (21) 8 0 2

306,130 excess deaths (21) 8 1 1

CRED’s two estimates

Sept. 2003-Jan. 2005: 134,000 total deaths (17) 1 3 6

Feb. 2005–July 2005: 36,237 total deaths (5) 2 5 2

Hagan et al. 

396,563 total deaths (26) 10 0 0

Reeves

Over 370,000 excess deaths (31) 10 0 0

State’s lower- and higher-end estimates 

 34Although they questioned the accuracy of the estimates and had differing views about the 

actual number of deaths that have occurred in Darfur, the experts did not question the

severity of the crisis. In discussing the importance of the estimates’ accuracy, several 

experts noted that estimates can be used in war crime proceedings, and one stated that it 

was important to account for all those who died so that this could become part of the

historical record. 

35One expert rated having a moderate level of confidence in the estimate by Hagan. 

Page 20 GAO-07-24 Darfur
Estimate (number of months in estimate) Too high About right Too low

Lower-end estimate: 98,000 total deaths (23) 0 1 9

Higher-end estimate: 181,000 total deaths (23) 4 4 2

WHO’s lower- and higher-end estimates

Lower-end estimate: 45,000 total deaths (7) 1 2 7

Higher-end estimate: 80,000 total deaths (7) 3 1 6

Source: GAO.

Note: We asked experts to rate the accuracy of total deaths when these were reported in the 

estimate. Coebergh and Reeves reported excess deaths only. Additionally, in responding to this 

question, one expert said that there was no basis to judge or that he or she was not sure about the 

accuracy of the estimates; one expert chose not to respond to this question; another expert chose 

two responses for the second CRED estimate, and his or her responses are excluded. 

In some cases, experts tended to agree as to whether the reported figures 

were too high or too low; in other cases, there was less agreement. For 

example, 9 of 10 experts rated the lower-end of State’s estimate as too low, 

and the majority of the experts viewed the estimates by Coebergh, Hagan, 

and Reeves as too high. In contrast, experts did not agree as to whether 

State’s higher estimate was too high, too low, or about right. For example, 

a few experts believed the higher end of State’s estimate was likely to be

closer to a reasonable midrange estimate. These experts also thought that

the mortality surveys State used for the estimate may have been 

conducted at places with higher levels of aid and subsequently lower 

levels of mortality, or that disease outbreaks may have been missed. 

However, one expert believed that State may have overestimated mortality 

by applying elevated mortality rates for too many months of the crisis. 

The experts’ overall assessment of the estimates’ methodological strengths

in terms of their data, methods, objectivity, and reporting did not produce 

any consistently high ratings, and experts noted shortcomings in each 

estimate.36 

Overall, the experts rated CRED’s estimates most highly in terms of data, 

methods, objectivity, and reporting of limitations. However, several 

Experts Found 

Methodological 

Shortcomings in Each 

Estimate 

CRED Estimates 

 36In evaluating each estimate’s methodological strength, the experts rated the source data

and whether such data were methodologically sound; the methods, including whether 

extrapolations were appropriate, assumptions were reasonable, and shortcomings that 

could result in over- or underestimation were sufficiently described; the level of objectivity, 

based on whether a particular bias appears to be part of the estimating procedure; and the

sufficiency of reporting and information contained in published documents describing the 

estimate. 

Page 21 GAO-07-24 Darfur
experts found shortcomings in the CRED estimates’ data and methods and 

thought that CRED could have provided more information and clarity in its 

reporting. 

• Source data. Most experts said that the data CRED used for its two 

estimates (drawing from a total of about 30 mortality surveys in Darfur 

and Chad37) were generally sound.38 CRED reported checking the reliability 

and validity of the surveys included in its estimates, which experts found 

to be a strength. However, several experts cited some shortcomings in 

CRED’s data sources. For example, some said that CRED could have 

better articulated the criteria used to select the survey data or weighted 

the surveys it used by sample size. A few experts also thought that CRED 

should have considered other sources besides mortality surveys, such as 

surveillance data on morbidity or nutrition. 

• Methods used, including extrapolations and assumptions. Some experts 

found CRED’s method of using disparate data sources to estimate total 

deaths to be innovative and logical.39 Additionally, more than half of the 

experts rated CRED’s assumptions and extrapolations as somewhat 

appropriate or reasonable.40 For example, several experts found the Sudan

baseline mortality rate that CRED used more accurate than the baseline 

mortality rates derived from a larger region of sub-Saharan Africa used in

some of the other estimates. However, some assumptions and 

extrapolations were questioned. For example, several experts thought that 

CRED’s assumption regarding a generally stable rate for nonviolent 

mortality during much of 2004 may have been inaccurate, owing to 

 37CRED selected or excluded surveys for reasons such as whether findings from a regional 

survey differed statistically from findings for statewide surveys or whether a survey’s 

findings were viewed as outliers. CRED used the selected surveys to analyze violent or 

nonviolent mortality rates depending on, for example, whether the surveys reported the

proportion of deaths due to violence. Although CRED primarily relied on mortality survey

data, it also used contextual information to refine its estimates and make some 

assumptions.

38Nine experts rated the data CRED used as generally sound, two rated the data as equally 

sound and unsound, and two rated the data as generally unsound. 

39CRED analyzed findings from the mortality surveys mentioned above to separately 

estimate nonviolent and violent mortality rates and combined these to get overall mortality 

rates for different regions and periods of the conflict. 

40Eight experts rated CRED’s extrapolations as very or somewhat appropriate, two experts 

rated them about as appropriate as inappropriate, one expert rated them as somewhat 

inappropriate, and one expert found no basis to judge or was not sure. Additionally, 10 

experts rated CRED’s assumptions as very or somewhat reasonable, and two experts rated 

them as somewhat or very unreasonable. 

Page 22 GAO-07-24 Darfur
possible changes in the affected population, camp formations, the level of 

aid, and the outbreak of disease. Additionally, some experts considered 

the method that CRED used to estimate deaths among refugees in Chad to

be somewhat unsystematic. 

• Level of objectivity. Overall, experts viewed CRED’s death estimates as 

having the highest level of objectivity.41 Two authors of other estimates 

also concluded that the CRED estimates were likely to be more reliable 

and more scientific than other Darfur death estimates, including their own. 

• Sufficiency of reporting. Experts noted that, among the estimates they 

evaluated, CRED most sufficiently reported the limitations and potential 

sources of over- or underestimation in its estimates. However, several 

experts believed that better descriptions of the methods used, including 

information on specific formulas and calculations, could have been 

provided. An author of another estimate also noted that the mortality 

surveys used in CRED’s estimates lacked complete citation information. In 

our review of CRED’s first estimate, we were able to replicate it to some 

degree only after the authors provided a substantial amount of

information, such as specific mortality rates and formulas used and 

citations for source studies, in addition to the information in the published 

document. 

In assessing State’s estimate, the experts identified methodological 

strengths related to each of the four elements but also noted some 

shortcomings. Strengths included its use of multiple types of information, 

including contextual data from other sources besides surveys, such as 

reporting of attacks. Many experts also believed the estimate had a high 

level of objectivity. However, the experts, as well as authors of other 

estimates, cited several shortcomings in State’s estimate. For example, 

many believed that the lower end of the estimate was too low, owing to 

several factors including the use of some data that underestimated 

mortality rates. Additionally, experts and other authors thought that the 

published documents containing the estimate lacked sufficient 

information, such as a clear description of the mortality rates used for all 

populations and time periods included in the estimate.

State Estimate 

• Source data. Many experts cited as one strength the estimate’s use of 

different types of data, including mortality survey data and contextual 

information, to triangulate findings and estimate mortality, and one expert 

 41Ten experts rated CRED’s level of objectivity as very high or high, one expert rated it as 

moderate, and one expert indicated that he or she had no basis to judge or was not sure. 

Page 23 GAO-07-24 Darfur
deemed this approach a “pioneering attempt” in the field of death 

estimates in humanitarian crises.42 However, just over half of the experts 

thought that the data used were methodologically sound.43 Some experts 

said that several of the mortality surveys used in State’s estimate may have

had methodological limitations in areas such as survey design, 

implementation, or accessibility to insecure regions, resulting in 

unrealistically low mortality rates. These experts believed that such 

limitations in source data, in addition to other problems—for example, the 

estimate’s lack of clarity regarding how missing populations are accounted 

for and use of a relatively higher baseline mortality rate—may have pulled 

down State’s estimate, in particular, its lower end. 

• Methods used, including extrapolations and assumptions. About half of 

the experts felt that State’s estimate applied somewhat appropriate 

extrapolations, and a similar proportion thought that it made reasonable 

assumptions. A few experts thought that its depiction of varying levels of 

mortality over time and affected regions was appropriate to estimate total 

deaths throughout the Darfur crisis. However, several experts thought that 

the assumptions used, in some cases, were based on insufficient rationale 

and evidence and that additional sensitivity analysis could have been 

conducted. For example, the State estimate assumes that mortality rates in

Darfur are 20 percent lower for affected residents than for IDPs, an 

assumption that some experts believed lacked sufficient rationale. 

• Level of objectivity. Nine of the 12 experts rated the State estimate’s level 

of objectivity as high.44 Several experts generally believed that the estimate 

represented a “good faith effort” to use available evidence in an unbiased 

way. 

• Sufficiency of reporting. Many of the experts found that the published 

documents containing State’s estimate lacked sufficient information to 

allow them to replicate the estimate and verify the accuracy and reliability 

 42State reports using various sources of information from sources other than surveys, such

as displacement patterns, village destruction, and retrospective reporting of attacks to, 

among other things, fill in data gaps and determine whether mortality rates derived from 

existing surveys could be applied as an overall rate to a broader population or if higher or

lower rates were warranted. 

43Seven of the 12 experts rated the State’s data as generally methodologically sound, 2 rated

the data as equally sound and unsound, 2 rated the data as generally not sound, and 1

indicated that he or she was not sure or had no basis to judge. 

44Two experts rated the objectivity as moderate, and one said that he or she had no basis to 

judge or was not sure. 

Page 24 GAO-07-24 Darfur
of the data and methods. For example, some experts noted that the data

used were not sufficiently described and that more specific citations, as 

well as a description of the criteria applied to select the data, would have 

been useful. Additionally, some experts felt that they did not have a good 

sense of the reliability and validity of the contextual data used in the 

estimate and the application of these data to determine mortality rates or 

total deaths. Moreover, several experts said that State should have 

included the specific formulas or mortality rates used for all populations,

time periods, and regions. Similar comments were made by several 

authors of other estimates. Our review of the State estimate also showed 

that it could not be replicated with the information contained in the report.

Further, one expert noted that the published document available on State’s 

Web site particularly lacked sufficient description of the estimate’s 

methods, data, and potential limitations. 

Several experts found strengths in the data and level of objectivity of the 

WHO’s estimate, which it presented in a short briefing in October 2004. 

However, several experts observed shortcomings in the WHO’s reporting 

of its estimate. 

WHO Estimate 

• Source data. The WHO estimate of IDP deaths in Darfur for 7 months in 

2004 relied primarily on findings from the organization’s 2004 mortality 

survey. Several experts noted that this survey followed standard methods 

and was generally reliable. However, a few experts and estimate authors 

said that the 2004 WHO survey may have underestimated mortality, owing

to local government restrictions that prevented researchers from asking 

respondents detailed questions about mortality. 

• Methods used, including extrapolations and assumptions. Experts 

provided mixed ratings on the appropriateness of the extrapolations and

the reasonableness of the assumptions used in the WHO estimate. For 

example, one expert believed that because the methodology of the 

organization’s 2004 survey was appropriate to gauge levels of mortality 

among a limited IDP population for 2 months, but not aggregate mortality, 

the survey findings should not have been extrapolated to generate a death 

estimate for the total IDP population for 7 months. 

• Level of objectivity. Half of the experts rated the level of objectivity of the

WHO estimate as high, three rated it as moderate, and two rated it as low.45

About half of the experts thought that the WHO estimate’s level of 

objectivity was equal to CRED’s and State’s. 

 45One indicated no basis to judge or not sure. 

Page 25 GAO-07-24 Darfur
• Sufficiency of reporting. Several experts thought that the WHO’s 

reporting of its estimate had several shortcomings. For example, they

thought that the briefing document describing the death estimate did not

make it sufficiently clear that the estimate included deaths of IDPs from all 

causes—both violent and nonviolent—but excluded deaths of affected

residents in Darfur and refugees in Chad. The experts found that the lack

of a clear description of such issues allowed for misinterpretation by 

readers, including the media and academia. 

The experts cited several methodological shortcomings in the Coebergh, 

Hagan, and Reeves estimates, including the use of problematic data and 

application of unrealistic assumptions about the levels of mortality over 

time and affected populations. 

Three Other Estimates

• Source data. Many experts found shortcomings in each of the three 

estimates’ use of certain survey data.

46 A number of experts noted 

problems in the design, sampling, and data collection in the Atrocities 

Documentation Team’s survey of Chad refugees on which all three 

estimates based, at least in part, their numbers of violent deaths. Experts 

also pointed out that, because the survey’s intended purpose was to 

document levels and types of victimization, the estimates by Dr. Coebergh, 

Dr. Hagan, and Dr. Reeves should not have extrapolated the survey 

findings to a broader population or time period in order to estimate total 

deaths. In addition, many experts observed that the estimates by Coebergh 

and Hagan inappropriately used findings from the 2004 WHO survey to 

calculate only nonviolent deaths without taking into account the fact that 

some deaths reported by the WHO were due to violence or injury.47

• Methods used, including extrapolations and assumptions. Most experts 

found that the Coebergh, Hagan, and Reeves estimates used unrealistic 

extrapolations and assumptions to fill information gaps and estimate total 

deaths. For example, many experts thought that each of the three 

estimates relied on too few data points extrapolated to an excessive 

degree. As a result of this type of extrapolation, the experts observed, a 

 46Each of the three estimates based, at least in part, its numbers of violent deaths on the

“Atrocities Documentation Team’s” survey of Chad refugees and its numbers of nonviolent

deaths on the 2004 WHO survey. The Coebergh and Reeves estimates also used additional

data sources. 

47The 2004 WHO survey reported that violence or injury accounted for 10 to 21 percent of

deaths in North and West Darfur and Kalma camp in South Darfur for a 2-month recall 

period in the summer of 2004. Jan Coebergh and John Hagan told us they knew that the

WHO survey reported a percentage of deaths due to violence but believed the survey did

not sufficiently capture violent deaths, particularly during the period prior to camp arrival.

Page 26 GAO-07-24 Darfur
sensitivity analysis changing one or two assumptions could swing the total 

number of deaths from 100,000 to half a million, making the estimates 

unreliable. Moreover, several experts believed that some of the authors 

had inappropriately assumed constant rates of mortality for different 

population groups in Darfur at different periods in the conflict.48 Two of 

the estimates (Coebergh’s and Hagan’s) also used “fixed” estimates of the 

affected population over time, a method that some experts thought was 

inappropriate because the affected population grew over the course of the 

conflict. Additionally, the three estimates involve assumptions that some 

experts viewed as questionable, such as using unsupported numbers to 

estimate the number of deaths among populations inaccessible to aid or 

assuming all missing persons were likely to be dead. 

• Level of objectivity. Most experts rated the level of objectivity of the three 

estimates as low, particularly those by Drs. Coebergh and Reeves. The

experts thought that the estimates were more characteristic of advocacy 

or journalistic material than of objective analysis. 

• Sufficiency of reporting. One expert noted that the estimate by Dr. Hagan 

was very straightforward and one could follow the logic of the data and 

methods applied. However, most of the experts found that the three 

estimates did not sufficiently describe limitations that may have resulted 

in under- or overestimation of total deaths. In reviewing the estimates, we

found we were able to replicate Dr. Hagan’s entire estimate based on its 

description in public documents. We were also generally able to replicate

Reeves’ estimate after receiving additional information about, among other 

things, his calculation of some numbers and the sources of his data. We 

had more difficulty in replicating Dr. Coebergh’s estimate based on the 

information in the published article and were able to do so only after the 

author provided, at our request, details about the specific data, methods, 

and formulas that were used. 

Estimating the total deaths in a humanitarian crisis such as that in Darfur 

involves a number of challenges, most notably related to collecting data in

the field and extrapolating from limited data. Difficulties in collecting 

sound, consistent survey data—including lack of access to particular 

geographical regions, the conditions under which the surveys are 

conducted, and limited resources and training for field staff conducting 

surveys—affect the quality of the data collected and result in data gaps. 

Because of such limitations, as well as the unavailability of data from 

Estimating Deaths in 

Humanitarian Crises 

Involves Many 

Challenges 

 48Reeves’s estimate assumes some change in mortality levels over time. 

Page 27 GAO-07-24 Darfur
other sources, the death estimates that we reviewed rely on potentially 

risky assumptions and limited contextual information. Limitations in 

estimates of Darfur’s population before and during the crisis also impacted 

the quality of the death estimates. Additionally, varying approaches to the

use of baseline mortality rates may have somewhat affected their 

accuracy. 

Difficulties in conducting health, nutrition, and mortality surveys in a 

crisis such as Darfur’s can affect survey data’s quality and completeness 

and, thus, the soundness of death estimates based on the data. In Darfur, 

these difficulties include lack of access to certain geographical regions, 

difficult survey conditions, researchers’ limited resources and training, 

and lack of consensus over sampling methods. In addition, the findings 

from the surveys in Darfur are not always publicly available, and few other 

reliable sources of mortality data exist in Darfur. 

Challenges in Collecting 

Source Data Affected Data 

Quality and Led to Data 

Gaps 

• Lack of access to some geographical regions. Researchers’ lack of access 

to some geographic areas in Darfur raises concerns about the 

completeness of the data collected. Because of security problems, 

humanitarian aid organizations that typically conduct the surveys, such as 

Médecins Sans Frontières, and other researchers found it difficult to 

access all areas of Darfur, particularly South Darfur, according to several

experts. In one instance, a survey conducted by the WHO in 2005 in South

Darfur was suspended because of security concerns. Other surveys were 

also curtailed because of security concerns or attacks on NGO staff. In 

general, surveys were conducted primarily in camps where humanitarian 

relief was being provided, which could lead to underestimates of actual 

mortality. For example, the WHO and World Food Program note in their 

respective reports that their 2004 surveys may underestimate mortality, 

owing to their samples’ exclusion of people in inaccessible areas. 

Additionally, one expert reported that the Sudanese government at times 

placed restrictions on the relief organizations collecting data and limited 

their access to particular camps and regions. 

• Challenging survey conditions. Challenges in conducting householdbased mortality surveys during and after humanitarian crises such as that 

in Darfur can affect the data’s accuracy, consistency, and completeness 

and lead to over- or underreporting of mortality. Such challenges include

linguistic and other cultural factors, difficulties resulting from the surveys’ 

conflict or postconflict environment, and issues of recall and precision. 

• Accurate translation of surveys into foreign languages can be difficult. 

Page 28 GAO-07-24 Darfur
• Surveys might not be conducted in all local languages.49

• Definitions of a household vary.50

• Some people are reluctant to talk about death. 

• Some cultures will not report the deaths of infants.51

• If all household members die, none remain to be surveyed.52

• The presence of government employees or parties to the conflict can 

lead to over- or underreporting. 

• A reluctance to forgo food rations may lead to underreporting of deaths 

in the household. 

• Dating deaths that occurred months prior to survey can be difficult.53

• The length of the survey’s recall period may lead to under- or 

overreporting of deaths and affect the precision of estimated mortality 

rates.54

 49In Darfur, for example, some surveys were conducted in Arabic and not in other local 

languages. 

50One expert stated that in Darfur, a household was defined as everyone who ate at the

same table the previous night. This definition is important because household size is a 

factor in calculating crude mortality rates. 

51One expert noted that certain cultures do not report the deaths of unnamed infants and,

in some of these cultures, naming does not occur until an infant has lived for 40 days. 

Because the first 40 days of life present the highest risk of mortality, unreported infant 

deaths could compromise mortality estimates. 

52Known as “survivor bias” in the research literature. For example, see World Health

Organization, “Module 4: Studying Health Status and Health Needs.” (Available at http:// 

www.who.int/hac/techguidance/tools/disrupted_sectors/module_04/en/index2.html p.3)

53Known as “recall bias” in the research literature. For example, see World Health 

Organization, “Module 4: Studying Health Status and Health Needs.” (Available at 

http://www.who.int/hac/techguidance/tools/disrupted_sectors/module_04/en/index2.html 

p.3.) 

54For example, a survey conducted by the World Food Program in the fall of 2004 had a 

recall period of 7 months. However, because the average length of displacement was 7.5

months, it is likely that the survey did not capture all mortality that occurred prior to 

displacement, some of which could have been due to violence. 

Page 29 GAO-07-24 Darfur
• Identifying some causes of death can be difficult.55

• Insufficient training and resources among research staff. Difficulties in 

training staff and assembling resources may also have affected the quality

of the surveys conducted. Mortality surveys require staff who are 

proficient in matters such as designing questionnaires, selecting samples 

of the local population,56 and conducting interviews in local languages. 

Although larger organizations that routinely conduct surveys generally

have staff that are experienced in designing and implementing surveys, 

other groups may not have as much knowledge and experience in 

collecting data, according to experts and the research literature. Although 

the NGOs provide some training, high turnover rates make it difficult to 

retain the levels of knowledge and skill that are required. 

• Challenges related to sampling methods. The research literature notes 

that samples drawn in IDP or refugee camps may not provide an accurate 

count of deaths that occurred in attacks on villages and when IDPs were 

fleeing to the camps.57 However, although it recognizes the difficulty of 

selecting and implementing an appropriate sampling method to measure 

violent deaths outside camps, the literature does not offer any definitive 

solutions. 

In addition, the NGOs that conduct mortality surveys in Darfur do so 

primarily to monitor conditions in the camps they serve, and they 

generally do not disseminate the survey results. To address this problem, 

CE-DAT was established in 2003, under the Standardized Monitoring and 

 55For example, identifying death from particular illnesses and diseases can be difficult for 

respondents.

56The research literature also questions whether appropriate guidance and training in 

sampling are routinely provided to field staff and whether guidance is always followed 

when provided. See Paul B. Spiegel, Peter Salama, Susan Maloney, and Albertine van der 

Veen, “Quality of Malnutrition Assessment Surveys Conducted During Famine in Ethiopia,”

JAMA, vol. 292, no. 5 (2004). 

57See Jennifer Leaning and Michael VanRooyen, “An Assessment of Mortality Studies in

Darfur, 2004-2005,” Humanitarian, vol. 30 (June 2005). Also see Michel Thieren, “Health

Information Systems in Humanitarian Emergencies,” Bulletin of the World Heath

Organization, vol. 83, no. 8 (2005). For a discussion of possible approaches to sampling in 

these situations, see Holly Reed (Rapporteur), “Demographic Assessment Techniques in

Complex Humanitarian Emergencies,” Summary of a Workshop (Washington, D.C.: 

National Academy Press, 2002). 

Page 30 GAO-07-24 Darfur
Assessment of Relief and Transitions (SMART) initiative,58 to provide quick

access to accurate and reliable data needed by humanitarian decision 

makers. However, the usefulness of the database is limited, because NGOs 

are not reporting the survey results systematically and because the quality 

of many of the reported surveys is problematic. 

Moreover, mortality data from other sources, such as prospective or 

ongoing surveillance systems59 that systematically record deaths, are 

generally unavailable in Darfur. The research literature notes that existing

systems for registering or reporting deaths generally collapse when crises 

occur and that prospective surveillance systems established in crisis 

situations have limitations.60 However, the literature also notes that data 

from graveyard counts and regular reporting and surveillance systems 

have been used in some crises, despite such limitations, in conjunction 

with survey-based data to estimate mortality in crisis situations.61 In 

Darfur, data from surveillance systems were generally not available,

although one expert reported that systems had been set up in some of the 

larger camps by the middle of 2004. However, the expert also reported that 

these systems were capturing data in ways that would allow them to be 

used only qualitatively, rather than quantitatively, for death toll estimates. 

 58SMART coordinated by USAID, involves experts from U.S. government agencies, 

multilateral organizations, universities, and nongovernmental organizations. SMART has 

issued guidance on how to measure mortality, nutrition, and food security in crisis 

situations. CE-DAT serves as SMART’s primary data source for trend analysis and 

monitoring and reporting. 

59Mortality surveillance systems require teams of trained home visitors to record deaths in

the population as they occur. The home visitors are assigned a specific sector of the camp

or village and instructed to visit their assigned areas regularly. During their visits, they 

record the population size of the area and all deaths that have occurred.

60See World Health Organization, pp. 2-3, and Bradley Woodruff, “Violence and Mortality in 

West Darfur,” The Lancet, vol. 364 (2004), p. 1,290. 

61See Bradley Woodruff. “Interpreting mortality data in humanitarian emergencies,” The 

Lancet, vol. 367 (2006), p. 9. Also see Romesh Silva and Patrick Ball, “The Demography of

Conflict-Related Mortality in Timor-Leste (1974-1999): Empirical Quantitative Measurement

of Civilian Killings, Disappearances and Famine-related Deaths,” in Human Rights and 

Statistical Objectivity, J. Asher, D. Banks, F. Schueren, eds. New York: Springer 

(forthcoming). Also see Francesco Checchi and Les Roberts, Interpreting and Using 

Mortality Data in Humanitarian Emergencies: A Primer for Non-Epidemiologists, 

Network Paper no. 52 (London: Humanitarian Practice Network, 2005). 

Page 31 GAO-07-24 Darfur
Gaps in data on mortality can lead to reliance on extrapolations based on 

potentially risky assumptions and limited contextual information. Because 

the available data for Darfur cover certain geographic areas and time 

periods, the estimates that we examined assume that mortality rates for 

surveyed locations and specific periods can apply to unsurveyed locations 

and longer periods. Most of the experts we consulted voiced concerns 

about such extrapolations. Some experts noted that factors such as 

patterns of attack and displacement, humanitarian relief efforts, and the 

incidence of disease might cause mortality rates in the surveyed areas to 

differ from rates in the unsurveyed areas. One expert stated that he could

only speculate on conditions and mortality rates in the areas that had not 

been surveyed. In addition, several experts expressed concern about 

extrapolations from limited time periods to longer periods, noting that 

mortality rates can change rapidly.

Some of the estimates’ assumptions are informed by contextual 

information in the absence of data; however, this information also had 

limitations. For example, some estimates relied on anecdotal reports of 

conditions in Darfur, satellite imagery on attacks of villages, and 

information about weather conditions to determine appropriate mortality 

rates. While the experts generally approved of the use of contextual 

information, they did not believe that the information used could 

compensate for the gaps in the data. 

Limitations in the population data before and during a crisis such as 

Darfur’s can also impact researchers’ ability to produce reliable death 

estimates. No definitive estimate of Darfur’s pre-crisis population exists, 

and estimates of the current population vary considerably, from around 4 

to 7 million. The difficulty of estimating the region’s population is 

compounded by the fact that migration was widespread in Darfur before 

the crisis, making it difficult to get accurate counts. Because the death

estimates essentially extrapolate the mortality rates from the surveys to 

the entire population affected by the crisis, an estimate of the affected 

population that is too high or too low will lead to an over- or 

underestimate of the death toll. The estimates we reviewed generally used 

data for the affected population that were reported in the UN

Humanitarian Profiles; however, the collection and reliability of the profile 

Gaps in Data Leads to 

Reliance on Assumptions 

and Limited Information 

Limitations in Population 

Data Affect Reliability of 

Death Estimates 

Page 32 GAO-07-24 Darfur
data had limitations.62 For example, the profiles estimated a population of 

more than 160,000 IDPs in Kalma camp, the largest in Darfur, prior to 

October 2005. Yet, a subsequent count by a relief agency in October 2005 

reported only 87,000 IDPs in the camp. An expert also noted that figures 

based on registration can be unreliable and depend greatly on the agency 

responsible for registration. He stated that sources of bias include poor 

coverage by the agency (underestimation), as well as problems with 

populations’ being “double-counted” to increase the amount of aid 

delivered (overestimation). 

Varying approaches to the selection of a baseline for normal mortality can 

raise or lower death estimates, possibly making the totals overly high or 

low. All but one of the Darfur death estimates that we examined subtract

baselines for normal or expected mortality from the total deaths, based on 

the assumption that some deaths from disease, old age, or malnutrition 

would have occurred without the crisis. For example, CRED’s use of a 

baseline mortality rate of 0.3 subtracted about 16,000 deaths from the total 

estimate of 134,000, resulting in 118,000 “excess” deaths. If CRED had used 

a baseline of 0.5, it would likely have subtracted about 26,000 deaths, 

obtaining a somewhat lower estimate of deaths from the crisis.63

The experts we convened proposed various baseline mortality rates, 

ranging from 0.3 to 0.75 deaths per 10,000 affected persons per day.64 One 

reason for the difference in the suggested rates is the range of methods 

that the experts used to select them—for example, a rate that prevailed in

the country before the crisis, a standard based on prior humanitarian 

crises, or a rate from a comparable country in the region. However, each

Varying Use of Baseline 

Mortality Rates Can Affect 

Estimates’ Accuracy 

 62Estimates of the affected population reported in the UN Humanitarian Profiles are based 

on data provided by international humanitarian agencies and their implementing partners; 

the majority of information comes from the UN World Food Program food registration. 

These estimates do not include residents in the three state capitals of Darfur, Nyala, El

Fasher, and Geneina because their number is relatively large compared with the IDP 

population they are hosting. Officials from the UN Office for the Coordination of

Humanitarian Affairs told us that they had had difficulty verifying the reliability of data 

reported and insuring the uniformity of such data. 

63The Darfur death estimates used different baseline mortality rates. For example, CRED

used a baseline of 0.3 because UNICEF reported that as a national average prior to the

conflict. State used 0.5 because that was the WHO baseline mortality rate for sub-Saharan 

Africa. 

64In addition, some experts felt that it was appropriate to use two rates, one for the 

“normal” period and one for the “emergency” period of the crisis. For example, one expert 

proposed 0.3 for the normal period and 0.6 for the emergency period. 

Page 33 GAO-07-24 Darfur
of these methods has acknowledged limitations that could result in 

inaccurate estimates of expected rates and, therefore, in over- or 

underestimations of mortality due to the crisis. 

In addition, the experts debated whether a baseline of any sort was 

justified for a humanitarian crisis such as Darfur, arguing ethical and

philosophical, rather than technical, considerations. About half of the 

experts said that deaths that would have occurred regardless of the crisis 

should be subtracted from the death toll attributed to the crisis. However, 

two experts took a contrary position, arguing that the concept of expected

or normal levels of mortality was not appropriate in the presence of 

genocide or ethnic cleansing because the perpetrators of those crimes 

against humanity should be considered culpable for all deaths that 

resulted from the crises they instigated.65 Using a baseline to estimate 

mortality would lead to a somewhat smaller excess death toll than not 

using a baseline. For example, State’s estimate of total deaths ranged from 

98,000 to 181,000, minus 35,000 expected deaths; thus, State estimated 

63,000 to 146,000 excess deaths directly resulting from the crisis. 

The group of experts proposed and rated a wide range of measures that 

U.S. agencies could take directly, or support other entities in taking, to 

improve the quality and reliability of death estimates and relevant data for 

Darfur and future humanitarian crises. The measure rated most likely to 

produce the most improvements was ensuring sufficient public

documentation of estimates’ data and methods to allow replication of the 

methods, verification of the findings, and confirmation of the estimates’ 

credibility and objectivity. Other highly rated measures included collecting 

and maintaining temporal and spatial data, housing responsibility for 

making the estimates in a reputable independent body, improving the 

training of nongovernmental organizations’ staff who collect survey 

mortality data, and promoting an interdisciplinary approach to estimating 

mortality. U.S. agencies are engaged in several initiatives that address 

some of these measures. 

Wide Range of 

Measures Could 

Improve Death 

Estimates for Darfur 

and Future 

Humanitarian Crises 

 65The remaining experts either presented views for or against or did not comment on this

issue. An expert who made this argument also noted that the countries with higher rates of 

normal mortality would attribute relatively more deaths to “normal” factors than in 

countries with lower mortality rates. 

Page 34 GAO-07-24 Darfur
The experts suggested a series of measures covering a broad range of 

activities that U.S. agencies or other, U.S.-funded or -supported entities 

could take to improve death estimates for future humanitarian crises.66

(See table 2.) Although the experts acknowledged the importance and 

necessity of estimating mortality—for example, to help hold perpetrators 

accountable and to have a complete historical record—they generally

believed that death estimates should be conducted with caution. To assist 

in this endeavor, the experts proposed and rated 19 measures as likely to

produce some improvements in data collection and mortality estimation. 

Some experts differed as to whether government agencies or other entities 

would be best placed to implement particular suggestions. 

Proposed Measures Cover 

Wide Range of Activities 

Table 2: Measures Rated by Experts as Likely to Greatly Improve Death Estimates for Future Crises, in Order of Ranking and 

Number of Endorsements

Measure Number of endorsements

Ensure that publicly available documents on U.S. government estimates provide sufficient

information on methods, data, assumptions, and limitations. 11

Support the collection and maintenance of temporal and spatial data. 10

House responsibility for mortality estimates in a reputable, independent body or group. 9

Improve training of NGO staff who collect mortality survey data. 9

Promote an interdisciplinary approach to estimating mortality (include epidemiologists and

demographers). 9

Create technical teams, under the auspices of an international body, that can conduct mortality 

estimates as needed. 8

Report mortality and morbidity information more routinely and systematically to provide an ongoing 

sense of the situation. 8

Promote data collection by NGOs on the ground at routine service points in addition to periodic 

assessments. 7

Promote the use of other measures of a conflict’s severity (e.g., displaced persons, number of 

attacks) in addition to mortality estimates. 7

Improve existing surveying techniques (e.g., cluster sampling) by incorporating spatial or temporal

information. 7

Provide guidance (minimum standards) on how to design and implement survey instruments in the 

affected region (e.g., pretesting or translation techniques). 6

Provide guidance on amalgamating existing mortality/morbidity surveys. 5

Define criteria for selecting and using data. 4

 66The experts suggested that the U.S. agencies best able to implement these suggestions are 

the CDC, USAID, and State. Other, nongovernment entities include academic institutions, 

such as the Johns Hopkins University; multilateral organizations, such as the WHO; and 

NGOs such as Médecins Sans Frontières. 

Page 35 GAO-07-24 Darfur
Measure Number of endorsements

Make satellite tapes and imagery available to researchers. 3

Provide guidance on triangulating different types of data. 3

Provide assistance to local statistical agencies to improve data collection. 3

Develop algorithms to track population change over time. 3

Create a statistical unit under appropriate agency (possibly under auspices of the Committee on 

National Statistics) to be responsible for these types of estimates. 3

Tap other resources, such as pro bono groups of statisticians, that could provide assistance. 1

Source: GAO.

Note: The table is based on analysis of 12 experts in epidemiology, demography, statistics, and the 

Darfur crisis. Numbers shown under “Number of endorsements” represent the number of experts who 

rated the measure as likely to “very greatly” or “greatly” to improve the estimates. In several 

instances, one or two experts noted problems with suggestions that had been highly rated by others. 

The experts rated an additional measure for the Darfur crisis, namely, whether an independent 

agency should conduct a retroactive assessment.

Majorities of the experts rated 10 of the 19 measures, and 9 of the 12

experts rated 5 of the measures, as likely to “very greatly” or “greatly” 

improve the estimates for future crises.67 Following are the 10 most highly 

rated measures: 

• Ensure that publicly available documents on the U.S. government 

estimates provide sufficient information on methodology. Sufficient 

documentation and transparency of data and methods are needed to allow

independent researchers to verify the reliability and validity of estimates. 

Although this suggestion would not improve the estimates per se, experts 

felt that it was important for establishing the credibility of the estimates. 

• Support the collection and maintenance of temporal and spatial data. 

Temporal and spatial databases would allow researchers to track mortality 

over time and across regions and could improve researchers’ ability to 

estimate mortality. Experts suggested that temporal and spatial data could 

be collected in a number of ways, including through the use of satellite 

imagery. 

• House responsibility for mortality estimates in a reputable, independent

entity or group. Experts indicated that an independent entity could be

perceived as free of bias, political goals, or both. However, such an entity 

would need adequate funding, and experts had differing views as to where 

it should be housed. One expert felt it could be placed under a U.S. 

 67Experts rated the measures using the following 5-point scale: “Very greatly improve,”

“Greatly improve,” Moderately Improve,” “Somewhat Improve,” and “Not improve.” 

Page 36 GAO-07-24 Darfur
organization such as the CDC, while another felt it should be under an 

international organization such as the WHO. 

• Improve training of NGO staff who collect mortality survey data. 

Because NGO staff conduct most of the surveys, improving their training 

would help improve the quality of data collected. Several experts reported 

the existence of initiatives to improve and standardize NGOs’ data 

collection in emergencies, such as the SMART initiative, and of guidance

on the topic from the World Food Program and the WHO. Some experts 

suggested expanding these efforts with additional U.S. government 

support. 

• Promote an interdisciplinary approach to estimating mortality (include

epidemiologists and demographers). Because epidemiologists and 

demographers bring different expertise and perspectives to mortality 

estimation, collaboration between the disciplines could lead to greater 

understanding of each other’s approaches and, ultimately, to more 

integrated methodologies for death estimates. 

• Create technical teams, under the auspices of an international body, 

that can conduct mortality estimates as needed. Technical teams, 

assembled as needed and operating under independent organizations, 

could—rather than a single, permanent independent entity—be 

responsible for conducting death estimates. Such teams might be 

perceived as free of bias and political goals, which would increase the 

credibility of their findings. However, some experts reported that such 

technical teams already exist, citing as examples the WHO, the 

International Rescue Committee, and Médecins Sans Frontières, among 

others.68

• Report mortality and morbidity information more routinely and 

systematically to provide an ongoing sense of the situation. This 

suggestion is intended to allow researchers and governments to monitor 

crises as they develop. Although this suggestion was relatively highly 

rated, experts had some questions about who would do the reporting, how 

the data would be gathered, and how the estimates would be made. One 

expert noted that groups like CRED are currently reporting such mortality 

and morbidity information somewhat routinely and systematically. 

 68One expert felt that it would be preferable for the U.S. government to support the existing

technical teams. 

Page 37 GAO-07-24 Darfur
• Promote data collection by NGOs on the ground at routine service 

points, in addition to periodic assessments. Proponents stated that 

humanitarian relief organizations could collect data on famine-related 

deaths and that human rights organizations could collect narrative 

testimonies about political violence and conflict-related displacement. 

Some experts believed that the NGOs could collect these data with little 

additional time or expense and that the data collection could help create 

an early warning system for famine and diseases. However, several experts 

were concerned that data collected at routine service points would 

provide biased estimates because certain segments of the affected 

population would not be likely to pass through these points. 

• Promote the use of other measures of a conflict’s severity (e.g., displaced

persons, number of attacks) in addition to mortality estimates. Such 

alternative estimates could provide insights into crises and help check the 

accuracy of the mortality data. These estimates could include conflictrelated displacement, property destruction, property looting, exposure to 

violence as a marker for psychological trauma, incidence of sexual and 

gender-based violence, forced displacement, and data from security 

incidence reports. One expert indicated that some of these data are 

probably being collected by various human rights agencies and 

organizations and suggested creating a body to collate and report their 

data. 

• Improve existing surveying techniques (e.g., cluster sampling) by 

incorporating spatial or temporal information. A few experts felt that 

improvements in surveying techniques were needed because of limitations 

in certain aspects of cluster sampling, the most commonly used technique; 

however, another expert stated that cluster sampling69 was extremely 

useful. One expert argued that existing surveying techniques should take 

into account the spatial and temporal distribution of the affected 

population. 

 69Cluster sampling is a simple method that can be used to draw a representative sample 

even where there is no listing, or known total number, of households (that is, no individual 

household sampling frame) and where households are arranged in a chaotic pattern, as if 

often the case in IDP camps. See Francesco Checchi and Les Roberts, Interpreting and 

Using Mortality Data in Humanitarian Emergencies: A Primer for NonEpidemiologists, Network Paper no. 52 (London: Humanitarian Practice Network, 2005).

Page 38 GAO-07-24 Darfur
Several current U.S. initiatives may enhance the availability and quality of

mortality data and produce more accurate death estimates for Darfur and 

other similar crises in the future. These initiatives embody several of the 

measures rated by the group of experts as likely to improve death 

estimates; however, several of the experts, as well as U.S. officials 

knowledgeable about the initiatives, whom we spoke with indicated that 

more can be done. 

Existing U.S. Initiatives 

May Improve Some 

Aspects of Data and Death 

Estimates 

• CE-DAT may improve death estimates through its promotion of more 

routine reporting of mortality and morbidity information and defining 

some criteria based on methodological requirements for assessing the 

quality of data. In addition, CE-DAT includes a cartographical database to

link indicators to maps, which aligns with the experts’ recommendation 

for collecting and maintaining spatial and temporal data. CE-DAT staff 

discussed some limitations, with respect to the data in the database, 

including the fact that information is not provided to CE-DAT routinely 

and CE-DAT is not widely known. CE-DAT staff also said that there is 

currently no systematic means of verifying data reliability and validity of 

data and making data publicly available in a timely manner. 

• SMART, an interagency initiative coordinated by USAID, may improve 

death estimates through its promotion of a standard methodology to 

design and implement survey instruments that measure crude mortality, as 

well as other indicators of need.70 According to a USAID official, SMART is 

also intended to help build the technical capacity of NGO field workers 

and host government partners to collect and report more reliable data. To 

this end, the SMART initiative aligns with some of the recommendations 

made by our group of experts, including, improving the training of NGO 

staff that collect mortality data and reporting mortality and morbidity data

more routinely and systematically. Experts and officials whom we spoke 

with generally agreed that the idea behind SMART represents positive 

steps toward improving mortality data collection. However, they also 

discussed various limitations of the initiative. For example, an official 

involved with the effort noted that that funding has been at times sporadic 

and that limited resources have stymied outreach efforts. Others stated 

that some of SMART’s methodological principles need further research 

and testing in the field and that no consensus currently exists regarding 

some of these principles. Officials from USAID indicated that SMART’s 

major components, including mortality, nutrition, and food security, have 

been piloted in several countries and that the SMART methodology is 

 70SMART initiative is aimed at improving the monitoring, reporting, and evaluation of 

humanitarian assistance interventions. (See www.smartindicators.org.) 

Page 39 GAO-07-24 Darfur
meant to be iterative with continuous upgrading based on further research 

and best practices. 

• The Humanitarian Information Unit, housed in State’s Bureau of 

Intelligence and Research, may enhance the quality and availability of 

spatial and temporal information used in future death estimates through 

its current efforts to develop of maps and other visual images of 

humanitarian crises. For example, for the Darfur crisis, it has developed a 

series of maps showing the number of destroyed villages or reported 

attacks in the region at different points in time based on data from satellite 

imagery or reports from on the ground organizations such as the African 

Union, the UN, and USAID. The unit’s work also addresses two additional 

suggested measures—promoting the use of other measures of conflict and

improving existing surveying techniques by incorporating spatial and

temporal data. However, the Humanitarian Information Unit, in some 

cases, has had difficulty obtaining standardized, reliable, or complete data. 

Additionally, the unit does not consistently and systematically collect data, 

such as preconflict population estimates, in part because the unit performs 

its work upon request. 

• The International Emergency and Refugee Health Branch of the CDC

may improve death estimates and the collection of mortality data through 

its provision of technical assistance to UN and U.S. agencies collecting 

data in complex emergencies and its support of the collection of spatial 

and temporal data.71 However, officials from the International Emergency 

and Refugee Health Branch stated that because it often works as a 

consultative body and its resources are constrained, the scope of its work 

is limited. 

Several international initiatives also address problems with data collection

and analysis. The WHO plans to implement a Health and Nutrition 

Tracking Service that would routinely monitor mortality and malnutrition 

during major crises and provide a central help desk, remotely located, for 

field staff conducting surveys. Additionally, the World Food Program has 

created a manual on survey techniques to measure health and nutrition 

indicators, including crude mortality. 

The group of experts we convened were generally aware of, or involved,

with these U.S. government and international initiatives, as well as some 

 71For example, members from the International Emergency and Refugee Health Branch

provided technical training on data collection to the World Food Program for two mortality 

surveys conducted in Darfur. 

Page 40 GAO-07-24 Darfur
NGO initiatives, that addressed certain aspects of their suggestions. For 

example, three experts assisted in the development of the SMART 

guidelines, and two of the three were also technical advisors for the CEDAT initiative. Some experts felt that certain suggestions could be met by

investing more in these existing initiatives rather than by creating new 

ones. U.S. officials responsible or aware of these initiatives also 

acknowledged limitations with several of these or thought that more could 

be done with respect to the measures suggested by the experts. 

Despite variance among the death estimates we reviewed, each of the 

estimates shows, tragically and unquestionably, that many thousands of 

civilians died in Darfur between February 2003 and August 2005. Given the 

continuing conflict, insecurity, and displacement in the region, many more 

have undoubtedly died since the estimates were conducted, as a direct 

result of violence or because of increased vulnerability to disease and 

malnutrition. However, despite the importance of the death estimates in 

showing the severity of the crisis, none of the estimates consistently 

received high ratings in terms of accuracy or methodological strengths 

from the experts we convened. Some of the shortcomings in the estimates’ 

source data, methods, and objectivity may be attributed to challenges in 

mortality data collection and extrapolation, characteristic of a

humanitarian crisis such as Darfur. However, in certain cases, a lack of 

transparent reporting of an estimate’s data, methods, assumptions, or 

limitations hindered the experts from replicating it and thus verifying its 

accuracy and credibility. In particular, many experts noted the 

unavailability of these published details—and the resulting lack of 

transparency—for State’s estimate. 

Several ongoing U.S. initiatives may produce enhancements that align with 

the experts’ suggested measures for addressing gaps in data and improving 

death estimates conducted for Darfur and any similar humanitarian crises 

in the future. For example, the SMART initiative and CE-DAT, 

respectively, provide guidance on how to design and implement surveys 

and promote more routine reporting of mortality and morbidity data. 

However, experts and U.S. government officials observed that gaps in 

these areas still exist. Among the measures that have not been addressed,

the one that the experts rated most highly—ensuring the public availability 

of information on estimate methods, data, assumptions, and limitations—

is essential to protect the credibility of U.S. government death estimates 

for Darfur and any future humanitarian crises. 

Conclusions 

Page 41 GAO-07-24 Darfur
To safeguard the Department of State’s credibility as a source of accurate

and reliable death estimates, we recommend that the Secretary of State

promote greater transparency in any of its future death estimates for 

Darfur or other humanitarian crises by ensuring that publicly available 

documents contain sufficient detail on the estimates’ data, methods, 

assumptions, and limitations to allow external researchers to replicate and 

verify the estimates. 

Additionally, to enhance the U.S. government’s capacity to assess the 

dimensions of, and respond appropriately to, any future humanitarian 

crises, we recommend that the Secretary of State and the Director of U.S. 

Foreign Assistance and USAID Administrator consider the experts’ other 

suggested measures to help address gaps in data and improve the quality 

and reliability of any future death estimates. 

We provided a draft of this report to the Department of State and USAID. 

State and USAID responded with formal comments, agreeing with our 

recommendations, and State provided additional perspectives on reporting 

and documentation regarding its death estimate. Reproductions of these 

letters, as well as our responses to the letters, can be found in appendixes 

VI and VII. We also provided a draft to the CDC for technical review, and 

we received technical comments from both the CDC and State, which we 

incorporated in the report as appropriate. 

We provided the authors of the other five estimates the portions of the 

report pertaining to their individual estimates. The authors of the CRED 

estimates agreed with the experts’ evaluation. Jan Coebergh did not 

provide any major comments regarding the relevant portion of the report

and indicated that he was aware of the limitations of his estimate. John 

Hagan expressed concerns regarding the experts’ relatively positive 

evaluation of the estimates by CRED and State and believed we should 

have included the estimate he did with Alberto Palloni in our evaluation. 

David Nabarro from the WHO disagreed with our inclusion of his estimate 

with the others in the report and said that we had not clearly conveyed the 

estimate’s purpose. Eric Reeves disagreed with the experts’ criticism of 

the Atrocities Documentation Team's survey of Chad refugees, stating that

the survey was well conducted and a critical source of data to estimate 

violent deaths, particularly early on in the conflict when little data was 

available. More details on the authors’ comments and our responses can 

be found in appendix VIII. 

As agreed with your offices, unless you publicly announce the contents of

this report earlier, we plan no further distribution until 30 days from the 

Recommendations for 

Executive Action 

Agency Comments 

and Our Evaluation 

Page 42 GAO-07-24 Darfur
report date. At that time, we will send copies of this report to the Secretary 

of State, the Director of U.S. Foreign Assistance and USAID Administrator, 

the Director of the CDC, relevant congressional committees, and other 

interested parties. We will also make copies available to others on request. 

In addition, the report will be available on GAO’s Web site at 

http://www.gao.gov. 

If you or your staff have any questions about this report, please contact 

either David Gootnick at (202) 512-3149 or gootnickd@gao.gov, or Nancy 

Kingsbury at (202) 512-2700 or kingsburyn@gao.gov. Contact points for 

our Offices of Congressional Relations and Public Affairs may be found on

the last page of this report. Key contributors are listed in appendix IX. 

David Gootnick, Director

International Affairs and Trade 

Nancy Kingsbury, Managing Director 

Applied Research and Methods 

Page 43 GAO-07-24 Darfur
Appendix I: Objectives, Scope, and Appendix I: Objectives, Scope, and Methodology 

Methodology 

This report (1) evaluates the relative accuracy and methodological 

strengths and shortcomings of six death estimates for Darfur, including 

one by the Department of State (State); (2) identifies general challenges to

estimating the total death toll in Darfur and similar humanitarian crises; 

and (3) discusses measures that the U.S. government could take to 

improve its death estimates for Darfur and any similar, future crises. 

To evaluate the relative accuracy and methodological strengths and 

shortcomings of the Darfur death estimates, we selected estimates that 

had been made public prior to March 2006. We imposed this cutoff point 

so that we could interview the authors, prepare materials for the experts, 

and give the experts time to review the materials. Because our cutoff point

was March 2006, the convening experts’ discussion was of death estimates 

that spanned various time periods between February 2003 and August 

2005. Our findings do not, therefore, discuss any estimates that were made 

after March 2006. 

Additionally, to address these objectives, we convened, in collaboration 

with the National Academy of Sciences, 12 experts for an all-day meeting 

on April 7, 2006, in Washington, D.C., to discuss the six death estimates. 

(Two additional experts also participated by phone for parts of the day.) 

The academy proposed lists of experts, and we approved their final 

selection. The selected experts had extensive knowledge of estimating 

mortality rates and death totals in conflict and postconflict situations or 

other types of humanitarian crises and were familiar with the sorts of data 

and estimates that have been produced for the Darfur crisis or other such 

crises in Africa. Additionally, the selected experts represented a range of 

professional experiences and backgrounds, including epidemiologists, 

demographers, statisticians, and directors of aid programs currently in 

Darfur. They were affiliated with various organizations, including 

universities, U.S. government and United Nations (UN) agencies, and 

humanitarian aid and nonprofit organizations based in the United States,

as well as overseas. All of the experts signed a form from the National 

Academy of Sciences certifying that they had no conflicts of interest that 

could compromise their ability to assess the death estimates objectively. 

Some experts reported they had had professional contact with several of 

the authors.

During the meeting, the experts discussed their evaluation of each of the 

Darfur death estimates, as well as challenges to estimating total deaths for

Darfur and similar crises and measures to improve such estimates. The 

meeting was recorded and transcribed to ensure that we accurately 

captured the experts’ statements, and we reviewed the transcripts as a 

Page 44 GAO-07-24 Darfur
Appendix I: Objectives, Scope, and 

Methodology 

source of evidence. (See app. II for a list of the experts’ names and 

affiliations and a summary of the meeting agenda and discussion 

questions.) 

In addition, prior to the meeting, we reviewed and analyzed public 

information on the estimates, including documents describing the 

estimates and source data for the estimates and provided this information 

to the experts.1

 The source data documents that we reviewed included 15 

reports of health, nutrition, and mortality surveys; State publication 

describing the July-August 2004 survey with refugees in Chad; and 

volumes of the UN Nutrition Information in Crisis Situations containing 

descriptions of additional health, nutrition, and mortality surveys of which 

we were unable to obtain full reports. (See app. III for a complete list of 

published documents that were provided to the experts.) We also provided 

the experts supplemental information on the design, sampling and 

implementation of the Atrocities Documentation Team’s survey of Chad 

refugees and a copy of the survey instrument, which we obtained from 

representatives of the Coalition of International Justice. Additionally, we

interviewed the estimates’ authors regarding the data, methods, and 

objectives of their work and replicated their estimates when we had 

sufficient information. We provided summaries of these interviews to the 

experts, as well as replications of the estimates, reviewed and approved by 

the authors, in advance of the meeting. We spoke with all of the estimate 

authors except State’s Bureau of Intelligence and Research, which 

declined to speak with us or provide additional information. Although this 

limited the ability of our experts to fully understand State’s methods of 

analysis as noted during their discussion, the experts were able to discuss 

State’s estimate in detail and provide assessments of its data, methods, 

objectivity, limitations, and accuracy.

Following the meeting, we also asked the experts to answer additional 

questions on each estimate, as well as questions on suggested measures to 

improve estimates through a follow-up data collection instrument. We 

developed the instrument with the help of survey specialists and based the 

questions in the instrument on the meeting agenda and points that arose 

during the meeting deliberations. We pretested the instrument with two 

 1

As noted in the report, we and the group of experts also reviewed a preliminary death 

estimate for West Darfur by John Hagan and Alberto Palloni, but we do not discuss the 

estimate in this report because the estimate had not been published. The authors told us 

that the version of the estimate they gave us was preliminary and that they were working

on a more refined version of the estimate using different methods. 

Page 45 GAO-07-24 Darfur
Appendix I: Objectives, Scope, and 

Methodology 

experts and made changes based on their input. We administered the 

instrument via e-mail and received responses from all 12 of the experts 

who attended the meeting in Washington, D.C. In some instances, we 

contacted the respondents by e-mail or phone to obtain greater clarity or 

details regarding their answers. 

Further, to identify challenges involved in estimating total deaths in 

humanitarian crises, such as that in Darfur, we asked the group of experts 

to highlight key challenges during their discussion, and we reviewed 

literature related to death estimates and mortality data for humanitarian 

crises. We summarized the parts of the experts’ discussion that most 

directly addressed challenges, and we identified themes that were raised 

by the experts. The literature we reviewed included articles that we 

identified through databases, such as ProQuest, Lexis Nexis, and Medline, 

using various search terms, such as mortality estimates, death estimates,

humanitarian crise(s), conflict(s), or (complex) emergency/

emergencies. We also reviewed literature cited in these articles and on 

Web sites related to humanitarian assistance or data analysis in 

humanitarian crises, as well as literature recommended by the group of 

experts2

 and the authors of the estimates. In total, we reviewed about 20 

articles. We organized the individual challenges identified by the experts 

and in the literature according to the two overarching themes we 

identified from the experts’ discussion and from the literature: (1) 

challenges affecting the source data used for the death estimates and (2) 

challenges affecting the generation of the death estimates. We also 

provided portions of the draft pertaining to the challenges to conducting 

death estimates as well as the background regarding mortality surveys to 

several of the experts we convened and incorporated their technical 

comments into the final version of the report. 

Finally, to identify measures that the U.S. government could take to 

improve death estimates for Darfur and in future humanitarian crises, we 

asked the experts to provide suggestions during the meeting and solicited 

further opinions on these suggestions in the follow-up instrument. The 

instrument listed 19 measures that the experts had suggested as likely to 

improve death estimates for Darfur and such crises that may occur in the 

future and asked the experts to rate them with a five-point scale, ranging 

 2

Over half of our experts were also authors or coauthors on some of the articles we 

reviewed. 

Page 46 GAO-07-24 Darfur
Appendix I: Objectives, Scope, and 

Methodology 

from “Very greatly improve” to “Not improve.” 3 We ranked the measures 

according to the numbers of respondents that rated them “Very greatly 

improve” and “Greatly improve.” The instrument also asked for experts’ 

comments on each of the suggested measures, and we followed-up with 

some experts to get additional information regarding comments on 

suggested measures, particularly with respect to current efforts. 

Additionally, we spoke with officials from the U.S. Agency for 

International Development, State, and the Department of Health and 

Human Services’ Centers for Disease Control and Prevention to learn of 

any current U.S. government initiatives related to the suggested measures. 

We also reviewed public information and documents provided by the 

officials and available on the Internet that describe efforts by the U.S. 

government, as well as other international initiatives. 

We conducted our work in San Francisco and Washington, D.C., from 

September 2005 to November 2006 in accordance with generally accepted 

government auditing standards. 

 3

The experts also rated an additional measure for the Darfur crisis regarding whether an

independent agency should conduct a retrospective assessment. 

Page 47 GAO-07-24 Darfur
Appendix II: List of

Ag

 Experts and Meeting 

enda 

Page 48 GAO-07-24

Appendix II: List of Experts and Meeting 

Agenda 

This appendix provides the names and affiliation of the experts 

participating in the meeting held on April 7, 2006, in Washington, D.C., and 

a summary of the day’s discussion questions. 

List of Experts The following experts attended the meeting: 

• Jana Asher, American Association for the Advancement of Science 

• Richard Brennan, Health Unit, International Rescue Committee 

• Francesco Checchi, London School of Hygiene and Tropical Medicine 

• Allan Hill, Harvard School of Public Health, Harvard University 

• Arif Husain, Vulnerability Analysis and Mapping Unit, United Nations 

World Food Program 

• Mark Myatt, University College of London 

• W. Courtland Robinson, Bloomberg School of Public Health, Johns 

Hopkins University 

• William Seltzer, Department of Sociology and Anthropology, Fordham 

University 

• Romesh Silva, Human Rights Data Analysis Group, The Benetech Initiative 

• Michael VanRooyen, Program on Humanitarian Crises and Human Rights, 

Harvard University 

• Ronald Waldman, Mailman School of Public Health, Columbia University 

• Bradley Woodruff, Maternal and Child Nutrition Branch, U.S. Centers for 

Disease Control and Prevention 

The following experts participated in the meeting by phone for parts of the 

day: 

• Bushra Gamar Hussein, Darfur Region, Sudan Social Development 

Organization 

• Jennifer Leaning, Harvard School of Public Health, Harvard University 

 Darfur
Appendix II: List of Experts and Meeting 

Agenda 

Darfur Death Estimates 

• Are the source data used in each of the estimates sufficiently 

representative and reliable? 

Summary of Meeting 

Discussion Questions 

• To what extent do the methods used in each of the estimates follow 

principles that are generally accepted in the field?

• Do the key assumptions made in the estimates seem reasonable? 

• How accurate or inaccurate do you think each of these estimates are in 

terms of representing the actual number of deaths that occurred in Darfur 

during the time period and regions under consideration? 

• For each of the estimates, is sufficient information presented in the 

reports to adequately assess the strengths and weaknesses and/or the 

reasonableness of the estimates? Do you understand the processes by 

which the estimates were derived? 

• What are the major strengths and limitations of each of these estimates for 

Darfur? 

Challenges to Estimating Total Deaths for Darfur and Similar Crisis 

• In your opinion, what sources of data for what time periods and locations

would be necessary to produce reasonably precise estimates of death 

totals for Darfur? 

• What general observations do you have concerning the challenges of 

estimating total deaths in Darfur? 

Implications for the U.S. Government: Ways to Improve Death Estimates 

for Darfur and Future Crises

• In your view, are there any lessons learned about conducting death 

estimates in Darfur for the U.S. government? If so, what are these? 

• Are there measures the U.S. government could take or promote to improve 

estimates for Darfur or similar crises in the future? 

Page 49 GAO-07-24 Darfur
Appendix III: L

Mortal

Additiona

ist of Death Estimates and 

ity Surveys Provided to Experts and

l Bibliographical References 

Page 50 GAO-07-24

Appendix III: List of Death Estimates and 

Mortality Surveys Provided to Experts and 

Additional Bibliographical References 

Death Estimates Estimate by the U.S. Department of State 1

• State Department’s Fact Sheet, “Sudan: Death Toll in Darfur,” March 25, 

2005. www.state.gov/s/inr/rls/fs/2005/45105.htm (Viewed, August 24, 2006)

• M. Phelan in D. Guha-Sapir and O. Degomme, “Darfur: Counting the 

Deaths” (Method 2). Centre for Research on the Epidemiology of Disasters 

(CRED). Brussels: May 26, 2005. 

www.cred.be/docs/cedat/DarfurCountingTheDeaths-withClarifications.pdf

(Viewed October 23, 2006) 

Estimate by the Centre for Research on the Epidemiology of 

Disasters: Debarati Guha-Sapir and Olivier DeGomme 

• Guha-Sapir, D., O. Degomme, and M. Phelan, “Darfur: Counting the 

Deaths” (Method 1). Centre for Research on the Epidemiology of Disasters 

(CRED). Brussels: May 26, 2005. 

www.cred.be/docs/cedat/DarfurCountingTheDeaths-withClarifications.pdf

(Viewed October 23, 2006) 

• Guha-Sapir, D. and O. Degomme, “Darfur: Counting the Deaths (2): What 

are the trends?” Centre for Research on the Epidemiology of Disasters 

(CRED). Brussels: December 15, 2005. 

www.cred.be/docs/cedat/DarfurCountingTheDeaths2.pdf (Viewed October 

23, 2006) 

Estimate by Jan Coebergh 

• Coebergh, J. “Sudan: genocide has killed more than the tsunami.” 

Parliamentary Brief, vol. 9, no. 7, pp. 5-6 (February 2005). 

www.thepolitician.org/ (Viewed, August 24, 2006)

Estimate by John Hagan, Wenona Rymond-Richmond, and Patricia

Parker and announced by the Coalition for International Justice 

• Coalition for International Justice Press Release, “New Analysis Claims 

Darfur Deaths Near 400,000.” April 21, 2005. 

 1

As noted in the report, the experts reviewed a death estimate of West Darfur by John 

Hagan and Alberto Palloni, but this estimate is not discussed in this report because it was 

not publicly available. 

 Darfur
Appendix III: List of Death Estimates and 

Mortality Surveys Provided to Experts and

Additional Bibliographical References 

• Hagan, J., W. Rymond-Richmond, and P. Parker, “The Criminology of 

Genocide: The Death and Rape of Darfur,” Criminology, vol. 43, no. 3, pp. 

525-561 (July 2005).

Estimate by Eric Reeves 

• Portions of mortality updates from www.sudanreeves.org. (Reviewed by

author.) Dr. Reeves began reporting his mortality updates in June 11, 

2004. The most recent update reviewed by the experts was from August 

31, 2005. 

Estimate by the World Health Organization 

• Nabarro, D. “Mortality Projections for Darfur,” World Health Organization 

Briefing presented October 15, 2004.

www.who.int/mediacentre/news/briefings/2004/en (Viewed, August 24, 

2006) 

State or regionwide estimates 

• U.S. Centers for Disease Control and Prevention and UN World Food 

Program, “Emergency Nutrition Assessment of Crisis Affected Populations 

Darfur Region, Sudan, August-September 2004.” 

www.reliefweb.int/library/documents/2004/wfp-sdn-26ocr.pdf. (Viewed, 

August 24, 2006) 

Surveys Used in Estimates 

as Data Sources 

• UN World Food Program, “Emergency Food Security and Nutrition 

Assessment in Darfur, Sudan.” (Rome, Italy: World Food Program, October 

2004).www.wfp.org/country_brief/africa/sudan/assessments/041025_food_

assessment.pdf (Viewed, August 24, 2006) 

• World Health Organization, “Retrospective Mortality Survey Among the 

Internally Displaced Population Greater Darfur, Sudan 2004.” September 

15, 2004. www.reliefweb.int/library/documents/2004/who-sdn-15sep.pdf

(Viewed, August 24, 2006) 

• World Health Organization, “Mortality Survey among Internally Displaced

Persons and Other Affected Populations in Greater Darfur, Sudan.” 

September 2005. 

www.emro.who.int/sudan/pdf/CMS%20Darfur%202005%20final%20report_1

1%2010%2005.pdf (Viewed, August 24, 2006) 

Page 51 GAO-07-24 Darfur
Appendix III: List of Death Estimates and 

Mortality Surveys Provided to Experts and

Additional Bibliographical References 

North Darfur 

• Dubray, C., “Health assessment in emergencies: Serif Umra, North Darfur, 

Sudan.” Epicentre and Medecins Sans Frontiers. November 2004. 

• Sibson, V., “Findings of a nutrition survey of Kutum town, Kasab camp and 

Fata Borno, North Darfur.” GOAL. May 2005.2

West Darfur

• Abadallah, S., “Report of rapid baseline survey Azirni, Sanidadi and Um 

Tagouk.” Center for International Emergency, Refugee and Disaster 

Studies, Johns Hopkins University and World Relief. October 12, 2004. 

• Deconinick, H. and O. Karouri, “Nutrition assessment; Fur Baranga, 

Administrative Unit, Habila Locality, West Darfur, Sudan.” Save the 

Children, USA. January 14-17, 2005. 

www.humanitarianinfo.org/darfur/uploads/assessments/fb%20nutrition%20

survey%20report%20jan%2005.doc (Viewed, August 24, 2006) 

• Depoortere, E., F. Checchi, F. Broillet, S. Gerstl, A. Minettia, O. Gayraud et 

al., “Violence and mortality in West Darfur, Sudan (2003-2004): 

epidemiological evidence from four surveys” Lancet, vol. 364, no. 9442, 

pp. 1,315-1,320 (2004). www.msf.fr/documents/base/2004-10-01-

Depoortere.pdf (Viewed, August 24, 2006) 

• Depoortere, E., “Health assessment in emergencies: Murnei and Zalingei, 

West Darfur, Sudan.” Epicentre and Médecins Sans Frontières. (Paris, 

France: Médecins Sans Frontières, June 2004). 

www.doctorswithoutborders.org/publications/reports/2004/epicentre_ 

report_darfur_05-2004.pdf (Viewed, August 24, 2006) 

• Hearns, A. and A. Agar, “Nutritional Survey, Mornei Camp, JanuaryFebruary 2005.” Concern Worldwide in collaboration with State Ministry 

of Health and UNICEF. 

 2

Survey included in CRED December 2005 update. Note: the report included here was only 

publicly available in HTML format. Therefore, some of the appendixes and formatting did 

not come through. 

Page 52 GAO-07-24 Darfur
Appendix III: List of Death Estimates and 

Mortality Surveys Provided to Experts and

Additional Bibliographical References 

• Rio, D., “Nutrition and Food Security Assessment, Wade Saleh and Mukjar 

Provinces, West Darfur, North-Sudan.” Médecins Sans Frontières Holland. 

April 2004. 

• West Darfur State, Ministry of Health in collaboration with UNICEF, 

Concern and Save the Children-US. “Nutrition and Mortality Survey in 

Ardamat, Dorti, Riyad and Abu-zar IDP camps.” June 14-18, 2005. 

South Darfur 

• Grandesso, F., F. Sanderson, J. Kruijit, T. Koene, and V. Brown, “Mortality 

and Malnutrition among Populations Living in South Darfur, Sudan.” 

JAMA, vol. 293, no. 12, pp. 1,490-1,494 (March 2005). http://jama.amaassn.org/cgi/content/abstract/293/12/1490 (Viewed, August 24, 2006) 

Chad 

• United Nations High Commissioner for Refugees et al., “Emergency 

Nutrition and Mortality Surveys Conducted among Sudanese Refugees and 

Chadian Villagers, Northeast Chad, June 2004.” 

http://www.cdc.gov/nceh/ierh/ResearchandSurvey/Chad_report04.pdf

(Viewed, August 24, 2006) 

Other Information Provided 

• UN Nutrition Information in Crisis Situations, volumes 3, 4, 6, 7 from 2004

and 2005. 

http://www.unsystem.org/scn/publications/RNIS/countries/sudan.htm

(Viewed, August 24, 2006) 

• Bureau of Democracy, Human Rights and Labor and Bureau of Intelligence 

and Research, Department of State, “Documenting Atrocities in Darfur,” 

Publication 11182 (Washington, D.C.: Bureau of Democracy, Human 

Rights, and Labor and the Bureau of Intelligence and Research, September 

2004). http://www.state.gov/g/drl/rls/36028.htm (Viewed, August 24, 2006) 

Boss, L., M. Toole, and R. Yip, “Assessment of Mortality, Morbidity and 

Nutritional Status in Somalia During the 1991-1992 Famine: 

Recommendations for Standardization of Methods,” JAMA, vol. 272, no. 5 

(1994). 

Additional Bibliographical 

References 

Page 53 GAO-07-24 Darfur
Appendix III: List of Death Estimates and 

Mortality Surveys Provided to Experts and

Additional Bibliographical References 

Checchi, F. and L. Roberts. “Interpreting and using mortality data in 

humanitarian emergencies: A primer for non-epidemiologists,” 

Humanitarian Practice Network, Network Paper no. 52. (London, UK, 

2005). 

Coghlan, B., R. Brennan, P. Ngay, D. Dofara, B. Otto, M. Clements, and T. 

Stewart, “Mortality in the Democratic Republic of Congo: a nationwide 

survey,” The Lancet, vol. 367, pp. 44-51 (2006). 

Guha-Sapir, D. and W. van Panhuis, “Conflict Mortality: Comparing the 

data.” CE-DAT Working Paper, series no. CD/2004/001. (2004). 

Hofman, C., L. Roberts, J. Shoham, P. Harvey, “Measuring the impact of 

humanitarian aid: A review of current practice,” Humanitarian Policy 

Group. (London, UK, 2004). 

Leaning, J. and M. VanRooyen. “An assessment of mortality studies in 

Darfur, 2004-2005,” Humanitarian Exchange, vol. 30, pp. 23-26 (2005). 

Myatt M., A. Taylor, and W. Courtland Robinson, “A method for estimating 

mortality rates using previous birth history,” Field Exchange, p. 13 (2002). 

Reed, H. (Rapporteur),“Demographic Assessment Techniques in Complex 

Humanitarian Emergencies, Summary of a Workshop.” National Academy 

Press, Washington, D.C. (2002). 

Reed, H. and C. B. Keeley (eds.), Forced Migration and Mortality. 

National Academy Press, Washington, D.C. (2001). 

Salama, P., P. Spiegel, L. Talley, and R. Waldman, “Lessons Learned from 

complex emergencies over past decade,” The Lancet, vol. 364, pp.1,801-

1,813 (2004).p

Silva, R. and P. Ball, “The Demography of Conflict-Related Mortality in 

Timor-Leste (1974-1999): Empirical Quantitative Measurement of Civilian 

Killings, Disappearances & Famine-Related Deaths.” In Human Rights 

and Statistical Objectivity. J. Asher, D. Banks, F. Schueren, eds. New 

York: Springer (forthcoming). 

Spiegel, P., P. Salama, S. Maloney, and A. van der Veen, “Quality of 

Malnutrition Assessment Surveys Conducted During Famine in Ethiopia,” 

JAMA, vol. 292, no. 5 (2004). 

Page 54 GAO-07-24 Darfur
Appendix III: List of Death Estimates and 

Mortality Surveys Provided to Experts and

Additional Bibliographical References 

Thieren, M., “Health information systems in humanitarian emergencies,” 

Bulletin of the World Heath Organization, vol. 83, no. 8 (2005). 

Toole, M. and R. Waldman, “The Public Health Aspects of Complex 

Emergencies and Refugee Situations,” Annual Review of Public Health, 

vol. 18, pp. 283-312 (1997). 

U.S. Centers for Disease Control and Prevention and The World Food 

Program, A Manual: Measuring and Interpreting Malnutrition and 

Mortality. Washington and Rome. (2005). 

Woodruff, B., “Violence and mortality in West Darfur.” The Lancet, vol. 

364, pg. 1,290 (2004). 

Woodruff, B., “Interpreting mortality data in humanitarian emergencies,” 

The Lancet, vol. 367, pg. 9 (2006). 

World Food Program, “Emergency Food Security and nutrition assessment 

in Darfur, Sudan 2005.” Rome. (December 2005). 

documents.wfp.org/stellent/groups/public/documents/ena/wfp089682.pdf

(Viewed October 23, 2006) 

World Health Organization, “Module 4. Studying Health Status and Health

Needs.” 

www.who.int/hac/techguidance/tools/disrupted_sectors/module_04/en/ind

ex2.html (Viewed October 23, 2006)

Page 55 GAO-07-24 Darfur
Appendix IV: Summary Description of Death 

Estimates Reviewed 

Estimate by Jan Coebergh 

Reported figures • Three possible point estimates for excess deaths: 218,449; 253,573;or 306,130 

represent a range of death estimates all above 200,000. 

Time period covered • April 2003 through December 2004. 

Total number of months • 21 

Data sources • Uses varied sources including Atrocities Documentation Team’s survey of Chad 

refugees based on number of people who reported seeing family member killed, 2004 

mortality surveys by World Health Organization (WHO) and Médicins Sans Frontiers 

and estimates by WHO officials. 

• Does not use any contextual or nonsurvey information. 

Mortality rates applied and

cause of death

• Calculates violent and nonviolent deaths separately. 

• 306,130: Includes nonviolent and violent deaths. About 170,000 from violence, 108,000 

from health causes, and 25,000 in inaccessible areas. 

• 218,449: Includes nonviolent and violent deaths. About 73,700 from violence and 

126,000 from health causes 

• 253,573: Includes nonviolent and violent deaths. About 111,000 from health causes 

and about 143,000 from violence 

• Applies some daily mortality rates; other mortality rates are monthly or for longer 

periods of time. 

Mortality changes over time and/or

region

• Assumes the same mortality levels over time and per region.

Baseline crude mortality rate assumed • 0.5 

Affected population included in

estimate

• Uses one “fixed” population estimate of 400,000 which is the average of an assumed 

starting point of 0 in February 2003 and 800,000 in April 2004. (Affected population 

does not change over time.) 

• Estimate includes affected population from all regions in Darfur and refugees in Chad. 

Appendix IV: Summary Description of Death 

Estimates Reviewed 

Page 56 GAO-07-24 Darfur
Appendix IV: Summary Description of Death 

Estimates Reviewed 

Centre for Research on the Epidemiology of Disasters (CRED): Two Estimates

Reported figures • 134,000 total deaths. 

• 118,142 excess deaths. 

• 36,237 total deaths. 

• 23,658 excess deaths. 

Time period covered • September 2003 through January 2005. • February 2005 through June 2005. 

Number of months • 17 • 5 

Data sources • Examines data from about 20 mortality surveys. About half of 

these were used to estimate nonviolent mortality rates, and half 

were used to analyze the proportion due to violence. (Surveys 

used depended on what surveys reported the proportion of 

deaths due to violence.) 

• Some steps were taken to examine the reliability of methods

used in the surveys and exclude outliers. 

• Examines about 10 surveys conducted 

between January and August 2005; 

however, primarily uses findings from 

the 2005 statewide WHO survey 

because these results did not differ 

with other smaller surveys that covered 

the same period. 

• Relies mainly on mortality survey findings and uses other sources of contextual or nonsurvey 

information to refine both estimates. 

Mortality rates

applied and cause 

of death 

• Estimates nonviolent and violent deaths separately. Total crude mortality is nonviolence crude mortality 

divided by (1 minus the proportion due to violence). 

• Nonviolent mortality is generally assumed stable except for the 

hunger (or rainy season) during June through August 2004. 

• Proportion due to violence is based on a regression analysis of 

survey data. 

• 2005 WHO survey findings include 

crude mortality rates and proportion of 

deaths due to violence or injury. 

Mortality changes

over time and/or

region

• Nonviolent mortality rate is assumed stable at 1.1 for all 

regions except June through August 2004 for which rates are

higher due to impact of rainy season. (Rates used are based on 

results from the 2004 WHO survey for each region.) 

• Based on regression analysis, proportion of violence generally 

decreases over time beginning December 2003 to January

2005. Prior to December 2003, assumes steady rate of 

violence. 

• Assumes some difference in violence between regions. 

• Assumes different mortality levels per 

region but no difference over time from 

February to June 2005. 

• Shows increase in violence for North 

Darfur from last estimate. 

Baseline crude

mortality rate 

assumed

• 0.3 based on national average from the United Nations 

Children’s Fund (UNICEF). 

Affected population 

included in estimate 

• Assumes different monthly estimates of the affected population 

based on information from the United Nations Humanitarian

Profiles for each Darfur state from September 2003 till January 

2005. 

• Excludes deaths from isolated areas with high violence rates in 

South Darfur after September 2004. 

• Adds additional 200,000 for Chad refugees; their deaths 

estimated as 10% of the total excess deaths estimated. 

• Assumes different monthly estimates 

separated by internally displaced 

persons (IDP) and affected residents 

based on information from the United

Nations Humanitarian Profiles. 

• Includes all regions in Darfur. 

• Applies different mortality rates to 

groups of affected residents, IDPs in 

camps, IDPs outside camps (did not do 

this in first estimate). 

Page 57 GAO-07-24 Darfur
Appendix IV: Summary Description of Death 

Estimates Reviewed 

Estimate by John Hagan and Others and Released by the Coalition for International Justice 

Reported figures • 396,563 total deaths. 

Time period covered • February 2003 through March 2005. 

Total number of months • 26 

Data sources • Uses Atrocities Documentation Team’s survey of Chad refugees to estimate violent 

deaths based on number of people who reported seeing a family member killed. 

• Uses 2004 WHO mortality survey findings for North and West Darfur to estimate 

nonviolent deaths. 

• Does not use any contextual or nonsurvey information. 

Mortality rates applied and

cause of death

• Nonviolent mortality rate is on a combination of findings from North and West Darfur 

WHO survey. 

• Violent mortality rate is calculated based on survey of Chad refugees and assumptions 

include that all missing persons have died and that the average family size is five. 

Mortality changes over time and/or

region

• Assumes the same mortality levels over time and per region. 

Baseline crude mortality rate assumed • 0 (Did not apply a baseline mortality rate because believed it was not appropriate.) 

Affected population included in

estimate

• Uses a single estimate of 1.5 million, as of April 2005, to include IDP population from 

all regions in Darfur and refugees in Chad. 1.5 million is the midpoint between 1.2 and 

1.8 million. (1.2 comes from State's Documenting Atrocities Report, and 1.8 was 

stated as widely cited as the affected population in early 2005.)

Page 58 GAO-07-24 Darfur
Appendix IV: Summary Description of Death 

Estimates Reviewed 

Estimate by Eric Reeves

Reported figures • Over 370,000 excess deaths.1 

Time period covered • February 2003 through August 2005. 

Number of months • 31 

Data sources • Uses varied sources such as the Atrocities Documentation Team’s survey of refugees 

in Chad, the 2005 WHO mortality survey, communication from David Nabarro, and 

projections from the Department of U.S. Agency for International Development. 

• Uses contextual or nonsurvey data in various calculations, such as mortality rates for 

inaccessible populations. 

Mortality rates applied and

cause of death

• Generally estimates nonviolent and violent deaths separately. 

• Survey of Chad refugees to estimate violent deaths based on number of people who 

reported seeing family member killed. 

• Estimates some mortality rates per day; estimates other rates per month or longer 

periods of time. 

Mortality changes over time and/or

region

• Assumes some changes in mortality over time. Generally does not assume different 

mortality rates per region except for populations in inaccessible areas. 

Baseline crude mortality rates • 0.3 based on UNICEF. 

Affected population included in

estimate

• Assumes affected population changes somewhat over time, although not on a monthly 

basis. Uses information from UN profiles, as well as other sources. For example, in

some cases, assumes some groups are excluded from UN profiles and increases 

numbers accordingly. 

• Includes affected population from all regions in Darfur and Chad refugees. 

 1

This figure is from an August 2005 estimate that was the latest available at the time the 

group of experts convened, and this was reviewed. At the end of April 2006, Eric Reeves 

provided a new estimate of over 450,000 people have died from violence, disease, and

malnutrition. 

Page 59 GAO-07-24 Darfur
Appendix IV: Summary Description of Death 

Estimates Reviewed 

Department of State Estimate 

Reported figures • 98,000 to 181,000 total deaths. 

• 63,000 to 146,000 excess deaths. 

Time period covered • March 2003 through January 2005. 

Total number of months • 23

Data sources • Uses data deemed sound from available mortality surveys in the Complex Emergency 

Database at that time for Darfur and refugees in Chad (refers to list of about 20 

surveys included in report). 

• Provides some information regarding which surveys and contextual data sources were 

used and why.

• Applies contextual information to fill in data gaps and determine if existing mortality 

rates should be applied to a broader population within that region or if a higher or lower 

rate was warranted. Refers to information such as displacement patterns, fire mapping 

remote sensing data, reporting of attacks, historical trends on humanitarian 

intervention, and disease outbreaks. 

Mortality rates applied and

cause of death

• Estimates high and low mortality rates that are assumed to include mortality due to all 

causes (nonviolent and violent). 

• Reports that some mortality rates were adjusted based on contextual information (e.g., 

report of attacks). For example, reports using West Darfur survey data as a basis for 

estimating mortality in North Darfur are based on nonsurvey data showing similar 

patterns of conflict in 2003 and early 2004, but estimates of higher total deaths in North 

Darfur are due to larger displaced population and more incidents of fighting. 

Mortality changes over time and/or

region

• Assumes differences in mortality over time and per region. 

• Divides conflict into four time periods: (1) the initial outbreak of violence (March 

through September 2003), (2) the breakdown of cease-fire/escalation of conflict 

(October 2003 through March 2004), (3) the second cease-fire (April through June 

2004) and (4) increased international humanitarian response (July 2004 through 

January 2005). 

• Reports that highest mortality and level of violence occurred during December 2003 to 

February 2004. Mortality began falling in March 2004. 

Baseline crude mortality rate assumed • Uses 0.5 deaths per 10,000 per based on estimates from the WHO of sub-Saharan 

African populations. 

Affected population included in

estimate

• Assumes different monthly estimates (separated by displaced and affected

populations) based on UN profiles for each Darfur state from September 2003 until 

January 2005. 

• Assumes an incremental increase prior to September 2003 because no source of 

reliable information was available. Uses data from the United Nations High

Commissioner for Refugees (UNHCR) on spontaneous settlements and camps to 

estimate Chad population. 

• Assumes affected residents experienced mortality rates that were 20% lower than 

displaced populations. 

Page 60 GAO-07-24 Darfur
Appendix IV: Summary Description of Death 

Estimates Reviewed 

WHO Estimate 

Reported figures • 45,000 to 80,000 total deaths. 

• 35,000 to 70,000 excess deaths. 

Time period covered • March 2004 through September 2004. 

Total number of months • 7 

Data sources • Primarily uses findings from 2004 WHO mortality survey, although looked at other 

surveys conducted. 

• Uses contextual information, such as the occurrence of disease outbreaks and the 

level of humanitarian assistance, to help refine estimates and define best and worst 

case scenarios. 

Mortality rates applied and

cause of death

• Estimates mortality rates include all causes of death (violent and nonviolent). 

Mortality changes over time and/or

region

• Estimates an average mortality rate for the entire region of Darfur. Does not assume 

different mortality rates over time or per region. 

Baseline crude mortality rate assumed • 0.5 

Affected population included in

estimate

• Assumes different monthly estimates of IDPs based on UN profile reports. 

• Estimate includes only IDPs in camps and excludes affected residents from all Darfur 

regions and refugees in Chad. 

Source: GAO analysis based on published documents of estimates and interviews with estimate authors. 

Note: The summaries of these estimates are based on the published information, as well as 

interviews with and additional information provided by estimate authors with the exception of State 

who declined to speak with us. Additionally, mortality rates discussed in table are crude mortality 

rates per 10,000 per day. 

Page 61 GAO-07-24 Darfur
Appendix V: Additional Follow-Up Survey Appendix V Results : Additional Follow-Up Survey 

Results 

This is a summary of 

responses to additional 

questions from our followup survey from the 12 

experts we convened with

the National Academy of 

Sciences in April 2006. 

Page 62 GAO-07-24 Darfur
Appendix V: Additional Follow-Up Survey 

Results 

Page 63 GAO-07-24 Darfur
Appendix VI: Comments from the Department Appendix VI: Comments from the of State 

Department of State 

Note: GAO comments 

supplementing those in 

the report text appear at 

the end of this appendix. 

Page 64 GAO-07-24 Darfur
Appendix VI: Comments from the Department 

of State 

See comment 1. 

See comment 2. 

Page 65 GAO-07-24 Darfur
Appendix VI: Comments from the Department 

of State 

Page 66 GAO-07-24 Darfur
Appendix VI: Comments from the Department 

of State 

Following are GAO’s comments on the Department of State’s letter dated 

October 4, 2006. 

1. State asserts that its death estimate was intended for internal 

purposes. We maintain that because State’s estimate was publicly 

available and discussed by State officials, sufficient detail on this and 

any future such estimates is necessary to safeguard State’s credibility 

as a source of accurate and reliable death estimates, particularly where 

such a serious topic is concerned. 

GAO Comments 

2. When asked, the CRED authors indicated that State provided them 

with the text on the methodology as included in the published report, 

but did not provide source information on the data used in State’s 

estimate.

Page 67 GAO-07-24 Darfur
Appendix VII: Comments from the U.S. Agency for International Development 

Page 68 GAO-07-24

Appendix VII: Comments from the U.S. 

Agency for International Development 

 Darfur
Appendix VIII: Summary of Authors’ Appendix VIII: Summary of Authors’ Comments 

Comments 

To the five other estimate authors, we provided portions of the report 

pertaining to their individual estimates. We summarize their comments 

and our responses below. 

• The authors of the Centre for Research on the Epidemiology of 

Disasters (CRED) estimates agreed with the experts’ evaluation. 

• Jan Coebergh did not provide any major comments regarding the 

relevant portion of the report and indicated that he was aware of the 

limitations of his estimate. 

• John Hagan expressed concern regarding the experts’ relatively 

positive evaluations of the estimates by CRED and the Department of 

State (State), stating that the respective estimates’ published 

documentation does not provide sufficient information on the data and

methodologies used. As discussed in the report, the experts noted 

limitations in the sufficiency of reporting for both these estimates, but 

they were able to discuss these estimates in detail and assess their 

respective levels of accuracy and methodological strengths and 

shortcomings. In addition to providing the experts the published 

reports containing the estimates, we provided them information 

regarding the source mortality surveys used in the estimates, as listed 

in appendix III. For the CRED estimate, we also provided the experts 

with additional details about the methodology based on our interviews 

with the authors. Dr. Hagan also believed that we should have 

included in our evaluation the estimate he did with Alberto Palloni 

published in the September 2006 issue of Science. We added 

information on the key findings of this estimate to the report, but as 

we note in the report, it does not include an analysis of the estimate 

because a final publicly available version was not available when the 

experts convened in April 2006. 

• Eric Reeves disagreed with the experts’ criticism of the Atrocities 

Documentation Team's survey of Chad refugees. He stated this survey 

was well conducted and a critical source of data to estimate violent 

deaths especially early in the conflict when little information was 

available. In the report, we note that the literature acknowledges 

difficulties in capturing deaths due to violence. Nevertheless, some of 

the estimates reviewed by the experts used other techniques to 

account for violent deaths, such as statistical analysis or contextual 

information. Moreover, many experts felt that Atrocities 

Documentation Team’s survey was not a reliable or appropriate source 

of data to estimate violent deaths for a cumulative death estimate on 

Page 69 GAO-07-24 Darfur
Appendix VIII: Summary of Authors’ 

Comments 

Darfur, based on public documentation on the survey, as well as 

supplemental information on the survey’s design, implementation, and 

sampling we obtained from representatives at the Coalition for 

International Justice. In addition, Dr. Reeves also expressed a concern 

regarding the 2004 survey by WHO and believed it did not sufficiently 

capture violent mortality in areas outside camps. To address this 

concern, we added information regarding the issue of accessibility as a 

potential source for underestimation of mortality and specifically 

discuss this survey. Dr. Reeves also stated that in a draft version of the 

report, we had incorrectly characterized his estimate’s use of the 2004 

WHO survey to account for only nonviolent deaths and, accordingly, 

we removed this characterization from the report. 

• Regarding the WHO estimate, David Nabarro emphasized in his 

comments that the estimate was undertaken to provide a rough order 

of magnitude of deaths in order to facilitate humanitarian relief efforts. 

He also pointed out that it applied to a shorter time period than the 

other estimates and was restricted to IDPs located in camps, without 

attempting to account for deaths in the larger population. We provided 

additional clarification in the report to emphasize the more restricted 

scope and coverage of the WHO estimate. Further, we stated that the 

purpose of the WHO estimate was to provide an order of magnitude 

estimate to assist in humanitarian relief planning. 

Page 70 GAO-07-24 Darfur
Appendix IX: 

A

GAO Contacts and Staff 

cknowledgments 

Page 71 GAO-07-24

Appendix IX: GAO Contacts and Staff 

Acknowledgments 

David Gootnick, (202) 512-3149, gootnickd@gao.gov

Nancy Kingsbury, (202) 512-2700, kingsburyn@gao.gov

In addition to the individuals named above, Emil Friberg, Assistant 

Director; Jim Ashley; Martin de Alteriis; Etana Finkler; Leslie Holen; 

Theresa Lo; Reid Lowe; Grace Lui; John F. Miller; and Chhandasi Pandya 

made key contributions to this report. 

 Darfur

GAO Contacts 

Staff 

Acknowledgments 

(320420) 
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER