DOT/FAA/AM-16/7

Office of Aerospace Medicine

Washington, DC 20591

Comparison of Pilot Fatalities 

and Number of Pilot Medical 

Examinations

Dennis V. Canfield

Kurt M. Dubowski

Guillermo J. Salazar

Estrella Forster

Civil Aerospace Medical Institute 

Federal Aviation Administration 

Oklahoma City, OK 73125

October 2016

Final Report
NOTICE

This document is disseminated under the sponsorship of 

the U.S. Department of Transportation in the interest of 

information exchange. The United States Government 

assumes no liability for the contents thereof.

___________

This publication and all Office of Aerospace Medicine 

technical reports are available in full-text from the 

Civil Aerospace Medical Institute’s publications website: 

http://www.faa.gov/go/oamtechreports
i

Technical Report Documentation Page

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.

DOT/FAA/AM-16/7

4. Title and Subtitle 5. Report Date

Comparison of Pilot Fatalities and Number of Pilot Medical 

Examinations

October 2016

6. Performing Organization Code

7. Author(s) 8. Performing Organization Report No.

Canfield DV, Dubowski KM, Salazar GJ, Forster E

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

FAA Civil Aerospace Medical Institute

P.O. Box 25082

Oklahoma City, OK 73125

11. Contract or Grant No.

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered

Office of Aerospace Medicine

Federal Aviation Administration

800 Independence Ave., S.W.

Washington, DC 20591 14. Sponsoring Agency Code

15. Supplemental Notes

This work was accomplished under the approved tasks AM-B-15-TOX-206.

16. Abstract

Introduction: A 2004 U.S. Government Accountability Office (GAO) report stated: “Accurate, 

comprehensive information on designee activities is an important prerequisite for designee oversight and is 

integral to monitoring and evaluating the programs.” In 2008, the GAO reported the Federal Aviation 

Administration (FAA) planned “to continue reviewing AME-issued certificates and collecting the results.” 

In 2012, the Aerospace Medical Research Division was tasked to conduct a study to determine the relative 

risk of a pilot fatality associated with Aviation Medical Examiners (AMEs). Method: Data for this study 

were extracted from the FAA toxicology database for all pilots that died between 2003 and 2012 in aviation 

accidents and AME pilot exams from the Aeromedical Certification System/Document Imaging Workflow 

System (AMCS/DIWS) database between 2000 and 2012 time frame. The method was compared to NTSB 

reports and a review of AMEs that were outside the limits of agreement for all AMEs in this study. Results: 

Eighty percent (5,872) of the 7,317 AMEs had zero fatalities, and 876 of the AMEs (12%) had only 1 

fatality in 10 years. One AME had 25 fatal pilots involved in aviation accidents during this 10 year study. 

The pilot fatality rate within the limits of agreement (LOA) was determined to be 1 fatality per 2,000 exams 

+/- 810. Approximately 254 AMEs (3.5%) had high fatality rates per exam and 120 (1.6%) had low fatality 

rates relative to the AMEs within the limits of agreement (6,942; 94.9%). Discussion: Most aviation 

accidents examined in this study were ruled pilot error by the NTSB but no reason was given for the pilot 

error. This research identified a small number of AMEs (254, 3.5%) with an atypically high pilot fatality rate 

per exam. Although this is a small number of AMEs the group accounted for 1,077 fatalities (37.7%) with 

690 of these fatalities exceeding the 387 projected fatalities for this group over the 10 year study. The 

majority of AMEs (96.5%) had fatality rates within the limits of agreement or above demonstrating the 

success of the Aerospace Medical Certification and Aerospace Medical Education divisions.

17. Key Words 18. Distribution Statement

Medications, drugs, aviation, safety, fatal, pilots Document is available to the public through 

the Internet:

http://www.faa.gov/go/oamtechreports/

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price

Unclassified Unclassified 8

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized
ii

ACKNOWLEDGMENTS

We acknowledge the Federal Aviation Administration Office of Aerospace Medicine and the Office of 

Aviation Accident Investigation & Prevention as the main sponsoring organizations for this research. We 

would also like to thank William Hathaway of the FAA Protection and Survival Laboratory (AAM-630) for 

providing the data used for the number of exams completed for each AME in this study group.
iii

Contents

COMPARISON OF PILOT FATALITIES AND

NUMBER OF PILOT MEDICAL EXAMINATIONS

INTRODUCTION............................................................................................................................................................ 1

MATERIALS AND METHODS ................................................................................................................................... 1

RESULTS............................................................................................................................................................................. 2

AME Fatal Pilot Distribution 2003-2012................................................................................................................... 2

AME Exam and Fatality Difference Graph .............................................................................................................. 3

Validation of the Method ............................................................................................................................................. 3

DISCUSSION..................................................................................................................................................................... 4

REFERENCES................................................................................................................................................................... 4
1

COMPARISON OF PILOT FATALITIES AND

NUMBER OF PILOT MEDICAL EXAMINATIONS

INTRODUCTION

In 2003, an investigation was conducted in the state 

of California by the Inspector’s General Office of the 

Department of Transportation and the Social Security 

Administration called “Operation Safe Pilot” to identify 

pilots that were fraudulently collecting Social Security 

benefits (1). This investigation led to criminal charges 

being filed against 45 airmen for falsification of medical 

applications (2). In addition, the U.S. House of Representatives Committee on Transportation and Infrastructure held hearings in 2007 and met with representatives 

from the Federal Aviation Administration (FAA) to 

consider the “FAA Oversight of Falsifications on Airman Medical Certificate Applications” and concluded 

that “Pilots that are physically or mentally unfit not only 

pose a danger to themselves and the flying public, they 

also jeopardize the lives and safety of anyone in their 

flight path” (3). The FAA has established stringent criteria to determine whether airmen are medically fit to fly.

The Congressional Committee and the FAA agreed to 

“Enhance our quality control and AME oversight processes” (4). A U.S. Government Accountability Office 

(GAO) report issued in September of 2008 stated the 

“FAA has established two quality assurance review programs—one evaluating certificates that the Aviation 

Medical Examiners (AMEs) issued and the other evaluating certificate decisions made by FAA application examiners” (5).

The Aerospace Medical Research Division of the 

FAA is tasked with conducting aerospace medical research studies to determine ways to improve medical 

certification and the safety of the public. In an effort to 

continue the improvement of the medical certification 

process, the FAA Aeromedical Technical Community 

Representative Group (TCRG) requested that the Aerospace Medical Research Division conduct a study to evaluate AMEs’ relative risk for a pilot fatality based on information stored in the agency’s Toxicology Database 

(ToxDB) for fatal pilots involved in an aviation accident. 

The FAA Office of Aerospace Medicine develops 

airman medical standards to protect the public’s interests and prevent death and injury of crewmembers, passengers, and general public on the ground. Pilots are 

required by FAA regulations to obtain a medical certificate after successfully completing a medical examination 

by a certified AME. In the event of an aircraft accident, 

an integral part of the FAA’s comprehensive safety mission includes conducting a forensic toxicology analysis 

of every pilot fatality where specimens are available. Following a fatal aviation accident, specimens from deceased pilots are collected by local pathologists and sent 

to the FAA Civil Aerospace Medical Institute’s 

(CAMI’s) Bioaeronautical Sciences Research Laboratory 

for toxicological analysis. The analysis determines the 

drugs present in postmortem specimens and the pathologies associated with the drugs of fatally injured pilots 

that may have been the probable cause or a factor in the 

aircraft mishap. 

The toxicological results are utilized for individual 

aircraft accident investigation to determine the probable 

cause(s) of a particular accident and to gather data for 

research to prevent accidents. The toxicological results 

for each accident also provide essential information for 

medical certification specialists in terms of ensuring that 

the pilot was in compliance with and had been appropriately medically certified to fly. In addition to the importance for individual aircraft accident analysis, the 

toxicological results are continuously reviewed to understand unusual trends associated with individual AME 

activities that might become evident, so that proactive 

and preventive measures can be initiated to ensure continuous safety improvements in aeromedical certification.

Trend analysis may provide evidence to support 

proactive changes in airman medical certification activities as well as proactive AME education programs that 

improve the aerospace medical certification process. All 

of these activities are a part of the FAA Administrator’s 

Priority Initiatives: “Risk-Based Decision Making: Build 

on safety management principles to proactively address 

emerging safety risk by using consistent, data-informed 

approaches to make smarter, system-level, risk-based 

decisions” (6). Earlier studies have suggested that medical conditions and medicines being taken were not being 

reported by the pilot nor discovered as a part of the pilot medical exam (7). A review of the literature did not 

identify research on the topic of determining AME relative risk of a pilot fatality. This 10-year study intends to 

test the null hypothesis that “Fatalities are directly proportional to the number of exams performed by an 

AME.” This study intends to determine the relative risk 

(RR) of a pilot dying in a fatal aviation accident relative 

to the AMEs performing the exams.

MATERIALS AND METHODS

All toxicology case results from civil aircraft accident fatalities are stored electronically in a database 

maintained at CAMI called the ToxDB. This database 

was searched by AME number to identify the number 

of pilot fatalities for each AME during the period from 

2003 to 2012. Information gathered from the ToxDB 
2

included class of airman certificate, class of medical certificate, age, accident location, and the type of flight certification of the associated accidents. Only pilots with a 

valid medical certificate in accordance with Title 14 of 

the Code of Federal Regulations (14 CFR), Chapter 1, 

Part 67 (8) were included in this study.

A search was also conducted of the CAMI Document Imaging Workflow System (DIWS) to identify the 

number of pilot exams performed by all practicing 

AMEs during the period 2000 to 2012. Some of the pilots in the study would have received a medical exam prior to the start of the study, and it was decided to include 

three additional years of exams to account for this possibility and because it would not impact the overall study.

The number of fatalities and exams performed were 

converted into a percentage of the maximum value 

found for fatalities and exams. This allowed a direct 

comparison of the number of fatalities and exams using 

a modified Bland-Altman difference graph (9) as follows: the percentage of fatal pilots was calculated by 

dividing the number of fatalities for each AME by the 

maximum number of fatalities for an AME. The percentage of exams was calculated by taking the number 

of exams per AME and dividing by the maximum number of exams performed by an AME. Using this method, the possible values for fatalities and exams can range 

from 1 to 100.

Statistical data reported were calculated with SigmaPlot 12.0 (Systat Software Inc., San Jose, CA) using 

the Bland-Altman method for comparing values that 

should be the same within statistical limits.

The relative risk (RR) was calculated using the 

number of fatal pilots found for each AME divided by 

the expected number of accidents for the AME using 

the calculated fatality rate per exam for the AMEs within the limits of agreement (LOA).

RESULTS

AME Fatal Pilot Distribution 2003-2012

During the course of this study there were 7,316 

practicing AMEs. From 2003 to 2012, there was an 

average of 4,003 practicing AMEs per year. The number 

of practicing AMEs declined from 4,668 in 2003 to 

3,390 in 2012. Of those AMEs, 1,444 (20%) had 1 or 

more medically certified pilots die while piloting an aircraft. Eighty percent (5,872) of the AMEs in this study 

had zero accidents over a 10 year period. Out of 1,444 

AMEs with an aviation pilot fatality in this study, the 

majority (876, 61%) had only 1 fatality over the 10 year 

period.

Three groups of AMEs were identified in this study: 

(a) AMEs within the LOA of the proposed hypothesis, 

(b) AMEs below the LOA (high fatality rate), and (c) 

AMEs above the LOA (low fatality rate). The calculated 

mean bias was -0.1104 with a standard deviation (SD) of 

3.4525. Thus, the LOA (Mean ± 1.96 SD) was -0.1104 

± 6.7669 resulting in the LOA range of -6.8773 to 

6.6565.

The AMEs within LOA range had approximately 1 

fatality per 2,000 exams ± 811. Only 254 AMEs (3.5%) 

were below the LOA for AMEs (High Fatality Rate).

Figure 1. Distribution of fatally injured pilots by number of AMEs
3

There were 120 AMEs (1.6%) above the LOA (Low 

Fatality Rate). The majority of the AMEs (6,942, 95%) 

were within the LOA. The distribution of AMEs relative 

to fatality rates can be found in Table 1.

Table 1. Distribution of AMEs

AMEs Totals %

With Fatals 1444 19.7

Without Fatals 5872 80.3

Below LOA 254 3.5

Within LOA 6942 94.9

Above LOA 120 1.6

Total 7316 100.0

The total number of fatalities for the 2003–2012 period was 2,858. The small number of AMEs (3.5%) below the LOA had approximately three times as many 

fatalities as the AMEs within LOA (95%) and accounted 

for 37.7% of all fatalities that occurred. The AME 

group below the LOA had 7 times the fatality rate as the 

AME group above the LOA. The Relative Risk (RR) 

and percentage of pilot fatalities for the three groups of 

AMEs in this study are presented in Table 2.

Table 2. Distribution of Fatalities and Relative Risk (RR)

Fatals Totals %:RR

Below LOA:% 1077 37.7

Estimated:RR 387 2.8

Above LOA:% 233 8.2

Estimated:RR 623 0.4

Within LOA:% 1548 54.2

Estimated:RR 1628 1.0

Total:% 2858 100.0

AME Exam and Fatality Difference Graph

A modified Bland-Altman difference graph was created using the percentage of fatalities and exams to determine if there was a proportional relationship between 

the number of exams and the number of fatalities (Figure 2). The 95% confidence limit (CL) for the Lower 

LOA of -7.0144 to -6.7403 with a 95% CL and a 95% 

CL for the Upper LOA of 6.5194 to 6.7935. Fatality 

rates that were outside the LOA were considered atypical.

Validation of the Method

National Transportation Safety Board (NTSB) 

dockets and DIWS medical records were examined for 

both high fatality rate and low fatality rate AMEs to determine the presence of medical issues that may have 

been missed during the pilot medical exam. For example, there were no medical causes of aviation accidents 

reported by the NTSB for the low fatality rate AME 

that had 12 fatalities for 48,120 exams; whereas, a high 

fatality rate AME had three fatal pilots out of 17 fatalities (5,595 pilot exams) with unreported medical conditions determined to be the probable cause or a factor in 

the accident.

The age, gender, class of medical certificate, and region were approximately the same for both the high and 

low fatality rate AMEs. The only difference noted was 

found in the class of airman certificate where there were 

an increased number of commercial pilots in the high 

fatality rate AME group (n = 7 of 17 total, 41%) compared to the low fatality rate AME group (4 of 12, 33%).

All other measurable conditions were found to be comparable for both the high pilot fatality rate AMEs and 

Figure 2. Difference graph for all AMEs in the study showing percent difference from mean for all AMEs
4

low pilot fatality rate AMEs which is consistent with 

earlier findings: “the distribution of medical certificate 

class and airman certificate type were very similar when 

the total fatalities group was compared to the group of 

fatalities found positive for drugs” (5).

DISCUSSION

In most aviation accidents, the NTSB found pilot 

error as the probable cause; however, it does not preclude the possibility that a medical condition, medications, or other substances being taken by the pilot interfered with pilot’s ability to control the aircraft. It is very 

rare for the NTSB to rule the probable cause of an accident as a medical event because of the difficulties in 

finding sufficient evidence such event impaired the pilot 

sufficiently to have caused the accident (10). Something 

that can be measured with some degree of certainty is 

the potential risk associated with the substances and 

medical issues identified through reported and unreported information to the FAA. However, further study 

is needed to determine what role, if any, the medical 

certification process played. The procedure used in this 

study was able to identify a small group of AMEs with 

atypical high fatality rate per exam; however, the significance is yet to be determined.

The null hypothesis for this study was rejected. This 

is, fatalities are not directly proportional to the number 

of exams performed by an AME. For 95% of the AMEs 

in this study, the original hypothesis was found to be 

true and only a small percentage was found to be atypical. However, this small atypical group of AMEs (3.5%) 

accounted for 1,077 pilot fatalities with 690 of these 

fatal pilots being above the number of fatalities expected 

for the number of exams performed. The 690 estimated 

fatal pilots only takes into consideration the fatal pilots 

and does not include passengers or ground fatalities related to these accidents that would make the loss of life 

even greater. Determining the reasons for the additional 

690 pilot fatalities above the expected number of fatalities could help reduce the number of fatalities caused by 

aviation accidents. Identifying AMEs below the limits of 

agreement resulting from the sample used for his study 

will make it possible to focus on a small group of AMEs 

that might need additional training and support to improve the overall FAA medical certification process.

Additionally, it could help identify a segment of the airman population requiring better outreach programs on 

the need to fully disclose medications and medical conditions, particularly if they develop in the interim period 

between airman medical examinations.

This study strongly suggests that Aerospace Medical 

Certification and Aerospace Medical Education have the 

potential to reduce some fatalities resulting from aviation accidents through improvements in the medical 

certification process and AME/Airman training. It 

seems obvious that this would be the case considering 

the dangers associated with sudden medical incapacitation or the reduced capabilities of a pilot that could have 

been avoided by early detection and remediation of the 

medical condition. This is consistent with the NTSB 

findings that concluded the “FAA medical certification 

requirements and DOT mandatory drug and alcohol 

testing requirements for safety-sensitive aviation personnel have been associated with fewer toxicological 

findings of impairing drugs and conditions among accident pilots subject to those requirements. Conversely, 

these results suggest that allowing pilots to fly without a 

medical certificate could contribute to an increased risk 

of pilot impairment while flying because study pilots 

without an FAA medical certificate were more likely to 

have toxicological evidence of impairing drugs and conditions” (11).

The relative danger associated with aviation accidents is often equated to surface accidents where a driver’s license is sufficient to operate a motor vehicle.

However, it is important to recognize major differences 

between surface and aviation accidents. Surface vehicles 

are relatively confined to predefined spaces, whereas in 

aviation there are no physical boundaries to prevent 

aircraft from traveling over hospitals, schools, and 

homes. Additionally, surface vehicle drivers do not have 

the physical stresses of flight, spatial illusions, the operational multi-tasking, the diverse piloting skill set, and the 

decidedly higher speeds and shorter reaction times. This 

results in additional risk to the public on the ground and 

increases the potential of a catastrophic event not typically seen with surface vehicle accidents. The U.S. 

House of Representatives Committee on Transportation 

and Infrastructure found that “Pilots that are physically 

or mentally unfit not only pose a danger to themselves 

and the flying public, they also jeopardize the lives and 

safety of anyone in their flight path” (3).

The fact that the majority of the AMEs (96%) are 

within the limits of agreement or above demonstrates 

that the aerospace medical certification processes and 

AME selection and training process seem to be effective 

in reducing fatalities by preparing AMEs to identify high 

risk medical conditions that can result in the sudden 

incapacitation or reduced performance of the pilot.

REFERENCES

1. Office of Inspector General. (2005) Falsification of 

FAA Airman Medical Certificate Applications by 

Disability Recipients. US Department of Transportation, Publication No. July 22, 2005, Washington, 

DC.
5

2. Tilton, F. (2007) Operation Safe Pilot. Federal Air 

surgeon's Medical Bulletin 45:2, 2.

3. Committee on Transportation and Infrastructure, 

FAA Oversight of Falsifications on Airman Medical 

Certificate Applications, (2007), US House of Representives, Date 03/27/2007

4. Tilton, F. (2007) Operation Safe Pilot Revisited. 

Federal Air surgeon's Medical Bulletin 45:3, 2.

5. US Government Accountability Office. (2008) 

AVIATION SAFETY - FAA Has Taken Steps to 

Determine That It Has Made Correct Medical Certification Decisions. GAO, Publication No. GAO08-997, Washington, DC.

6. FAA Strategic Initiatives - Make aviation safer and 

smarter, Available at 

http://www.faa.gov/about/plans_reports/media/F

AA_Strategic_Initiatives_Summary.pdf, (2015).

7. Canfield, D.V., Salazar, G.J., Lewis, R.J., et al. 

(2006) Pilot medical history and medications found 

in post mortem specimens from aviation accidents. 

Aviat Space Environ Med 77:1171-1173.

8. Title 14 of the Code of Federal Regulations (14 

CFR). Chapter 1, Part 67, U.S. Government Publishing Office, 732 North Capitol Street, NW, 

Washington, DC 20401-0001 

9. Bland, J.M. and Altman, D.G. (1986) Statistical 

methods for assessing agreement between two 

methods of clinical measurement. Lancet 1:307-310.

10. Cullen, S.A., Drysdale, H.C., and Mayes, R.W. 

(1997) Role of medical factors in 1000 fatal aviation 

accidents: case note study. BMJ 314:1592.

11. NTSB. (2014) Drug Use Trends in Aviation: Assessing the Risk of Pilot Impairment. Publication 

No. NTSB/SS-14/01, Washington, DC.