Using R to Visualise Data on Aviation Tragedies in the US since 1948 | by Hamza Bendemra, Ph.D. | Towards Data Science

In this post, I look at a dataset sourced from the NTSB Aviation Accident Database which contains information about civil aviation accidents. A dataset is available on Kaggle also.
This Exploratory Data Analysis (EDA) aims to perform an initial exploration of the data and get an initial look at relationships between the various variables present in the dataset.
My aim is to also show how a simple understanding of Data Analysis and Wrangling in R coupled with domain knowledge can provide a better understanding of relationships between variables in a dataset. The R Markdown file can found in this GitHub repo.
First, a quick intro of the dataset I’ll be exploring. The dataset features 81,013 observations of 31 variables which are related to aviation accidents recorded.
## [1] 81013 31
Variables provide information on a variety of topics including date and location of observations, model and type of aircraft, information on the sustained injuries to passengers and to the aircraft, and the reported weather conditions at the time.
## [1] "Event.Id" "Investigation.Type" ## [3] "Accident.Number" "Event.Date" ## [5] "Location" "Country" ## [7] "Latitude" "Longitude" ## [9] "Airport.Code" "Airport.Name" ## [11] "Injury.Severity" "Aircraft.Damage" ## [13] "Aircraft.Category" "Registration.Number" ## [15] "Make" "Model" ## [17] "Amateur.Built" "Number.of.Engines" ## [19] "Engine.Type" "FAR.Description" ## [21] "Schedule" "Purpose.of.Flight" ## [23] "Air.Carrier" "Total.Fatal.Injuries" ## [25] "Total.Serious.Injuries" "Total.Minor.Injuries" ## [27] "Total.Uninjured" "Weather.Condition" ## [29] "Broad.Phase.of.Flight" "Report.Status" ## [31] "Publication.Date"
Since this an NTSB database from the United States, the majority of accidents (over 94%) in this databased are observations in the US. Hence, I will be focusing on the accidents that took place in the US in this analysis. After removing international observations the new dataframe now features 76,188 observations.
## [1] 76188 31
Some data wrangling was necessary of course (details of which can be found in the R Markdown file in the GitHub repo). For instance, the listed location names (city, state) was separated into two variables: one for the city and one for the state, for each observation.
The variable related to the observation’s event date was broken down into the observation’s event date by day, month, and year to investigate if there are any correlations between number of accidents and particular periods within a year. More data wrangling was necessary to
Also, a better way of displaying data related to total fatalities for a given observation was to group number of fatalities in buckets (or bands). This will give us a better representation of the distribution of fatalities across the observations in the dataset.
In this section, I will create univariate plots for variables of interest.
Let’s plots frequency histograms for the year, month, and weekday of accidents in the dataset. The majority of the observations in the dataframe are from after the early 1980s onwards. So let’s generate a plot from 1980 to 2017.
The number of accidents has overall decreased by approx. 47% between 1982 and 2017 from approx. 3400 observations to approx. 1600 observations.
Next, let’s look at observations distribution by months of the year.
The highest number of accidents in the dataset for a given year take place during northern hemisphere summer time (Jun-Jul-Aug). This is also likely to be correlated with the increased numbers of flights during the summer holiday period.
And finally, let’s look at the observations distribution by day of the week.
The highest frequency of accidents in a given week take place during the weekend (Sat-Sun). Again ,this is also likely to be correlated with the increased numbers of flights during the summer holiday period.
The next variable of interest relates to the Total Fatal Injuries for each observation in the dataset. This is quantified by the number of people fatally injured for each recorded observation. Let’s group the number of fatalities in buckets as shown in the plot below. Note the use of the Log10 scale for the y-axis in the plot below.
The bulk of the recorded accidents have a number of fatalities <10 while some observations are displaying numbers of fatalities >100.
Next, I look at the aircraft engine types recorded in the dataset. I’ve abbreviate engine type names to improve labelling of the x-axis. Note the use of the Log10 scale for the y-axis in the plot below.
According to the plots above, the bulk of engine types in the reported accidents are Reciprocating engine types which were prevalent in commercial aircraft, particularly in aircraft built during the 20th century. Recent aircraft, like the Airbus A380 or the Boeing 787 Dreamliner rely on Jet engines (e.g. TurboFan, TurboProp).
Next, I look at the weather conditions recorded in the dataset. Two key aviation weather conditions to be familiar with here: VMC which means that conditions are such that pilots have sufficient visibility to fly the aircraft maintaining visual separation from terrain and other aircraft. IMC which means weather conditions require pilots to fly primarily by reference to instruments.
The bulk of accidents in the dataset take place during VMC weather conditions, which are great conditions for flying as VMC requires greater visibility and cloud clearance than IMC.
I imagine this would go against most people’s intuition when it comes to the relationship between weather conditions and aviation accidents. Pilots are indeed well trained to fly in all sorts of weather conditions, relying solely on the avionics instruments at their disposal.
Next, let’s look at the phases of flight for the recorded accidents in the dataset. According to the plot, the bulk of accidents took place during landing or take-off. It is well known in the industry that these are high-risk — often referred to as “critical phases of flight”.
Let’s look at the relationship between pairs of variables that could show interesting relationship.
The bulk of the distribution has a total fatal injuries under 10, let’s zoom in on that portion of the data. The R function geom_jitter was used to amplify the visualisation of the data points.
According to the plot, the bulk of the data for fatalities under 10 is with the engine type reciprocating engine type. The first plot shows that the Turbo-Fan engine has more outliers with higher number of fatalities than other engines. This is likely correlated to the use of Turbo-Fan engines use on large commercial aircraft.
As previously noted ,weather conditions do not show a particularly strong relationship with total fatal injuries. The bulk of the distribution is associate with VMC weather conditions. However, that is likely to the fact that the vast majority of flights are flown in VMC conditions.
Let’s look at the relationship of Phase of Flight and Total Fatal Injuries
The plots show that Take-Off and Approach are associated with outliers with high number of fatalities. As previously noted, these two phases of flight are often referred to as “critical phases of flight” for that particular reason.
Let’s look at the relationship of Event Date and Total Fatal Injuries. We’ll focus on the bulk of the distribution with fatalities of less than 10. There doesn’t seem to be any specific month or weekday showing a particularly high frequency of accidents.
The plots indicates that there is higher frequency of recorded observations for certain combinations of weather and phases of flight — for example, IMC flying conditions while during “cruise” or “approach” phases of flight.
The plots indicates that there is a higher frequency of recorded observations for Northern summer months during Landing and Take-off. Across all months, the heat map also shows that the Take-off and Landing register the highest number of observations.
Plotting the Latitude vs Longitude of the accidents essentially gives us the map of the US. The plots also indicates that the coastal states are more heavily impacted compared to mid-western states and most of Alaska. This can be explained by the volume of flight to/from destinations those areas of the US. A sad chart however as it shows that the vast majority of US States suffered an aviation tragedy between 1948 and 2017.
Now let’s look at multivariate plots.
The distribution of accident months across latitude and longitude is fairly spread across the US, with a slightly higher prevalence of observations during the winter in southern states like Florida.
Let’s now look at latitude vs longitude add layer for the weather condition.
Weather condition VMC seems to be quite consistent except for patches of primarily IMC conditions for certain discrete areas of the mid-west.
Let’s now look at the relationship of Broad Phase of Flight vs Month by Weather Conditions. I decided to leave the observations with “unkown” weather conditions as there is what I perceive to be non-negligible number of observations, particularly for the “Cruise”phase of flight.
When looking at the relationship between Broad Phase of Flight vs Month and Weather Condition, we can see that accident primarily take place during VMC weather condition. However, for certain months of the year such as December an January, IMC conditions are a non-negligible portion of the observations, particularly during Approach and Cruise phases of flight.
Next, let’s look at Total Fatal Injuries vs Broad Phases of Flight by Weather Condition. I decided to focus on total fatal injuries of less than 40 to highlight the bulk of the distribution of recorded fatal injuries.
IMC weather conditions are associated with accidents during “Cruise” and “Approach” phases of flight with higher frequencies than in VMC weather conditions.
This plot is interesting as it shows how certain engines have become prevalent in different time periods. For instance, Turbo-Jet and Turbo-Fan powered aircraft show a higher number of fatalities in later years whereas reciprocating engines show a distribution of fatalities in earlier years, this corresponds to the increasingly use of jet engines in modern aircraft.
I hope you enjoyed this EDA and learned a few things about aviation along the way. I hope that this post also shows the power of simple data visualisations with ggplot2 in R. This skills is particularly useful when exploring a dataset in the initial phase of the data science pipeline. I certainly enjoyed this fun little study done over the weekend.