An Analysis of the Fatal Police Shootings in the US | by Soner Yıldırım | Towards Data Science

The Washington Post has released a dataset containing fatal shootings by police in the US between 2015 and 2020. In the midst of racism discussions, this dataset is likely to shed light on the current situation.
We will use data analysis tools and techniques to reveal some numbers that summarize the dataset. This post is aimed to be a practical guide of data analysis and a discussion that approaches police shootings in the US from many perspectives.
The dataset is available in this repo by the Washington Post. I will be using python data analysis and visualization libraries. Let’s start with importing these libraries.
# Data analysisimport numpy as npimport pandas as pd# Data visualizationimport matplotlib.pyplot as pltimport seaborn as snssns.set(style='darkgrid')%matplotlib inline
We can now read the dataset into a pandas dataframe.
df = pd.read_csv("/content/fatal_police_shootings.csv")print("Dataset has {}".format(df.shape[0]),"rows and {}".format(df.shape[1]), "columns")
Each row represents a shooting and the columns give information about the details of shootins. Let’s see what kind of data we have in these 14 columns.
The names are too personal to use and might be against individual rights. The id column is redundant. Thus, I will drop these columns.
df.drop(['id','name'], axis=1, inplace=True)
Remaining 12 columns are:
We have data about the person being shot, location of shooting, and the shooting action itself. The next step is to check missing values and handle them.
df.isna().sum()
“Armed”, “age”, “gender”, “race”, and “flee” columns have missing values. A more informative tool on missing values is the missing value matrix provided by missingno library. It gives an idea about the distribution of missing values in the dataframe.
import missingno as msnomsno.matrix(df)
It seems like missing values on “gender” and “race” columns overlap to some extent (i.e. they are likely to be in the same rows). We can check the heatmap of missing values to confirm:
msno.heatmap(df, figsize=(10,6))
The missing values in “race” and “age” columns are correlated.
The “flee” and “armed” columns describe the action of the person being shot.
df.flee.value_counts()
The action of “not fleeing” dominates “flee” column. We can fill in the missing values with “not fleeing”. Please note that this is not a strict rule for handling missing values. You can choose a different way to handle them (e.g. drop them).
df.flee.fillna('Not fleeing', inplace=True)
Let’s check the “armed” column.
df.armed.value_counts()
“Gun” is the most frequent value so I will use that to fill in missing values. We can use the index of the returned series from value_counts function:
df.armed.fillna(df.armed.value_counts().index[0], inplace=True)
I will drop the rows that have missing values in “race”, “age”, and “gender” columns because they describe the person being shot and thus it could be misleading to make assumption without accurate information.
df.dropna(axis=0, how='any', inplace=True)print("There are {}".format(df.isna().sum().sum()), "missing values left in the dataframe")There are 0 missing values left in the dataframe
The dataframe does not have any missing values left.
Data types are important in the process of data analysis as they determine how certain actions and computations are handled.
The categorical variables can be represented with “object” or “category” data type. The age can be represented with “integer” or “float” data types and true/false type of data is handled with “bool”. All of the data types seem appropriate except for date column.
I will convert it to datetime which is the data type of pandas to handle dates. After converting, I will extract “year” and “month” from the date and create new columns. We can use them to see yearly or monthly shooting rates.
df['date'] = pd.to_datetime(df['date'])df['year'] = pd.to_datetime(df['date']).dt.yeardf['month'] = pd.to_datetime(df['date']).dt.month
Let’s see if there is any continuous upward or downward trend in the daily number of shootings from 2015 to 2020.
One way is to group dates and the number of shooting at each day.
df_date = df[['date','armed']].groupby('date').count().sort_values(by='date')df_date.rename(columns={'armed':'count'}, inplace=True)df_date.head()
“Armed” column is chosen randomly just to count the number of rows for each day. We can now create a time series plot.
plt.figure(figsize=(12,6))plt.title('Daily Fatal Shootings', fontsize=15)sns.lineplot(x=df_date.index, y='count', data=df_date)
This does not tell much. It will look better if we plot 10-day averages.
df_date.resample('10D').mean().plot(figsize=(12,6))plt.title('Fatal Shootings - 10 day average', fontsize=15)
We can observe some peaks but there is not a continuous trend.
Let’s check how numbers are changing in different states. I will use sidetable which is pandas utility library. It is like a superior version of value_counts.
!pip install sidetableimport sidetabledf.stb.freq(['state'], thresh=50)
684 fatal shootings have occurred in CA which consist of approximately 14% of all shootings. The top 3 states in terms of the total number of shootings are CA, TX, and FL. This is not strange when the populations of states are considered.
I think the “age” is an important aspect to be considered. Preventive actions can be designed according to different age groups.
plt.figure(figsize=(12,8))plt.title('Age Distribution of Deaths', fontsize=15)sns.distplot(df.age)
Most of the people who were shot are younger than 40. Every life equally matters but it makes it harder for the family when a young person dies.
Racism is the most severe disease of the history of humankind. It is more dangerous than coronavirus or any other pandemis that people have been struggled with. Unfortunately, there is a difference in the number of fatal shootings with respect to different races.
We will first create a new dataframe that contains the number of yearly shootings for each race. The dataset consists of 6 different races which are:
df_race = df[['race','year','armed']].groupby(['race','year']).count().reset_index()df_race.rename(columns={'armed':'number_of_deaths'}, inplace=True)df_race.head()
Only the number of deaths do not tell us much because these races are not proportionately represented in terms of population. Thus, I will use the number of deaths per 1 million people as baseline.
I will use the populations in 2019 which is available on US Census website. Although the ratios have changed from 2015 to 2020, it is not a dramatic change like 10–15 percent. I think the ratios remain within a margin of a few percents. However, you can use exact populations in each year to be more accurate.
df_pop = pd.DataFrame({'race':['W','B','A','H','N','O'],'population':[0.601, 0.134, 0.059, 0.185, 0.013, 0.008]})df_pop['population'] = df_pop['population']*328df_pop
Population column represents the population of each race in millions. We can now merge df_race and df_pop dataframes.
df_race = pd.merge(df_race, df_pop, on='race')df_race['deaths_per_million'] = df_race['number_of_deaths'] / df_race['population']df_race.head()
We can create a barplot that shows deaths_per_million of each race by police shootings from 2015 to 2020.
plt.figure(figsize=(12,8))plt.title("Fatal Shootings by Police", fontsize=15)sns.barplot(x='year', y='deaths_per_million', hue='race', data=df_race )
The ratio for black people (B) is clearly higher than other races. The native (N) and other (O) have very low population so a more logical comparison would be among black (B), white (W), hispanic (H), and asian (A) races.
The overall ratio of deaths_per_million:
If ratio of deaths_per_million for black (B) people is double the ratio of hispanic (H) people. The difference between black (B) and white (W) people is even more.
Racism is something we should not even be discussing. It should not exist. When a child hear the word “racism”, the response should be “what does it mean?”. Any type of racism in any where in the world needs to disappear. Unfortunately, it is not the case in the world now. But, we can educate our children in a way that the word “racism” stops existing.
Thank you for reading. Please let me know if you have feedback.