Analyzing London Crimes Data in R | by Hamza Rafiq | Towards Data Science

I was playing around with this dataset on crimes in London. The analysis and visualizations went a lot better than I expected so I thought I might as well write an article about it. I got this data while I was experimenting with bigquery but you can also download it from here on Kaggle. Let's begin by loading the libraries and our data.
library(tidyverse) ## For data wrangling and visualization library(lubridate) ## To work with dateslibrary(ggpubr)    ## Extra visualizations and themeslibrary(patchwork) ## Patch visualizations togetherlibrary(hrbrthemes)## extra themes and formattinglibrary(scales)    ## For formatting numeric variableslibrary(tidytext)  ## Reordering within facets in ggplot2library(pier)      ## Make interactive piecharts in Rlibrary(ggalt)     ## Extra visualizationslondon_crimes <- read_csv("C:\\Users\\ACER\\Downloads\\bq-results-20190830-212006-dh1ldd1zjzfc.csv",trim_ws = TRUE) %>%   mutate(Date_Column = dmy(Date_Column))
After loading the libraries, I used the read_csv function and gave it a path to where the file is on my computer, set trim_ws to TRUE in case there is any whitespace and remove it followed by the mutate function to change the Date_Column to date type using the dmy function from the Lubridate package.
To get an overview, I am going to visualize how the overall crime incidents have changed over the years. Let's walk through the code:
london_crimes %>%   group_by(Year = floor_date(Date_Column,unit = "year")) %>%     summarise(Incidents=sum(total_incidents,na.rm = TRUE)) %>%     ungroup() %>%                                                 mutate(pct_change= (Incidents-lag(Incidents))/lag(Incidents),         pct_change=replace_na(pct_change,0)) %>%     ggplot(aes(Year,Incidents))+                 geom_bar(stat="identity",fill="firebrick",color="black")+          geom_line(color="steelblue",size=1.5,linetype="dashed")+       geom_text(aes(label=percent(pct_change)),vjust=-1,color="black",face="bold")+  geom_text(aes(label=comma(Incidents)),vjust=1,fontface="bold",color="white")+   scale_y_comma(expand = c(0,0),limits = c(0,800000))+           scale_x_date(breaks = "year",date_labels ="%Y")+               theme_classic()+                                               labs(title = "Total Incidents over the years")
To get a yearly analysis, we group by the date column and round it on a yearly level using the floor_date function and then get a sum of total incidents against each year using the summarize function
It is always good practice to ungroup after every group_by operation to remove the grouping context
We then use the mutate function to create a new column — pct_change which is basically calculating the percentage change between each year using the lag function
After getting the data in order, we make a standard bar chart using ggplot2. Since ggplot2 does not allow combo charts to show the percentage growth line graph, I just used the geom_line function to draw a trend line and annotate it with the percentage change values
If we just compared 2008 to 2016, you would think that crime incidents remained almost the same but it was actually its lowest in 2014 and has been increasing since. Lets now analyze our data on a borough level.
Let's visualize a simple barplot and see total incidents on a borough level. If you are familiar with the Tidyverse meta-package, then the following should not be hard to understand.
london_crimes %>%                                       group_by(borough) %>%   summarise(Incidents=sum(total_incidents)) %>%   ungroup() %>%   ggplot(aes(reorder(borough,Incidents),Incidents))+  geom_bar(stat = "identity",aes(fill=borough),color="black")+  coord_flip()+  scale_y_comma()+  geom_text(aes(label=comma(Incidents)),hjust=1)+  theme_classic()+  theme(legend.position = "none")+  labs(x=" ",y=" ",title = "Total Incidents for boroughs from 2008-2016 ")
I am not familiar with London’s geography but Westminister takes the lead by a considerable margin. I googled a bit and found that Westminister is one of the most densely populated areas in London.
How has the crime rate changed over the years for these boroughs?
For that, I am going to compare the data in 2016 with 2014 rather than in 2008 because crime incidents were lowest in 2014 and comparing it with 2016 could show some interesting changes.
london_crimes %>%   group_by(borough) %>%  summarise(Incidents_2014=sum(total_incidents[year(Date_Column)==2014]),         Incidents_2016=sum(total_incidents[year(Date_Column)==2016])) %>%   ungroup() %>%   mutate(Pct_Change=(Incidents_2016-Incidents_2014)/Incidents_2016)    %>%        ggplot(aes(reorder(borough,Pct_Change),Pct_Change))+                         geom_bar(stat = "identity",aes(fill=borough),color="black")+                 coord_flip()+  scale_y_continuous(labels = percent_format())+                               geom_text(aes(label=percent(Pct_Change)),hjust=1)+  theme_classic()+  theme(legend.position = "none")+                                             labs(x=" ",y="Percentage Change ",title = "Percentage Change in Incidents from 2014-2016")
Again we group the data by borough and calculated two conditional sums — Incidents_2004 and Incidents_2016 and calculate the percentage difference between the two to see change across every borough
Then its a standard procedure as before to plot our barplot, reordering based on highest to the lowest percentage change
The highest percentage for the City of London is a bit of a misrepresentation since it had a minuscule number of crime incidents
Surprisingly, Westminister is second-to-last even though it is number one in total incidents. At least, it is not experiencing a considerable increase.
I would like to see in which year did each borough experienced their historical highest number of crime incidents and also which year had the most of number of boroughs which experienced their highest numbers.
london_crimes %>%   group_by(borough,Yearly=floor_date(Date_Column,unit = "year")) %>%   summarise(Incidents=sum(total_incidents)) %>%  ungroup() %>%   group_by(borough) %>%  filter(Incidents==max(Incidents)) %>%    ungroup() %>%   mutate(borough_max_year = paste0(borough,"-","(",year(Yearly),")")) %>%                                                                      ggplot(aes(reorder(borough_max_year,Incidents),Incidents))+  geom_col(aes(fill=borough_max_year),color="black")+  scale_y_comma()+  coord_flip()+  geom_text(aes(label=comma(Incidents)),hjust=1)+  theme_classic()+  theme(legend.position = "none")+  labs(title = "Max Incidents for each Borough",x="Borough and year of max Incidents ",y="Incidents")
We start by grouping by year and borough and get the total incidents against each combination of borough and the corresponding year
Since I want to filter for the max year for each borough based on incidents, I group the data by borough and then filter where the Incidents value of each borough is equal to their max value. This will also show the year of that value
For visualization sake, I created a new column — borough_max_year which is basically a concatenated version of the Yearly and Incidents columns using the paste0 function. The rest is the same code I have been using for visualizing my bar plots.
Now we can see in which year each borough experienced their max number of crime incidents but I am now going to make another visualization to see which year had the most number of boroughs experiencing their highest numbers.
london_crimes %>%   group_by(borough,Yearly=year(floor_date(Date_Column,unit = "year"))) %>%   summarise(Incidents=sum(total_incidents)) %>%  ungroup() %>%   group_by(borough) %>%      filter(Incidents==max(Incidents)) %>%    ungroup() %>%   count(Yearly,sort = TRUE,name = "Boroughs_with_max_incidents") %>%  ggplot(aes(reorder(Yearly,Boroughs_with_max_incidents),Boroughs_with_max_incidents))+  ggalt::geom_lollipop(aes(color=factor(Yearly)),point.size = 10)+  scale_y_comma()+  coord_flip()+  geom_text(aes(label=Boroughs_with_max_incidents),hjust=0.5,color="white",fontface="bold")+  theme_classic()+  theme(legend.position = "none")+  labs(title = "Number of boroughs that experienced their highest ever incidents",x="Year",y=" ")
Using the same code as before, I added an extra step where I counted the number of boroughs against each year
Bored of bar plots, I used the geom_lollipop function from the ggalt package to plot a lollipop chart
So 2016 had the most boroughs even though 2016 is third place in terms of total crime incidents preceded by 2008 and 2012
I am now going to check the overall trend of crime incidents for each borough.
london_crimes %>%   group_by(Year=floor_date(Date_Column,unit = "year"),borough) %>%   summarise(Incidents=sum(total_incidents,na.rm = TRUE)) %>%   ggplot(aes(Year,Incidents))+  geom_line(aes(color=borough),size=0.75,style="--")+  theme_pubclean()+  scale_y_comma()+  expand_limits(y=0)+  facet_wrap(~borough,scales = "free")+  labs(y="Total Incidents",x=" ")+  theme(legend.position = "none",strip.background = element_rect(fill="firebrick"),strip.text=element_text(color = "white",face="bold"))
I thought this would have been a really messy graph but it turned out a lot better than I expected. We can generally see on overall increasing trend from 2014 to 2016 for most boroughs.
Let's see how the contribution of each major category in total crime incidents and their trend over time.
# Piechartlondon_crimes %>%   group_by(major_category) %>%   summarise(Incidents=sum(total_incidents,na.rm = TRUE)) %>%   ungroup() %>%   mutate(color=RColorBrewer::brewer.pal(9, 'Spectral')) %>%  select(label=major_category,value=Incidents,color) %>%  pier() %>%   pie.size(inner=70, outer=100, width = 600, height = 450) %>%    pie.header(text='Crimes', font='Impact', location='pie-center') %>%    pie.subtitle(text='by major category')%>%    pie.tooltips()# Trend over timelondon_crimes %>%   group_by(Yearly=floor_date(Date_Column,unit = "year"),major_category) %>%   summarise(Incidents=sum(total_incidents,na.rm = TRUE)) %>%   ggplot(aes(Yearly,Incidents))+  geom_line(aes(color=major_category),size=0.75)+  theme_pubclean()+  scale_y_comma()+  expand_limits(y=0)+ # making sure that the yaxis for every facet starts at 0 otherwise the trend may look misrepresentitive  facet_wrap(~major_category,scales = "free")+  labs(y="Total Incidents",x=" ")+  theme(legend.position = "none",strip.background = element_rect(fill="firebrick"),strip.text=element_text(color = "white",face="bold"))
I used the pier package to plot an interactive piechart here because I have always found that plotting decent piecharts in ggplot2 is a bit trickier than it needs to be
pier requires that the columns need to be only labeled as “label”, “value” and “color”
The code for the trend graphs is the same as the one for boroughs. The only difference is that I replaced the borough variable with the major_category variable
Do note that I used the expand_limits(y=0) argument to make sure that y-axis in every line graph starts at 0 otherwise, they can look a bit misrepresentative
Major contributors like Theft and Handling and Violence Against the Person have seen a noticeable increase from 2014–2016
Seems weird that there has been no incidence of Sexual Offences and Fraud or Forgery after 2008. Probably because these crimes were later recorded under different categories
Now I want to see how different boroughs rank in each major category. I will only be looking at the top 40% since visualizing all the boroughs in a faceted plot looks very messy.
london_crimes %>%   group_by(major_category,borough) %>%   summarise(Incidents=sum(total_incidents)) %>%   filter(Incidents>quantile(Incidents,0.6)) %>% #filtering for only the top 40% to make the plot more readable  ungroup() %>%   mutate(borough=reorder_within(borough,Incidents,major_category)) %>%                                                            ggplot(aes(borough,Incidents))+  geom_bar(aes(fill=borough),stat = "identity",color="black")+  coord_flip()+  scale_x_reordered()+         scale_y_comma()+  geom_text(aes(label=comma(Incidents)),hjust=1)+  theme_classic()+  theme(legend.position = "none")+  facet_wrap(~major_category,scales ="free")+  labs(x=" ",y=" ",title = "Top 40% Boroughs by Incidence",subtitle = "Segmented by Major Category")+  theme(legend.position = "none",strip.background = element_rect(fill="firebrick"),strip.text=element_text(color = "white",face="bold"))
After grouping and summarizing, I keep the grouping context to filter for the top 40% boroughs under each major category using the quantile function
Then I reorder the boroughs under each major category based on crime incidents using the reorder_within function from the tidytext package. Reason for doing this is so that the barplots are ordered within each major category facet
Also, I want to see which are the dominant minor categories within each major category.
london_crimes %>%   group_by(major_category,minor_category) %>%   summarise(Incidents=sum(total_incidents)) %>%  ungroup() %>%   mutate(minor_category=reorder_within(minor_category,Incidents,major_category)) %>%  ggplot(aes(minor_category,Incidents))+  geom_bar(aes(fill=minor_category),stat = "identity",color="black")+  coord_flip()+  scale_x_reordered()+  scale_y_comma()+  geom_text(aes(label=comma(Incidents)),hjust=1)+  theme_classic()+  theme(legend.position = "none")+  facet_wrap(~major_category,scales ="free")+  labs(x=" ",y=" ",title = "Incidents by Minor Category",subtitle = "Segmented by Major Category")+  theme(legend.position = "none",strip.background = element_rect(fill="firebrick"),strip.text=element_text(color = "white",face="bold"))
So most of these are cases of harassment, assaults, and thefts. I am guessing Sexual Offences were categorized under Violence Against the Person after 2008.
Seems like businesses are mostly and personal properties relatively get robbed a lot. Although there is a lot of potential overlap between Theft and Handling, Robbery and Burglary.
Despite the grim nature of the dataset, I had a lot of fun exploring it. There is potential for more exploratory analysis as well. You could do an analysis of the safest boroughs in London or dig deeper on minor categories of incidents.