How safe are the streets of Santiago?
By: Marcelo Rovai
Tags: Geography, Python, Mapping, Data Science, Geopandas

How safe are the streets of Santiago?

Let's answer it with Python and GeoPandas!

Costanera Center, Santiago / Benja Gremler

Some time ago I wrote an article, explaining how to work with geographic maps in Python, using the “hard way” (mainly Shapely and Pandas): Mapping Geography Data in Python. Now it is time to do it again, but this time, explaining how to do it in an easy way, using GeoPandas, that can be understood as Pandas + Shapely at the same package.

Geopandas is an open source project to make working with geospatial data in Python easier. GeoPandas extends the datatypes used by Pandas to allow spatial operations on geometric types.

The motivation for this article was a recent project proposed by professor Oscar Peredo and developed with my colleagues, Fran Gortari and Manuel Sacasa for the Big Data Analytics course of our UDD’s (Universidad del Desarrollo) Data Science Master Degree.

The objective of that project was to explore the possibility of, taking advantage of state of the art Machine Learning Algorithms, to predict crash risk score for an urban grid, based on public car crash data from 2013 to 2018. By the other hand, the purpose of this article is simply to learn how to use GeoPandas, on a real problem, answering a question:

“How safe are the streets in Santiago?”.

If you want to know what we have done with the proposed project for our DS Master deegre, please visit its GitHub repository.

Installing GeoPandas

The first thing that you should do when working with GeoPandas is to create a new and fresh Python Environment and from that, install the package. You can simply install it using PIP, if all dependencies are installed as well:

pip install geopandas

But as recommended on GeoPandas official page, the best way to do it from a fresh environment is using conda (GeoPandas and all its dependencies are available on the conda-forgechannel):

conda install --channel conda-forge geopandas

Starting with GeoPandas

A great start to learn GeoPandas is to follow the article from Benjamin Colley: Let’s make a map! Using Geopandas, Pandas and Matplotlib to make a Choropleth map and also take a look on the work of Eduardo Graell-Garrido, in Spanish, Workshop de Cartografía en Python.

When working with geographic maps, it is crutial to define what kind of Earth projection will be used. In this article, the actual coordinates used are in lat-long (EPSG: 4326) mode, with its units in decimal degrees, and on the surface of a sphere or ellipsoid of reference:

WGS 84 (also known as WGS 1984, EPSG:4326)

Another possibility would be to work with Transverse Mercator Projection, that would be a 2D map in meters (For Chile ‘ESPG: 5361’):

This is your choice, but take in consideration that you can not mix them.

Let’s Map!

Once you have GeoPandas installed, let's start importing some basic libraries:

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

import seaborn as sns

import geopandas as gpd

from shapely.geometry import Point, Polygon

The first shape that we will download, will be the polygon that will define the area where we will work. In our case, Santiago urban area! The Vectorial map can be found at Chilean National Congress Library (BCN).

The available map, contains in shapefile format, all 400 urban areas of Chile. We should filter it to “Santiago” and converted it to ESPG:4326:

sf_path = "../data/BCN/areas_urbanas/areas_urbanas.shp"

sf = gpd.read_file(sf_path, encoding='utf-8')

stgo_sf = sf[sf.NOMBRE == 'Santiago']

stgo_shape = stgo_sf.to_crs({'init': 'epsg:4326'})

stgo_shape

At this point, we have a geopandas dataframe, that has only one line, which includes besides some data as length and area, the ‘geometry’, that is the coordinates of the polygon which “envelop” all city:

We can plot this geodataframe, the same way we are used to do with a normal pandas dataframe:

stgo_shape.plot()

Note that Longitude (horizontal axis), goes from around -70.80 (West) to -70.45 (East) and Latitude (Vertical axis), goes from -33.65 (South) to -33.30 (North).

You can confirm the exact city bounders using:

stgo_shape.total_bounds

Also, it is possible to get the center coordinates of the shape (or centroid):

stgo_shape.centroid

Below you can see the same area on Google maps, with the Pacific Ocean to the left (West) and the Andes and the Argentinian border to the right (East)

Santiago Urban Area — Google Maps

Importing Roads from OpenStreetMap

OpenStreetMap(OSM) is a collaborative project to create a free editable map of the world, built by a community of mappers that contribute and maintain data about roads, trails, cafés, railway stations, and much more. Rather than the map itself, the data generated by the project is considered its primary output.

The site GeoFabrik has data extracts from the OpenStreetMap project which are normally updated every day. This open data download service is offered free of charge by Geofabrik GmbH.

Start downloading the data from this link, and saving it on your /data/ depository under “/OSM/”.

From there, let’s open the shapefiles regarding roads:

roads_path = "../data/OSM_Chile/chile-latest-free/gis_osm_roads_free_1.shp" roads = gpd.read_file(roads_path, encoding='utf-8')

This file contains almost 455,000 roads. Let’s start filtering it with our Santiago shape (for that we will use .sjoin), so we can handle only the roads that are inside our area of interest.

roads = gpd.sjoin(roads, stgo_shape, op='intersects')

Even filtering, we finished with around 83,000 roads. Examining the roads geodataframe, we can see that it includes a column named fclass. Let’s give a look at it:

That means that several roads are mainly in residential areas and also the ones that are used as service, footway, cycleway, pedestrian, etc. Once we will be interested in mapping car crashes, let’s keep only the roads were most probably we will find them, which will reduce the dataset to about 12,000 roads.

We can also filter only the main roads (primary and motorway):

main_roads = car_roads[(car_roads.fclass == 'primary') |

(car_roads.fclass == 'motorway')

]

main_roads.plot()

Importing Car Crashes from a public dataset

In Chile, it is possible to find a public database for disaggregated crash data from 2013 to 2018 at CONASET website. From there we will download the data regarding the events that happened last year (2018).

The dataset contains almost 24,000 events reported but unfortunately, not all of them are geolocalized. So, first, let’s find how much data have the geometry data missed:

p0 = Point(0,0)

df_p0 = df_2018['geometry'] == p0

df_p0.sum()

The result is 3, 537 points. Let’s take them out and see how much clean data we have:

s_2018 = df_2018[df_2018['geometry'] != p0]

s_2018.shape

The result: 20,402 events. Very good! We can work with those data. Let’s filter them, to get only the events inside the Santiago area and plot them:

ax = stgo_shape.plot(figsize=(18,16), color='#EFEFEF', edgecolor='#444444')

main_roads.plot(ax=ax, color='green', markersize=0.2)

crashes.plot(ax=ax, color='red', markersize=8)

plt.title("2018 Road Crashs - Santiago, Chile");

plt.axis('off');

Wow! Visually seems that almost every road in Santiago had a car crash in 2018!

Digging into data

To really understand what happened, we need to go deeper into data.

Looking into the dataset, we can see that besides ‘geometry’, that contains the geolocalization of every crash event, also we will find relevant information as:

When the crash happened (Day and hour)

Location and zone (Rural and Urban)

Type of location (crossing road, roundabout, straight road, curve, etc.)

Number of lanes and type of construction (asphalt, concrete, etc.)

Road condition (dry, humid, oil, etc.)

Weather condition (rain, snow, clean, etc.)

Type of event (Collision, Bump, human hit, fire, etc.)

Severity (Fatal, Serious, Moderate, Minor, Non-Injury)

Let’s visualize some of those events and their main characteristics: