How to create a 3D time-series map using Python and Kepler.gl
By: Erik Yan
Tags: Visualization, Data Science, Data Analysis, Data Visualization, Data

Visualizing time-series data using Kepler.gl

In the wake of protests rippling throughout the U.S., I wanted to better understand just how often interactions between police and victims lead to death. This presented a great opportunity to test out Uber’s Kepler.gl toolbox and create an interactive 3D map of police interactions that have resulted in death.

In this post I’ll talk about the map, what it does, and how we can interact with the data. I’ll also talk about the creation process and how you can easily make interactive geospatial visualizations (for those that are interested in the technical aspects of this project).

About the map

When you open the map you’ll notice a time series slider with a play/pause button. You can click the play button to animate the data and see how the number of deaths change over time (ranging from January 2013 to December of 2019).

Time series functionality in-action. (Image by author)

If you click on the > button (in the top left corner of the page), you can interact with the data to change what you see on the map. For example you can: hide layers, add new ones, filter the data by variables (such as race, gender, etc). I added a hidden layer that visualizes aggregated data based on geospatial radius. When the layer is visible, you’ll see calculated clusters:

Time series with cluster mapping layer enabled. (Image by author)

I won’t talk in-depth about everything you can do within the map interface. Feel free to check out a live version of the map here and play around with the map’s features / explore the data.

How the map was made

Let’s walk through the process of creating this map. To do this we leverage open-access datasets and Kepler.gl, Uber’s open-source geospatial analysis tool. Uber recently released a great article talking about Kepler.gl. It was built using React & Redux, Deck.gl, and WebGL.

If you’ve worked with Deck.gl, using Kepler.gl is even easier. Deck.gl already makes it easy to create WebGL-based visualization of large datasets. Kepler.gl further simplifies the process by providing an on-screen interface where you can quickly configure your data visualization. Map configurations can be saved for archiving or edited directly (if you want to make changes to your configuration script manually).

Your dataset will need to include three variables:

a ‘datetime’ variable (to enable time series functionality) a latitude variable (from the polygon centroid for each county) a longitude variable (from the polygon centroid for each county)

If you have a date variable already, you’ll want to modify the variable to include at least hours and minutes (i.e. convert the date to a date time format). Currently, Kepler.gl can only process date time stamps and not dates only (see Kepler.gl Github Issue #78).

A quick way to do this (if your date column is formatted as mm/dd/yy):

kepler_mvp_df[‘datetime’] = kepler_mvp_df[‘datetime’].astype(str) + ‘ 0:00’

We’ll want to include longitude and latitude information for Kepler time series maps because rather than assigning a geojson shape to each time stamp, we’ll use the centroid of each polygon as the central longitude and latitude of each distinct area/region.

The police interaction dataset I used (see dataset source/validity section) includes the street address, city, state, zip code, and county information. We’ll map the data by state counties because we can use the National Weather Service’s GIS shape files to extract each county’s central longitude and latitude using the centroid of each county’s respective geospatial shape.

I recommend using geopandas to extract the shapefile data:

shapefile_data = gpd.read_file("c_03mr20.shp")

print(shapefile_data) shapfile_raw = pd.DataFrame()

shapfile_raw = shapfile_raw.append(shapefile_data) shapfile_raw

Once you’ve extracted the longitude and latitudes, we simply merge the coordinate dictionary with the dataset using state and county variables (which should be present in both datasets). In this case, we’ll merge the data using State abbreviations and County names:

kepler_mvp_df = pd.merge(MVP_raw,merged_dict_df, on=[‘STATE’,’County’])

Now we’re ready to load our data into Kepler.gl and begin creating our visualization!

Setting up Kepler.gl for Jupyter Notebook:

There are many ways to use Kepler.gl. For this map, I used the Kepler.gl Jupyter widget. This method is useful because Kepler.gl loads within a cell in Jupyter Notebook, allowing you to manipulate your data and load it directly into Kepler without having to toggle between environments or transfer your dataset.

If you don’t have Jupyter Notebook, I highly recommend this tutorial. The easiest way to install Kepler is using pip:

pip install keplergl

I recommend pushing your data to a DataFrame using pandas. In this case our dataset was prepared within Jupyter Notebook, so we just directly import the dataset into the Kepler Jupyter widget:

from keplergl import KeplerGl kepler_map = KeplerGl(height = 800, data={‘data_name_here’: dataset_df}) kepler_map

In the above snippet, we’re importing the Kepler widget, specifying a window height, and defining/importing our dataset into the widget.

Once the data is loaded you can configure the map visuals to your liking using the Kepler.gl built-in interface.

(Image by author)

For this map, a heat map was overlaid along the x- and z-axis, where the heat density ranges from dark red to yellow as the density of deaths increase. 3D rendering of the map allows us to take advantage of the y-axis, using yellow lines to show the intensity of the heatmap density color (seeing a yellow dot in the center of a heat spot doesn’t tell you about its intensity relative to other yellow-colored hot spots, so the hexbin lines help us better visualize hotspot density by utilizing the y-axis).

Your configuration script will looking something like this:

new_config = {'version': 'v1',

'config': {'visState': {'filters': [{'dataId': ['police action that resulted in death'],

'id': 'o7g4tr5v',

'name': ['datetime'],

'type': 'timeRange',

'value': [1357006005000, 1374938411000],

'enlarged': True,

'plotType': 'histogram',

'yAxis': None},

{'dataId': ['police action that resulted in death'],

'id': 'p34jx073r',

'name': ["Victim's race"],

'type': 'multiSelect',

'value': [],

'enlarged': False,

'plotType': 'histogram',

'yAxis': None}],

'layers': [{'id': 'e136xu9',

'type': 'heatmap',

'config': {'dataId': 'police action that resulted in death',

'label': 'Heat',

'color': [231, 159, 213],

'columns': {'lat': 'LAT', 'lng': 'LON'},

'isVisible': True,

'visConfig': {'opacity': 0.5,

'colorRange': {'name': 'Global Warming',

'type': 'sequential',

'category': 'Uber',

'colors': ['#5A1846',

'#900C3F',

'#C70039',

'#E3611C',

'#F1920E',

'#FFC300']},

'radius': 40},

'hidden': False,

'textLabel': [{'field': None,

'color': [255, 255, 255],

'size': 18,

'offset': [0, 0],

'anchor': 'start',

'alignment': 'center'}]},

'visualChannels': {'weightField': None, 'weightScale': 'linear'}},

{'id': 'm9ia9z',

'type': 'hexagon',

'config': {'dataId': 'police action that resulted in death',

'label': 'Hex',

'color': [221, 178, 124],

'columns': {'lat': 'LAT', 'lng': 'LON'},

'isVisible': True,

'visConfig': {'opacity': 0.4,

'worldUnitSize': 8,

'resolution': 8,

'colorRange': {'name': 'Global Warming',

'type': 'sequential',

'category': 'Uber',

'colors': ['#5A1846',

'#900C3F',

'#C70039',

'#E3611C',

'#F1920E',

'#FFC300']},

'coverage': 1,

'sizeRange': [0, 500],

'percentile': [0, 100],

'elevationPercentile': [0, 100],

'elevationScale': 40,

'colorAggregation': 'count',

'sizeAggregation': 'count',

'enable3d': True},

'hidden': False,

'textLabel': [{'field': None,

'color': [255, 255, 255],

'size': 18,

'offset': [0, 0],

'anchor': 'start',

'alignment': 'center'}]},

'visualChannels': {'colorField': None,

'colorScale': 'quantile',

'sizeField': None,

'sizeScale': 'linear'}},

{'id': 'l2vlgiq',

'type': 'cluster',

'config': {'dataId': 'police action that resulted in death',

'label': 'Cluster',

'color': [23, 184, 190],

'columns': {'lat': 'LAT', 'lng': 'LON'},

'isVisible': True,

'visConfig': {'opacity': 0.05,

'clusterRadius': 110,

'colorRange': {'name': 'Uber Viz Diverging 1.5',

'type': 'diverging',

'category': 'Uber',

'colors': ['#00939C',

'#5DBABF',

'#BAE1E2',

'#F8C0AA',

'#DD7755',

'#C22E00']},

'radiusRange': [1, 40],

'colorAggregation': 'count'},

'hidden': False,

'textLabel': [{'field': None,

'color': [255, 255, 255],

'size': 18,

'offset': [0, 0],

'anchor': 'start',

'alignment': 'center'}]},

'visualChannels': {'colorField': None, 'colorScale': 'quantize'}},

{'id': 'ci0b6l',

'type': 'point',

'config': {'dataId': 'police action that resulted in death',

'label': "Victim's Name",

'color': [28, 27, 27],

'columns': {'lat': 'LAT', 'lng': 'LON', 'altitude': None},

'isVisible': False,

'visConfig': {'radius': 0,

'fixedRadius': False,

'opacity': 0.8,

'outline': False,

'thickness': 0.5,

'strokeColor': None,

'colorRange': {'name': 'Global Warming',

'type': 'sequential',

'category': 'Uber',

'colors': ['#5A1846',

'#900C3F',

'#C70039',

'#E3611C',

'#F1920E',

'#FFC300']},

'strokeColorRange': {'name': 'Global Warming',

'type': 'sequential',

'category': 'Uber',

'colors': ['#5A1846',

'#900C3F',

'#C70039',

'#E3611C',

'#F1920E',

'#FFC300']},

'radiusRange': [0, 50],

'filled': False},

'hidden': False,

'textLabel': [{'field': {'name': "Victim's name", 'type': 'string'},

'color': [255, 255, 255],

'size': 3,

'offset': [0, 0],

'anchor': 'start',

'alignment': 'center'}]},

'visualChannels': {'colorField': None,

'colorScale': 'quantile',

'strokeColorField': None,

'strokeColorScale': 'quantile',

'sizeField': None,

'sizeScale': 'linear'}}],

'interactionConfig': {'tooltip': {'fieldsToShow': {'police action that resulted in death': ["Victim's name",

"Victim's age",

"Victim's gender",

"Victim's race",

'URL of image of victim']},

'enabled': True},

'brush': {'size': 0.5, 'enabled': False},

'geocoder': {'enabled': False},

'coordinate': {'enabled': False}},

'layerBlending': 'additive',

'splitMaps': [],

'animationConfig': {'currentTime': None, 'speed': 0.5}},

'mapState': {'bearing': 12.35033335232777,

'dragRotate': True,

'latitude': 33.612636906131925,

'longitude': -98.63889376921583,

'pitch': 55.12552722162369,

'zoom': 3.5734484899775754,

'isSplit': False},

'mapStyle': {'styleType': 'dark',

'topLayerGroups': {},

'visibleLayerGroups': {'label': True,

'road': True,

'border': True,

'building': False,

'water': True,

'land': True,

'3d building': False},

'threeDBuildingColor': [9.665468314072013,

17.18305478057247,

31.1442867897876],

'mapStyles': {}}}}

Be sure to save your configuration script for future reference (or if you want to make changes manually to your config script):

current_config = kepler_map.config current_config

You can also export the map as an interactive html map:

kepler_map.save_to_html(file_name=”kepler_map.html”)

Always be sure to keep a copy of the final configuration files. You can use the config files to re-load map visuals in the Kepler Jupyter widget:

kepler_map = KeplerGl(height=800, data={‘data_name_here’: dataset_df}, config=current_config) kepler_map

OR you can also use saved config files to re-create interactive html maps quickly:

kepler_map.save_to_html(data={‘data_name_here’: dataset_df}, config=config, file_name=”kepler_map.html”)

And that’s it! Creating an interactive 3D map doesn’t get much easier than that.

Dataset source / validity:

The dataset I used came from mappingpoliceviolence.org, which was created by Samuel Sinyangwe and DeRay McKesson. I can’t attest to the accuracy nor validity of the dataset, but the dataset does contain deep detail about each individual data point listed in the set (such as demographic information, geographic information, and relevant links).