Visualizing DC Real-Time Traffic During The Curfew After Riots
By: Skanda Vivek
Tags: Society, Traffic, Python, Data Science, Science

Unfortunately, today’s riots resulted in chaos and loss of life. Many data providers are giving free access to real-time detailed information, which can be extremely useful for responders and citizens to react to sudden incidents like we saw today. Let’s delve a bit into the potential for quantifying real-time incident and flow data using one such API.

Google Maps is great when it comes to figuring out which route to take that minimizes your trip time. However, for the data enthusiast, it doesn’t give access to this traffic information. TomTom, Waze, HERE are all providers of real-time traffic information. Personally, I find HERE easiest to use and its free API very useful for obtaining detailed traffic information such as speeds and incidents. In a previous blog, I went through how to use the HERE traffic flow API to extract traffic speeds at the individual road segment level:

Here, I will illustrate how to pull out traffic incident data, and compare speeds/incidents from HERE with images from Google Maps.

%matplotlib inline

import numpy as np

import requests

from bs4 import BeautifulSoup

import xml.etree.ElementTree as ET

from xml.etree.ElementTree import XML, fromstring, tostring

soup = BeautifulSoup(page.text, "lxml") page = requests.get(' https://traffic.api.here.com/traffic/6.2/incidents.xml?app_id=BLAH&app_code=BLAH2&bbox=38.92,-77.06;38.88,-77.00&responseattributes=sh,fc' soup = BeautifulSoup(page.text, "lxml") incidents = soup.find_all('traffic_item')

You can register with HERE to get an app id as well as app code for credentials (BLAH, BLAH2 in the above example). Next, you need the bounding box within which you want to obtain flow/incidents. For this, right on google maps and you will see the lat-long pair. You need the top left and bottom right ones to get the coordinates of the bounding box. The response attributes fc and sh are important, useful later for filtering out highways, streets, etc., but I will not be using that information in this post. The last line gives all the unique incidents at that time.

myxml = fromstring(str(incidents[0]))

for child in myxml:

print(child.tag) traffic_item_id

original_traffic_item_id

traffic_item_status_short_desc

traffic_item_type_desc

start_time

end_time

entry_time

criticality

verified

abbreviation

rds-tmc_locations

location

traffic_item_detail

traffic_item_description

traffic_item_description

traffic_item_description

Looking at the first incident, there are 16 tags with useful incident information such as when the incident started and ended, descriptions, and the location. For now, I’m just interested in the location (lat, long) of the incident. It turns out this is not so simple as in the traffic speeds case. But with a little bit of poking around, I found it hidden within the location tag.

myxml = fromstring(str(incidents[0]))

for child in myxml:

if (child.tag==’location’):

for chelds in child:

print(chelds.tag,chelds.attrib) intersection

geoloc

political_boundary

navtech

length

Finally, I found the lat and long of the incident hidden in the geoloc tag:

incidents[0].location.geoloc <geoloc><origin><latitude>38.88507</latitude><longitude>-77.01081</longitude></origin><to><latitude>38.88602</latitude><longitude>-77.01051</longitude></to><geometry><shapes><shp fc="5" fw="SD" le="0.042374034341961575" lid="1186813844F">38.88507,-77.01081 38.88544,-77.01069</shp><shp fc="5" fw="SD" le="0.06625476521319096" lid="1254739814F">38.88544,-77.01069 38.88602,-77.01051</shp></shapes></geometry></geoloc>

You can also see what types of incidents are present. There were a total of 173 incidents at around 9 PM on 1/6/2021 in DC. All of these were described as either road closures or planned events. Here are the first 10. Note: HERE has a time which is clearly not EST. It might be GMT, though I’m not sure as of now.

for inc in incidents:

print(inc.start_time.text, inc.end_time.text,inc.traffic_item_type_desc.text) 12/15/2020 15:46:16 02/15/2023 17:00:00 ROAD_CLOSURE

01/06/2021 21:46:56 01/07/2021 21:45:44 ROAD_CLOSURE

01/06/2021 22:52:32 01/07/2021 22:52:07 ROAD_CLOSURE

12/15/2020 15:46:16 02/15/2023 17:00:00 ROAD_CLOSURE

01/06/2021 21:46:56 01/07/2021 21:45:44 ROAD_CLOSURE

12/15/2020 15:46:16 02/15/2023 17:00:00 ROAD_CLOSURE

01/06/2021 23:04:11 01/07/2021 23:01:47 ROAD_CLOSURE

12/15/2020 15:46:16 02/15/2023 17:00:00 ROAD_CLOSURE

12/15/2020 15:46:16 02/15/2023 17:00:00 ROAD_CLOSURE

01/06/2021 23:04:11 01/07/2021 23:01:47 ROAD_CLOSURE

Finally, here is a visual comparison of HERE traffic speeds/incidents with Google Maps as promised:

Left — Google Maps, Right — HERE data in Washington DC | Skanda Vivek

There are some interesting differences. HERE has more incidents and more data on speeds in some smaller road segments. Otherwise, in the absence of Google Maps data, it is hard to visually identify how similar/ different the speeds are. Also, I’m not sure how Google Maps identifies its speed by color. Google Maps has 4 colors for speeds from slow to fast (dark red, red, yellow, green). I made a similar color map for HERE, where road colors were segmented by speed relative to speed limit from: 0 — 0.25, 0.25 — 0.5, 0.5 — 0.75, 0.75 — 1 (0 corresponding to average speed of 0 mph, and 1 the speed limit).

HERE is a great resource for obtaining real-time information that you can’t get from Google Maps. Also, HERE has seemingly more detailed traffic information of smaller road segments, and they claim to be the World’s #1 location platform, quite a bold claim. The only downside is that they don’t have historical traffic data available — but hopefully that’s a future option.

As we saw today, sometimes groups of people can act collectively in ways that are previously unthinkable and cause great harm. When this happens, we need to have an accurate picture of evolving landscapes. Today illustrates how the DC government was largely unprepared for the scope of protests and the suddenness with which the Capitol was infiltrated.

Real-time traffic data integrated with other sources of real-time data including social media messages like tweets; serve as a pulse of society. These data give powerful, actionable information that city stakeholders such as law-enforcement, emergency responders, and even citizens can use to correctly assess the severity of the situation, and make key decisions, when time is of the essence.

Follow me if you liked this article — I frequently write at the interface of complex systems, physics, data science, and society