Visualizing real-time traffic patterns using HERE traffic api | by Skanda Vivek | Towards Data Science

While Google Maps shows live traffic, there’s no way to access the underlying traffic data. HERE technologies offers a variety of location based services including a REST API that provides traffic flow and incidents information.
HERE has a pretty powerful Freemium account, that allows up to 250K free transactions. However, while there’s some decent developer documentation, there’s a lack of online tutorials on how exactly to go about pulling traffic information and analyzing it. In this article I’m going to go through the basic process of getting traffic data from a network of roads, using python and xml parsing. Here’s a GIF of 3 hours of DC highway traffic data between 1:30 and 4:30 pm on a random Thursday in December 2019:
%matplotlib inlinefrom matplotlib import pyplot as pltimport numpy as npimport matplotlib.cm as cmimport requests#import dillfrom bs4 import BeautifulSoup#from datetime import datetimeimport xml.etree.ElementTree as ETfrom xml.etree.ElementTree import XML, fromstring, tostringpage = requests.get('https://traffic.api.here.com/traffic/6.2/flow.xml?app_id=BLAH&app_code=BLAH2&bbox=39.039696,-77.222108;38.775208, -76.821107&responseattributes=sh,fc')soup = BeautifulSoup(page.text, "lxml")roads = soup.find_all('fi')
First you need to register with HERE, and generate app id and app code credentials (BLAH and BLAH2 in the example above) — it’s pretty straightforward. Next you need to figure out the region of road networks you want to find traffic flow for, you can do that in Google Maps by clicking on some location and then right clicking on ‘What’s here?’which will give lat long coordinates. HERE traffic API needs 2 lat long pairs corresponding to the top left and bottom right, in order to give traffic flow information on roads contained within the rectangular bounding box. The response attributes are the information that will be included in the API response. In the example above, ‘sh’ denotes shapefile attributes, which are important as they give the geometry of the road and ‘fc’ denotes functional class, useful for later filtering only highways or streets etc. The last line of code gives an XML document where each child denotes a specific road segment, each with a unique shape file.
a1=[]loc_list_hv=[]lats=[]longs=[]sus=[]ffs=[]c=0for road in roads:    #for j in range(0,len(shps)):    myxml = fromstring(str(road))    fc=5    for child in myxml:        #print(child.tag, child.attrib)        if('fc' in child.attrib):            fc=int(child.attrib['fc'])        if('cn' in child.attrib):            cn=float(child.attrib['cn'])        if('su' in child.attrib):            su=float(child.attrib['su'])        if('ff' in child.attrib):            ff=float(child.attrib['ff'])    if((fc<=2) and (cn>=0.7)):        shps=road.find_all("shp")        for j in range(0,len(shps)):            latlong=shps[j].text.replace(',',' ').split()            #loc_list=[]            la=[]            lo=[]            su1=[]            ff1=[]                        for i in range(0,int(len(latlong)/2)):                loc_list_hv.append([float(latlong[2*i]),float(latlong[2*i+1]),float(su),float(ff)])                la.append(float(latlong[2*i]))                lo.append(float(latlong[2*i+1]))                su1.append(float(su))                ff1.append(float(ff))            lats.append(la)            longs.append(lo)            sus.append(np.mean(su1))            ffs.append(np.mean(ff1))
Above, I filtered all roads by functional class≤2 which denotes highways, and cn denotes confidence number, 0 is low confidence, 1 is the highest possible. In total, this DC highway system had around 5000 segments.
fig=plt.figure()plt.style.use('dark_background')#plt.plot(np.linspace(0,10,10),np.linspace(0,10,10))plt.grid(False)for i in range(0,len(lats)):    if(sus[i]/ffs[i]<0.25):        plt.plot(longs[i],lats[i], c='brown',linewidth=0.5)    elif(sus[i]/ffs[i]<0.5):        plt.plot(longs[i],lats[i], c='red',linewidth=0.5)    elif(sus[i]/ffs[i]<0.75):        plt.plot(longs[i],lats[i], c='yellow',linewidth=0.5)    else:        plt.plot(longs[i],lats[i], c='green',linewidth=0.5)    #print(i)#plt.xlim(-77.055,-77.015)#plt.ylim(38.885,38.91)plt.axis('off')plt.show()
Finally, I plot all the traffic speeds on the roads, with colors brown, red, yellow, green indicating worst to least traffic, similar to google maps. The one downside is that there’s no option to retrieve previous data, it’s all real time :/.
There’s a bunch of other services out there. For example TomTom offers road segment data very similar to HERE, except that it’s the speed information from a single segment closest to a given lat-long location. But to get a whole region, one needs to be familiar with zoom levels and tile grids. For zoom level 0, the world is displayed on a single tile. At zoom level 22, the world is divided into 2^44 tiles. It’s a big headache to figure out how a particular location like DC corresponds to what tile. Waze also seems to have some options to get it’s traffic data, however you need to be enrolled as a Waze partner, and that’s a whole nother process.
So there you go, you can now visualize real time traffic and patterns! There’s a lot more cool stuff one can do, for example meshing this with incidents data to predict how traffic changes based on incidents. Or making it nice looking in conjunction with a web mapping platform like Leaflet or Mapbox. Location and traffic based services are here to stay. Think Uber, Lyft, Bird, Amazon delivery, etc. All these companies need real-time location and traffic information under their hood. In the near future, fleets of autonomous delivery trucks might rely on such information make smart decisions. The future is boundless!
Follow me if you liked this article — I frequently write at the interface of complex systems, physics, data science, and society.
If you are not yet a Medium member and want to support writers like me, feel free to sign-up through my referral link: https://skanda-vivek.medium.com/membership