Google Maps Feature Extraction with Selenium | by Kyle Pastor | Towards Data Science

I plan this to be the first in a multi-part series of articles about the extraction of Google Maps data and doing some interesting path and time-based studies. A few examples are looking at traffic volumes over time and calculating a “Beauty Score” for a given route. The first step in this journey is to load image data from Google Maps automatically and extract features from those images. Let’s get started.
As for my toolkit, we are using Python for everything and leveraging the packages for Selenium (web browser emulator used for automation), Pillow (image handling library) and Matplotlib (plotting data values).
My working environment is with Kaggle which is an online data and notebook website which is super useful. To start I must first install the required software to allow us to emulate a browser in order to extract the image data from Google Maps (in addition to some other data).
The first step is to download and unpack a distribution of Firefox which will be our automated browser. This is the one I used:
http://ftp.mozilla.org/pub/firefox/releases/63.0.3/linux-x86_64/en-US/firefox-63.0.3.tar.bz2
Once extracted note the location of your directory. To make it clean I created a new working directory for this project:
mkdir ~/working/firefox
Now copy the extracted files into this new directory and set the permissions to allow execution by all.
cp -a firefox-63.0.3.tar.bz2/. ~/working/firefoxchmod -R 777 ~/working/firefox
Next, I needed to install a driver to allow for communication between python and firefox. While at it I also loaded the Selenium package.
Using pip:
pip install webdriverdownloaderpip install selenium
Finally, I got to my python code. In the Kaggle notebook, I ran the following to actually install the driver.
from webdriverdownloader import GeckoDriverDownloadergdd = GeckoDriverDownloader()gdd.download_and_install("v0.23.0")
Last but not least install some of the extra packages needed for running the code
apt-get install -y libgtk-3-0 libdbus-glib-1-2 xvfb# Setting up the virtual display for Firefoxexport DISPLAY=:99
Remember that I am going to run a web browser in the background and use it to load Google Maps and extract some data. To do this in Python first load the packages from Selenium and instance a Firefox session.
# Import the libraries for seleniumfrom selenium import webdriver as selenium_webdriverfrom selenium.webdriver.firefox.options import Options as    selenium_optionsfrom selenium.webdriver.common.desired_capabilities import    DesiredCapabilities as selenium_DesiredCapabilities
Next, I set up the browser options. I do not want an actual window of Firefox to get generated so instead I start it in “headless” mode.
browser_options = selenium_options()browser_options.add_argument("--headless")
Finally, I add the capabilities and instance the browser.
# Even more setup!capabilities_argument = selenium_DesiredCapabilities().FIREFOXcapabilities_argument["marionette"] = True# This is the money.  We now have the browser object readybrowser = selenium_webdriver.Firefox(    options=browser_options,    firefox_binary="~/working/firefox/firefox",    capabilities=capabilities_argument)
At last, I can start to grab some Google Maps data.
The next stage was to figure out the Google Maps URL I needed to use so I could navigate to it in Selenium. The format is as follows:
https://www.google.com/maps/@{LAT},{LNG},{Z}z
Where:
LAT: The latitude of the location we would like to see
LNG: The longitude of the location
Z: The zoom level on the map (bigger means closer)
Now I decided to select somewhere near the Toronto waterfront so I could get some park data for the area.
# Toronto Waterfront Coordinateslat = 43.640722lng = -79.3811892z = 17
I use this to generate a URL and navigate with my browser:
# Build the URLurl = 'https://www.google.com/maps/@' + str(lat) + ',' + str(lng) +    ',' + str(z) + 'z'# Setting up the browser window sizebrowser.set_window_size(1024,512)browser.get(url)
At this point, the browser was holding all of the information for the website as if I had done it manually (Inspect in the browser). I wanted to take a look at the site so I just used the save_screenshot method in Selenium.
browser.save_screenshot("before.png")
This is what it saw.
One cool thing about Selenium is that you can:
A) Parse the HTML to extract any page data
B) Execute javascript on the page
One extra thing I wanted to do was get the proper scaling of the image. For example, I want to know the actual area of parkland in a given map. What’s nice is that Google Maps includes a scale bar in its render. This means I can select those elements to get the number of feet (what's shown on the actual page) as well as the width of the scale bar in pixels. Therefore I have the number of feet per pixel in the image!
Since I am a Canadian I just converted this to meters:
# Conversion factor from foot to meterfoot2meter = 0.3048# Doing some light cleanup of the string and converting to floatscale_in_feet = float(browser.find_element_by_id('widget-scale    label').text.replace(' ft',''))# Now we have the scale in metersscale_in_meters = scale_in_feet*foot2meter# Get the number of pixels the scale is drawn on, again cleaningpixel_length = float(browser.find_element_by_class_name('widget    scale-ruler').value_of_css_property("width").replace('px',''))# Taadaa we have the final scaling MetersPerPixel = scale_in_meters/pixel_length
Next I had to clean up the image. As you can see there are some of the overlayed elements on the image (Search bar etc). If I want to extract features from the raw map then I had to get rid of them.
# Remove omniboxjs_string = "var element = document.getElementById(\"omnibox    container\");    element.remove();"browser.execute_script(js_string)# Remove username and iconsjs_string = "var element =    document.getElementById(\"vasquette\");    element.remove();"browser.execute_script(js_string)# Remove bottom scaling barjs_string = "var element = document.getElementsByClassName(\"app    viewcard-strip\");    element[0].remove();"browser.execute_script(js_string)# Remove attributions at the bottomjs_string = "var element = document.getElementsByClassName(\"scene    footer-container\");    element[0].remove();"browser.execute_script(js_string)ow I take another screenshotbrowser.save_screenshot("waterfront.png")
I have a clean image
Now that I have my image I want to extract some information on it. For this specific example, I want to know what the area of the green parkland covers in both a percentage and absolute terms (which is why I needed the scaling above). On top of this I want to make a nice animation of how the image is getting scanned and extracting the green.
Since I am making animations I want to store the images in separate folders to keep things clean. I do this for the scanning and line chart plotting:
mkdir ~/working/framesmkdir ~/working/matplotlib
Now I do the meat of the coding. I define a function that is able to scan over the pixels of the image and determine if they fall within a certain RGB range (with some slack). I also pass in an image object to allow for snapshots to be taken during the extraction. I basically overwrite all pixels to be white, and only if the pixel falls within the range I keep the original colours:
# The most important part!def find_pixels(img,pixels,colour_set,slack, size):    num_px = [] # List to hold the amount of pixels that match    # Set the value you want for these variables    r_min = colour_set[0]-slack    r_max = colour_set[0]+slack    g_min = colour_set[1]-slack    g_max = colour_set[1]+slack    b_min = colour_set[2]-slack    b_max = colour_set[2]+slack    # Loop over the pixel array    for x in range(size[0][0]):        num_px_col_count = 0        for y in range(size[0][1]):            # Extract the pixel colour data            r, g, b,a = pixels[x,y]            # Set the pixel to be white by default            pixels[x,y]=(int(255), int(255), int(255),int(0))            # Check the see if there is a match!            if r >= r_min and r <= r_max and b >= b_min and                 b <= b_max and g >= g_min and g <= g_max:                # If there is a match add one to count                num_px_col_count = num_px_col_count + 1;                # Colour the pixel back to the original                pixels[x,y]=(colour_set[0], colour_set[1],                   colour_set[2],int(255))        # Append the count to the list        num_px.append(num_px_col_count)        # Save the image every 10 frames for animation              if x % 10 == 0:            img.save('~/working/frames/' + str(x)+ '.png')return num_px
Now that the function is defined I can actually use it. Start with importing our image and animation libraries
from PIL import Image,ImageDraw,ImageFontimport matplotlib.pyplot as plt
Open the image, convert to RGBA and load the pixels into an array
img = Image.open('~/working/waterfront.png')img = img.convert('RGBA')pixels = img.load()
One thing that I had to do manually was determine the colour parkland is given in Google Maps. I just used a tool to select the pixel on the screen to extract this. Now I set it up and give it a slack. The slack is used as a buffer in case some pixels have small colour variability. I pass this into the function we defined above
park_colour = [197,232,197];park_slack = 2;num_park = find_pixels(img,pixels,park_colour,park_slack,[img.size])
At this point, I actually have everything I need! Great work! Time to get to animations. Remember how num_park contains the number of pixels that match at each column? This means I can plot this and create yet another animation. Using the image size I create a matplotlib figure and set all the background, axis and elements to white to get a blank slate.
# All this is to make a totally blank matplotlib imagefig=plt.figure(figsize=(1024/100,438/100))ax = fig.add_subplot(111)ax.set_facecolor("white")fig.patch.set_facecolor("white")fig.patch.set_alpha(0.0)for spine in ax.spines.values():    spine.set_edgecolor('white')plt.grid(False)plt.axis('off')
For image processing I want the images to be of the same scale so I went ahead and set the scaling in the figure
ax.set_xlim([-10,1024])ax.set_ylim([-10,438])
Finally, loop over every 10th elements and create a line plot. Once created simply save it!
for i in range(0,1024,10):    ax.plot(num_park[:i],color='#377e4d',linewidth=3,        antialiased=True)    plt.savefig('~/working/matplot/'+str(i)+'.png',        transparent=True,dpi=130)
I have all the images and data I need to make the final results.
First I get the total_area from the extraction. Using the MetersPerPixel and the image size I can get the total area in km2
total_area = img.size[0]*img.size[1]*MetersPerPixel/(1000*1000)
0.432 km2 ~20% of image
Next, use PIL to create an animated GIF. In the bottom example, I am looping over every 10th frame to capture load the pre-generated images from above and adding them into a list of frames. This frame list will get passed to the animation function to produce the final file.
# Loop over every 10 framesframes = []for i in range(0,1024,10):    # Load the other images we will overlay    googlemap = Image.open('/kaggle/working/frames/'+ str(i)+".png")    # Do resampling to get the smoothing effect    googlemap = googlemap.resize(googlemap.size,        resample=Image.ANTIALIAS)# Append the framesframes.append(googlemap)# Save the final .gifframes[0].save('googlemap.gif', format='GIF',    append_images=frames[1::], save_all=True, duration=1, loop=0)
As a final measure, I also overlay the animated matplotlib plot. The only difference is that we will use the .paste method of PIL to overlay the images.
# Loop over every 10 framesframes = []for i in range(0,1024,10):    # Load the other images we will overlay    googlemap = Image.open('/kaggle/working/frames/'+ str(i)+".png")    matplot = Image.open('/kaggle/working/matplot/'+str(i)+'.png')   # Overlay the matplotlib plot animation frames (NB we include   offsets for the image here has well (-175,-52) for alignment   googlemap.paste(matplot, (-175, -52),matplot)    # Do resampling to get the smoothing effect    googlemap = googlemap.resize(googlemap.size,        resample=Image.ANTIALIAS)# Append the framesframes.append(googlemap)# Save the final .gifframes[0].save('FINAL.gif', format='GIF',    append_images=frames[1::], save_all=True, duration=1, loop=0)
And there you have it! Now we can do some really interesting stuff like longitudinal traffic scanning or even look at a running route to see how cool the area around is!