Spatial data science to solve the business problem of opening optimally-located sales agencies (Python)
By: Courriol Obouckang
Tags: Data Visualization, Unsupervised Learning, Python, Data Science, Spatial Analysis

Let’s quickly compare K-MEANS (Partitioning) vs DBSCAN (Density), by running them on a popular dataset called “multishapes” (embedded in the R-package factoextra) which looks like

We “visually” detect 5 distinct clusters + a set of loose points around (noise). We can validate this number of clusters displaying the cluster-label feature came with the dataset

plt.figure(figsize=(10,8))

plt.scatter(multishapes["x"],multishapes["y"], c=multishapes["shape"], cmap='tab20')

plt.title("Multishapes dataset")

plt.show()

There are indeed 5 clusters…

Let’s then run a K-MEANS for K=5.

k=5 from sklearn.cluster import KMeans kmeans = KMeans(n_clusters=k)

kmeans.fit(multishapes[["x","y"]])

kmeans_predict=kmeans.predict(multishapes[["x","y"]]) plt.figure(figsize=(10,8))

plt.scatter(multishapes["x"],multishapes["y"], c=kmeans_predict, s=50, cmap='tab10') plt.title("Clustering with K-MEANS")

plt.show()

As we can see above, K-MEANS didn’t work well, It has wrongly identified clusters.

Let’s now try with DBSCAN.

from sklearn.cluster import DBSCAN db = DBSCAN(eps=0.2, min_samples=10).fit(multishapes[["x","y"]])

labels_db = db.labels_ plt.figure(figsize=(10,8))

plt.scatter(multishapes["x"],multishapes["y"], c=labels_db, s=50, cmap='tab10') plt.title("Clustering with DBSCAN")

plt.show()

In contrast, DBSCAN has perfectly identified the clusters

So, which method to pick out?

PAM vs K-MEANS

Both centroïd (K-means) and medoïd(PAM) are suitable positions to host new agencies, but the centroïd seems more interesting for business: medoid is at the central position regardless of weights, whereas centroïd structurally tends (weighted sum!) to be nearer to the “weightful” zones, which sounds business-meaningful ! …

Illustration centroïd position vs medoïd

K-MEANS vs DBSCAN?

DBSCAN is more suitable to capture weird shapes and doesn’t require the number K of clusters as well, whereas K-MEANS have those parameters as cons. But, if data look “normal” and you are really comfortable with the number K of clusters, K-MEANS should also work well.

(Fig 3) DISTANCE AND CAR TRAVEL TIME

After having built the dense zones and found the best positions inside each of them (medoïd, centroïd,…) as likely future locations of agencies, we can get started to analyze how long it takes on average to serve cluster from those recommended locations in terms of distance and car travel duration.

Those metrics (distance and car travel time within-cluster) are crucial for the business to “measure” how well the recommended locations are located.

Before jumping on the dedicated tools that provide those metrics, we can have a theoretical look at the question: given the shape of the earth is a sphere…the distance between 2 positions identified by their geospatial coordinates longitude & latitude (in radians)

is measured by

or by the Haversine-formula more suitable for short distances:

where

On the side of dedicated tools, stands the API from Google called Google Distance Matrix API which is the most popular and precise to compute distance and car travel time.

Let’s quickly illustrate how to call it from Python.

Some parameters are required: to enable the connection with the API (key), and to customize the route. The complete documentation is here.

#required libraries

import pandas as pd

import requests

import datetime as dt #API Parameters #Get a key from Google Cloud Platform to enable the connection

API_KEY="your__api__key__from__google_cloud" #departure_time = in integer seconds since January 1, 1970

departure_time=str((dt.datetime(2019,10,24,8,30,0)-dt.datetime(1970,1,1,23,59,59)).total_seconds())

departure_time=str(departure_time).split('.')[0] #Mode="driving" #default value, other values : walking, .. #Units="metric" #default value (output in KM) #Traffic_model="best_guess" #default value, other values : pessimistic, optimistic #longitude, latitude of two random locations

Origins_coordinates= "48.7944,2.1414800000000014"

Destination_coordinates="48.9065,2.3215900000000147"

url="https://maps.googleapis.com/maps/api/distancematrix/json?units=imperial"

url_call_api="{}{}{}{}{}{}{}{}{}{}{}".format(url,"&origins=",Origins_coordinates,"&destinations=",Destination_coordinates,"&traffic_model=",Traffic_model,"&departure_time=",departure_time,"&key=",API_KEY) # urlurl_call_api="{}{}{}{}{}{}{}{}{}{}{}".format(url,"&origins=",Origins_coordinates,"&destinations=",Destination_coordinates,"&traffic_model=",Traffic_model,"&departure_time=",departure_time,"&key=",API_KEY) #API request

res=requests.get(url_call_api) #API output in .json

data=res.json()

data

Above is the .json output from the API: with our “timestamp” parameter (departure_time), it takes 23mn (~1406 sec) to reach the destination from the origin point over a distance of 9.3 miles (~14.983KM).

We need parsing this .json to get the distance and the travel time (and then put them in a structured data frame) as in the snippet below :

res=requests.get(url)

data=res.json()



if data["rows"][0]["elements"][0]["status"]=="OK":

distance=round(data["rows"][0]["elements"][0]["distance"]["value"]/1000,2)

travel_time=round(data["rows"][0]["elements"][0]["duration"]["value"]/60,2)

travel_time_in_traffic=round(data["rows"][0]["elements"][0]["duration_in_traffic"]["value"]/60,2)

result=[distance,travel_time,travel_time_in_traffic, data["rows"][0]["elements"][0]["status"]]



#KO

else:

result=[0,0,0, data["rows"][0]["elements"][0]["status"]]

Let’s curiously have a look at the gap between the distances from the sphere-based formula vs the Google API (column “DELTA” below), both computed on a random dataset of itineraries:

plt.figure(figsize=(7,5))

plt.bar(random_geocode_df.index,random_geocode_df["DELTA"])

plt.xticks(random_geocode_df.index)

plt.title("gap Distance sphere-formula vs API Google

by journey", fontsize=15)

plt.show()

We therefore note a large gap between those 2 measures and the distance based on the sphere-formula tends consistently to minimize the Google-distance. It’s indeed better to compute those metrics using the API because it reflects real life (the distance on the roads, the travel time in the traffic), while the trigonometric-distance is “as the crow flies”.

Let’s also mention that the Google API is not free! even if Google monthly gives free credits to use the API, which can be enough depending on the volume of your requests… I spent 0 $ for my real whole project. You obviously need to properly and optimally code to avoid calling the API uselessly.

There are other tools: open sources, Azure (https://azure.microsoft.com/fr-fr/services/azure-maps/) which doesn’t currently seem to have a Python SDK to call the API… Anyway, none of them, in my opinion, have the pros of the Google API: easiness, several possibilities to customize journeys,…

What about the model output?

You will likely need slicing the whole area into smaller “pieces” / “regions” depending on how big it is. You could start by using the official geographical division of the country, but my recommendation is to have this discussion with sales stakeholders. They better know the field and then will help you to make an optimal geographical slicing which fits the problem. So, at the end of your modeling, you’ll deliver N models for N smaller regions.

Anyway, as the model output, I recommend producing, for each cluster, the Top K (K=5, 6, …) of best-locations instead of 1 location. You then give more choices to stakeholders to find the final optimal location by combining our mathematical model and their own “field constraints”.

What about the model performance?

It sounds senseless here to produce similar mathematical metrics of performance as we do for predictive models. The model will be better evaluated by stakeholders: if the dense areas and the optimal locations recommended match well with their “field knowledge”.

However, it’s uncovered here, but if you’re interested in the performance evaluation of the intrinsic unsupervised model, there are some metrics such as those provided by Scikit-learn: Adjusted Rand index, Mutual Information based scores, … The generic idea to evaluate an unsupervised model is to compare the model vs an existing labeled-dataset.

What about automatization & industrialization?

It is not a model from a recurrent business need unless to be in a specific case. So, it seems meaningless to industrialize the process, just make sure that you can run your model on new data every time when it’s needed.