Working with Geospatial Data in Machine Learning
By: Younes Charfaoui
Tags: Feature Engineering, Machine Learning, Heartbeat, Python, Guides And Tutorials

Techniques & Ideas

After seeing what the data looks like, it’s time to shed some light on the feature engineering phase. Here, we’ll explore some ideas and techniques to extract and build better features that will improve our predictive model.

1- Use features as they are

The first thought that comes to mind is to use longitude and latitude as they currently are in the predictive model.

But bear in mind that the scale of the two variables is different, so you need to use a model that doesn’t require any normalization, like tree-based algorithms. Otherwise, you’ll have to perform normalization and include them in the feature set.

You can take this a step further by using a model that doesn’t need any normalization. Instead, you can just round the features to 3 or 4 decimal places, which will improve your model’s performance. Here’s how we can do this in pandas:

train = train.round({"Pickup Lat":4, "Pickup Long":4})

Another simple but powerful transformation is to convert the latitude and longitude to radians instead of degrees. This will basically represent the same thing but in a different unit. You can achieve that using the NumPy function radians() .

import numpy as np

train["Pickup Lat"] = np.radians(train["Pickup Lat"])

train["Pickup Long"] = np.radians(train["Pickup Long"])

2- Perform Clustering

One of the approaches to handle point coordinates is to treat them in relation to each other.

In order to do that, we perform clustering and then assign each cluster-id to a point. By doing this, we’ll create a categorical variable that we can one hot encode to get better results.

The choice of the clustering algorithm matters. I tested many algorithms like K-means, DBSCAN, and hierarchical clustering—the latter two seem to give better results when it comes to geospatial features.

This is true given that K-means works well when trying to maximize variance, which is good if the feature space is linear in nature. But if it’s non-linear, then hierarchical clustering and DBSCAN are the best choices.

The number of clusters will depend on the problem, but in general, we have to test and see what gives better results.

Here is a sample that illustrates using K-means and hierarchical clustering algorithms with 5 clusters each:

If we re-plot the data using the newly created clusters, we obtain the following plot:

Data points with their cluster IDs.

3- Reverse Geocoding

It often happens that the point’s coordinates are far enough from each other and the data rows contain coordinates from different cities or countries in the world.

We can make use of this fact to perform reverse geocoding, which basically converts point coordinates to a readable address or place name (i.e. cities or countries). That way, we can create a new categorical variable.

And from that, we can create a field for city or country, but in some cases, where the measured points are in the same city, that method won’t work well— it will create a city variable with only one unique value.

For that, you have to extract the street or something else that varies from point to point.

4- Distance Feature:

To demonstrate this next technique, let’s consider the problem of determining ETA (Estimated Time of Arrival). For this, the dataset will contain departure and arrival coordinates.

From that, you can create a distance feature. This will improve your model a lot if the problem relies on the said feature—and ETA problems are very much in this category.

If you don't have two coordinates, you can calculate the distance to a fixed point (e.g origin), which will depend on the context of your problem.

The distance formula we use here is the Harvsine Formula, which takes into consideration that the earth’s shape is a sphere and not flat, so the distance is more realistic.

Here’s an implementation of the Harvsine Formula and its application on our dataset:

5- Extract X, Y, and Z:

I found this method on the data science stack exchange platform. It can moderately help our model but might be less effective than the previous methods.

The idea here is to represent those latitudes and longitudes via 3 coordinates. The problem with latitude and longitude is that they’re 2 features that represent a 3-dimensional space. We can, however, extract X, Y, and Z (our 3rd dimension) using sin and cosine functions. In this way, these features can be normalized properly.

The rule to derive these coordinates is the following:

x = cos(lat) * cos(lon)

y = cos(lat) * sin(lon)

z = sin(lat)

Here is the Python code we need to apply on our dataset:

6- Other derived features

Besides all the aforementioned features that you can extract from geospatial data, you can handcraft some more with the following ideas:

Nearest city name Distance to the nearest city Nearest city population Nearest big city Distance to the nearest big city

These features include populations, city names, and distances—and they require a lot of code.