How To Engineer Features with Geography
By: Nadim Kawwa
Tags: Programming, Machine Learning, Data Science, Business, Education

How To Engineer Features with Geography

Theory and Real Data Science Example

Every business that sells goods or services has a physical address. There is no doubt that location can make or break a business. For data scientists, extracting geodetic features, features relating to the Earth’s geometric shape, can provide clear insights and a significant increase in performance of machine learning models.

In this article we are going to use the Yelp Dataset, specifically the business.json file to extract features dependent on location.

We will also draw from the Handbook of Mathematics by Bronshtein, Semendyayev, Musiol, Mühlig (link to PDF) and the corresponding summary by Jan Philip Matuschek

Data Overview

The Yelp documentation gives a good overview of the features, below is a brief snippet.

{

// string, 22 character unique string business id

"business_id": "tnhfDv5Il8EaGSXZGiuQGg",



// string, the business's name

"name": "Garaje",



// string, the full address of the business

"address": "475 3rd St",



// string, the city

"city": "San Francisco",



// string, 2 character state code, if applicable

"state": "CA",



// string, the postal code

"postal code": "94107",



// float, latitude

"latitude": 37.7817529521,



// float, longitude

"longitude": -122.39612197,



// float, star rating, rounded to half-stars

"stars": 4.5,



// integer, number of reviews

"review_count": 1198

Certain areas tend to be highly developed and have a lot businesses. In addition, businesses that are similar tend to be clustered close together.

For example the business above is located in San Francisco’s SoMA neighborhood (and has great guacamole), and has many other restaurants clustered around it.

We will be using the latitude and longitude to query the dataframe for businesses around it. To do so we need a geography refresher.

Spherical Coordinate Systems

The Earth is an imperfect sphere with an an approximate radius of 6357 kilometers. The geographic coordinate system is a method that allows to locate points on the spheres and is often used in land surveying among other professions.

We define latitude as an angle which ranges from zero degrees at the equator to 90 degrees at the poles. Conversely, longitude runs from pole to pole and ranges from 0 to 180 degrees. The picture below illustrates these two concepts on a sphere.

Query Points in A Neighborhood

Now that we have our coordinate system defined, we need a method that defines a bounding box around a point of interest and grabs all points in a delimited area around it.

Two Points on a Sphere

As a first approach we can calculate the orthodromic distance between two points P(lat1, long1) and Q(lat2, long2) on a sphere with radius R :

This method is prohibitively slow since it means calculating the distance between a point and all others for all points within the dataset.

Angular Radius

A better approach the angular radius to describe how large a sphere or circle appears from a given point of view. For example if we want to cover a distance d on a sphere of radius R then the angular radius r is:

Bounding Box

We then define the nodes that completely enclose the area captured by the angular radius. Moving along the latitude means we can move along a meridian, keeping the longitude fixed. Therefore the bounding latitude is defined as:

Moving along a circle of latitude in order to find the bounding longitude does not work in the same way as above. In the image below the points T1 and T2 have the maximum and minimum longitude respectively. They are not on the same circle of latitude as the point of interest M , but are closer to the pole.

The latitude of T1 and T2 is:

Therefore the limiting longitudes are calculated as follows:

Where:

Math to Python

We can interpret the methodology above in python code:

In the example of Garaje, our code will return a dataframe that lists all businesses within a d_query=2km radius from Garaje per the image below:

Feature Engineering

Given a dataframe like the one we have, we can write a code that returns a copy (note: not a view) of a geographically bounded dataframe

What can we extract from this information? It turns out we can learn quite a lot:

How many businesses are there nearby (density)

Price range comparison

Stars rating comparison

Review count

etc…

When doing comparisons, it is often a good idea to scale the features. For example when comparing prices we can standardize (z score normalization) the price by using the mean and standard deviation of the queried dataframe.

Stars Rating

As a first feature we can plot the standardized star rating of each entry against its closest neighbors. It appears to be a Gaussian Distribution, such that most businesses in one neighborhood are fairly homogenous in terms of rating.

Price Range

The price comparison appears to be bimodal, a more detailed analysis might make the query more stringent to only include businesses of the same type.

Density Count

We define density as the count of businesses within the bounded box. There is no need to do any standardization for this feature.

Note that the plot below will vary a lot depending on the query radius and the amount of development in the area.

Conclusion

Proper feature engineering can give your machine learning model the edge it needs to perform better. By taking geography into account we can derive features that are dependent on location.

In the case of the Yelp dataset the AUC for each model improved by at least 5% when plugging in geodetic features. They also appeared at least once in the top 5 most important features across all algorithms!