How to use APIs to get spatial features for your models | by Philipp Spachtholz | Towards Data Science

In this post, I would like to show you how you can use APIs to quickly obtain many features for your spatial datasets to build better data science models.
On my own way to becoming a data scientist and subsequently in my work as a data scientist I built several models based on spatial data, e.g. to predict house prices, or more recently to predict the expected usage of charging stations for electric vehicles. What I quickly found out was that these models performance (at least for smaller datasets) doesn’t profit as much from models complexity or extensive parameter search but instead from the availability of good spatial features.
I also realized that extracting these spatial features can be quite cumbersome: You need to identify and evaluate data sources, set up a new set of tools like a PostGIS database, OpenStreetMap tools (like Overpass, Nominatim, etc.), and learn a lot of new things like map projections, spatial indexing, specific query languages and so on slowing down your feature extraction a lot.
However, there is also an easier and quicker way to get your set of spatial features: Using an API that handles all these tasks, so you don’t have to.
If you are just starting to get into data science or don’t come from a computer scientist educational background APIs may appear intimidating at first (with specialized terms as resources, endpoints, basic authentication, etc.). They did for me, at least, which to some degree prevented me from using them.
However, by learning more about APIs in general and playing around with APIs offering free accounts, you will quickly become familiar with using APIs and be able to integrate them as a new useful resource into your modelling workflow. As most APIs are built in a similar way (which is one reason that makes them so popular), knowing how to use one will help you understand how others work, too.
I am currently building a new API at features4.com (which is now publicly open for testing) which provides free and easy access to a rich set of spatial features such as the number of restaurants in a radius of 500m or the distance to the nearest doctors). I will use the Features4 API as an example on how to use APIs.
So how do we use the Featurees4 API to obtain your spatial features?
The birds eye view
From a birds eye view using an API is much like calling a normal function. You need the function name, you pass parameters to it and you get back a result. That’s basically the same with an API (only with different names): You need a URL, send it some data to it and get back a response.
URL
The most important difference is that while your functions are hosted on your computer, APIs are hosted on web servers. Consequently the API “function name” will always contain an address where it is located, which usually is an URL (just as you would type in a browser to retrieve a website). For example, all URLs for the features4 API start with https://api.features4.com/v1, because this is where the API can be reached on the server. The number “function” for example is located at https://api.features4.com/v1/number.
Passing parameters
Once you know the URL you need to specify the arguments you want to send it to. How would you find out which parameters a function takes? You would just take a look at its documentation. With APIs you do the same: In order to know which parameters an API endpoint takes you take a look at the API reference. In such a reference you’ll usually find all information about the parameters, including their types, descriptions and allowed values.
One thing I did not mention yet is that APIs, in general, are more flexible than the typical function. Often times you can send your parameter values in different ways. One way you’ll find often is to pass them in the following form (so-called form urlencoded as you can append them to the URL like this):
param1=value&param2=300&param4=true
Another common way to specify the parameters is by sending them JSON encoded:
The good thing about JSON is that it supports both nesting parameters and different data types, such as strings, numbers and lists:
Method
There is one piece of information you need to give the API: The request method tells the server what kind of action you want the server to take. This again makes APIs more flexible, so that the same URL can serve multiple different responses. For example, GET indicates that an item should be fetched or POST means that data is pushed to the server (creating a resource, or generating a temporary document to send back).
The method also tells the server where to look for the parameters you passed. While with the GET method you append the parameter values to the URL, with the POST method parameter values will be sent as part of the request.
As with passing parameters the API documentation will let you know exactly what methods the server accepts and what response it will give you as a result. The Features4 API currently only uses the POST method for consistency and simplicity so that all calls to the API can be made the same way.
Response
APIs are very reliable. Unless their web server cannot be reached, they will always return you a response, whether your request succeeded or failed. To let you know whether all went well or if something went wrong it will send you back a 3 digit number (e.g. 200) the so called status code. Each status code has a specific meaning and there is a large list of status codes with standardized meaning. No worries you don’t have to memorize all of them, or the beginning just remember this:
If the status code starts with:
2 everything is good
4 you probably made a mistake (such as omitting a parameter)
5 the API made an error (so, not much you can do about it)
If everything worked well and you received a response with a status code of 200, you will also receive the data you asked the API for. For example, the Features4 API would return you the spatial feature you asked for, such as the number of restaurants within a radius of 500m. The format of the response depends on the API, but oftentimes you’ll receive it formatted as JSON, which you already know from passing the parameters:
If you receive a response with a status code starting with 4, meaning that something went wrong on your side, a good API will tell you more about the type of error you made, for you to be able to fix it quickly. For instance, if you omit a parameter named “radius” the Features4 API will tell you what went wrong:
So, this was a very brief introduction into how APIs work in general. There is a lot more to learn about it and the HTTP protocol in general and a good source of information is provided by Mozilla for example.
Let's move on to a concrete example on how to use the Features4 API to retrieve spatial features for a location. We will use Python as a language but basically all languages provide libraries for making requests to APIs.
Let’s say you are building a model to predict house prices and you think that centrality will be an important aspect of it. Assuming that bars and restaurants are located at higher densities in central parts of a city, you are interested in getting the number of restaurants in a radius of 500m around a given location.
Getting access
Almost all APIs require you to provide some credentials to access it, such as a username and password. Features4 allows you both to try out the API with a test username API_TEST_KEY and no password or your private API key after registration for a free account (including more features).
Getting your first feature
The first thing to do is to find out the URL, method and parameters for the feature Number of elements in Radius in the API Reference:
You can find the URL and method in the upper right corner, which says that you should use the POST method to the /number URL. Clicking on the URL will reveal the full URL https://api.features4.com/v1/number.
In the central part of the documentation you see all the parameters the API takes for this feature, which are:
lat: Latitude of the location
lng: Longitude of the location
element: The map element you are interested in (e.g. restaurant)
radius: The radius around the location in meters (e.g. 500)
To get latitude and longitude for a location you can, for example, use this website.
Once you have the URL, method, credentials and the parameters you can use the requests library to retrieve the feature. For your convenience, the API reference provides code samples in different languages you can just copy and paste to have the basic structure and then you just change the parameters as you wish. So in the example just replace bar with restaurant.
Great: You can see that the number of restaurants in a radius of 500 meters of a location (the city center of Munich) is 153.
Similarly, you can retrieve another feature: the distance to the nearest map element. For house prices, the distance to the nearest kindergarten might, for example, be of interest. The procedure to get this feature is the same as before:
Take a look at the appropriate section in the API Reference
Identify these pieces of information: URL /distance, method POST, and parameters lat, lng, element
Make the request
You have learned some basics of how to use an API to retrieve spatial features for your data science models using the new Features4 API.
Feel free to explore the API Reference for more features or have a look at the Documentation to see which map elements are available for the features.
Having knowledge of APIs will help you a lot as a data scientist, because for many things you don’t have to reinvent the wheel, but instead can use the work others have already done for you. Therefore leveraging an existing API will let you be productive a lot more quickly.
I hope this article helped you getting started using APIs.