Handling Geospatial Data in AWS
By: Kovid Rathee
Tags: Data, Data Engineering, Software Development, Geospatial, Towards Data Science

DATA ENGINEERING

Handling Geospatial Data in AWS

With databases, warehouses, analysis, and visualization tools extending their capability to handle geospatial data, we are in a position to capture and use GIS data more than ever. While individual products supported GIS, cloud providers didn’t offer complete GIS solutions early on. With time, that has changed a lot. Cloud providers like AWS have extended support wherever it was required. For a software company like AWS, we need to understand that the choice of products they build is directly impacted by the requirements of usually their high-paying clients.

Now that a lot of companies see the need for geospatial data and that those companies are able to acquire and store it at a reasonably low cost (as location data is omnipresent and storage has gotten cheaper), we have seen wider adoption of geospatial databases, analysis, and visualization tools. Although AWS doesn’t have a full-fledged solution for Geospatial data, it does provide some great features in different services. In this post, we are going to discuss a bit about that.

Relational Databases

AWS has two managed relational database services — RDS and Aurora. You can obviously install your own relational database using EC2 but that isn’t a managed solution so we won’t count that here. Efficient storage of geospatial data is very important for anything we want to do with that data down the line.

AWS RDS simply supports whatever version of the relational database service it is running so the support for GIS is dependent upon the version of the database. For instance, if you’re running RDS for MySQL, you’ll have the PostGIS features supported for that MySQL version. If you’re running PostgreSQL, you’ll need to install the PostGIS extensions on PostgreSQL running on RDS.

Sample script to install PostGIS extensions on AWS RDS PostgreSQL.

On the other hand, using Aurora, which is an engine by AWS written for different relational databases, you can get some additional features, improvements, and optimizations on top of the usual features supported by the original versions of the relational databases. For instance, read about how Aurora has optimized geospatial indexing by using Z-order curves.

Warehouses, Lakes & Visualization

Redshift, although based on Redshift, didn’t support geospatial data for the longest time, finally launched support for it in 2019. Currently, in Redshift, we can use find and use native GIS data types. Here’s the full list of functions supported by Redshift for GIS data.

This is for Redshift but what happens when you have geolocation data stored in S3. How do you query that? Given that you have stored your geolocation data in the GeoJSON format, you should be able to create an external table in Athena using nature GIS types and query the data using Athena. See the following example for more details.

A complete Data Lake solution using Lambda & Step Functions for Geospatial data.

A quick note about visualization too — with AWS, you can use Quicksight to create geographical/geospatial charts using data from Athena and elsewhere. Although not as rich and powerful as full-fledged visualization tools like Looker and Tableau, Quicksight does do the trick when simple visualizations are required.

Services like SNS, SQS don’t care about what kind of data is being received by them as long as it is in the prescribed format. This means that if there is a producer of geolocation data, then it can be consumed by events/streaming services in AWS. Having said that, AWS IoT Events makes it really easy to tackle geolocation data (IoT events, in general).

Amazon Location Service

I don’t know if that phrase fits right with Amazon Location Service (ALS) which is set to compete with the likes of Google Maps in providing point-of-interest data to companies without compromising on data privacy and security, as there will be provisions of the data not leaving the AWS network ever, although this may come at an additional cost.

We are yet to see how mature this service is. Is it going to provide address translation, refinement, and normalization too? A couple of days ago I came across an amazing service called Placekey which tries to solve the problem of address normalization with great accuracy. We’ll wait and watch if Amazon has some tricks up its sleeve. Currently, Amazon Location Service is in preview.

Conclusion

With Amazon Location Service already in preview, is AWS going to concentrate more on getting more geolocation data specific services, maybe a specialized geolocation database or a visualization engine? In the last couple of years, AWS has definitely taken up a share in the time-series, graph, document database market. More geolocation services might actually be on the cards.