Python’s geocoding — Convert a list of addresses into a map | by Vaclav Dekanovsky | Towards Data Science

Every one of us has a database of clients with addresses, a list of factories or stores, regions we sell our products to or cooperate with. Displaying these data on a map is a must in the modern dashboards. In order to do so, we have to transform the addresses (city names or regions) into geospatial data — latitudes and longitudes. Then we can present them on compelling maps.
What will you learn
In this tutorial, we will explore how to:
choose the ideal mapping service
connect to the API using requests (old-fashioned way)
get locations (and more) with Python’s geopy library
convert a list of addresses into geopoints
display the collected data on a map with folium and plotly
save the map into .html file
You can follow along with me using this jupyter notebook downloadable from Github - Address to Location.ipynb.
To transform addresses into coordinates you can either buy a database of addresses with the geographical position or you have to query some geolocation service provider. Anyone who offers maps usually provides a geolocation API (often for a price). Let’s review some of the major providers:
Google Maps Platform
Mapbox API
Bing API
Nominatim, OpenStreetMap API — (free)
Yandex Map API
This article review some of the major providers:
medium.com
Most APIs allow direct calls. You send your address to an URL and receive the geolocation. For years we use python requests library to do it. Let’s explore requests to Google API.
To connect to Google API you need an API key. You have to ask for it (guideline) and then keep secret, because you pay for the service. Google gives $200 free each month which is enough to do 40000 geocoding requests.
Geocoding calls are usually quite simple:
Base address: https://maps.googleapis.com/maps/api/geocode/json?Parameters:   # blank spaces are turned to %20   address=1 Apple Park Way, Cupertino, CA   key=API_KEY
You prepare the data you want to search for. With the help of urllib you easily turn parameters into an URL and you call requests.get(). The response is either XML, but more often a json which you process by json.loads
import requests
import json
import urllib

# https://developers.google.com/maps/documentation/geocoding/intro
base_url= "https://maps.googleapis.com/maps/api/geocode/json?"
AUTH_KEY = "Your Key"

# set up your search parameters - address and API key
parameters = {"address": "Tuscany, Italy",
             "key": AUTH_KEY}

# urllib.parse.urlencode turns parameters into url
print(f"{base_url}{urllib.parse.urlencode(parameters)}")

https://maps.googleapis.com/maps/api/geocode/json?address=Tuscany%2C+Italy&key=Your+Key

r = requests.get(f"{base_url}{urllib.parse.urlencode(parameters)}")

data = json.loads(r.content)
data

{'results': [{'address_components': [{'long_name': 'Tuscany',
     'short_name': 'Tuscany',
     'types': ['administrative_area_level_1', 'political']},
    {'long_name': 'Italy',
     'short_name': 'IT',
     'types': ['country', 'political']}],
   'formatted_address': 'Tuscany, Italy',
   'geometry': {'bounds': {'northeast': {'lat': 44.4726899, 'lng': 12.3713555},
     'southwest': {'lat': 42.2376686, 'lng': 9.6867213}},
    'location': {'lat': 43.7710513, 'lng': 11.2486208},
    'location_type': 'APPROXIMATE',
    'viewport': {'northeast': {'lat': 44.4726899, 'lng': 12.3713555},
     'southwest': {'lat': 42.2376686, 'lng': 9.6867213}}},
   'place_id': 'ChIJezSAEFMr1BIRq1kgW7rDxro',
   'types': ['administrative_area_level_1', 'political']}],
 'status': 'OK'}
data.get("results")[0].get("geometry").get("location")

{'lat': 43.7710513, 'lng': 11.2486208}
You can struggle with the documentation of mapping provider API to get the most of it, but it is easier to take advantage of an existing library like GeoPy. With Geopy you can do the same as above with only a few lines of code.
Geopy provides a class for popular mapping services. Nominatim is the service behind the popular OpenStreetMap that allows you to geocode for free. But you should comply with the usage policies in order to allow everyone to use it:
As a general rule, bulk geocoding of larger amounts of data is not encouraged. smaller one-time bulk tasks may be permissible, if these additional rules are followed
limit your requests to a single thread
limited to 1 machine only, no distributed scripts (including multiple Amazon EC2 instances or similar)
Results must be cached on your side. Clients sending repeatedly the same query may be classified as faulty and blocked.
Do you have a small project or just want to impress your boss? Let’s geocode with Geopy.Nominatim. First, we initialize Nominatim into geolocator variable:
from geopy.geocoders import Nominatimgeolocator = Nominatim(user_agent="example app")
Then you request information about a region, I’ve picked Tuscany in Italy:
[In]: geolocator.geocode("Tuscany, Italy").raw[Out]: {'place_id': 232933113, 'licence': 'Data © OpenStreetMap contributors, ODbL 1.0. https://osm.org/copyright', 'osm_type': 'relation', 'osm_id': 41977, 'boundingbox': ['42.237615', '44.4725419', '9.6867692', '12.3713544'], 'lat': '43.4586541', 'lon': '11.1389204', 'display_name': 'Toscana, Italia', 'class': 'boundary', 'type': 'administrative', 'importance': 0.6870417219974091, 'icon': 'https://nominatim.openstreetmap.org/images/mapicons/poi_boundary_administrative.p.20.png'}
From the response, you can get the latitude and the longitude
[In]: geolocator.geocode("Tuscany, Italy").point[Out]: Point(43.4586541, 11.1389204, 0.0)
Similarly, you can request geolocation from a full address, for example, the headquarter of Apple.
# request geolocator.geocode("1 Apple Park Way, Cupertino, CA")# extract the coordinates:('37.3337572', '-122.0113815')
Geopy allow geocode operation to turn an address into the coordinates and the opposite reverse to turn the locations into an address:
[In]: geolocator.reverse('37.3337572, -122.0113815')[Out]: Location(Apple Park, 1, Apple Park Way, Monta Vista, Cupertino, Santa Clara County, California, 94087, United States of America, (37.3348469, -122.01139215737962, 0.0))
# https://geopy.readthedocs.io/en/stable/
from geopy.geocoders import Nominatim
geolocator = Nominatim(user_agent="sample app")

data = geolocator.geocode("1 Apple Park Way, Cupertino, CA")

Extracting the data from the raw response can be tidious, because each provider return different data struture
data.raw.get("lat"), data.raw.get("lon")

('37.3337572', '-122.0113815')
using the .point, .point.latitude and .point.longitude makes it easy no matter what provider you are asking. The third tribute is altitude.
data.point

Point(37.3337572, -122.0113815, 0.0)
data.point.latitude, data.point.longitude

(37.3337572, -122.0113815)
.reverse helps you find the address based on the geolocation
geolocator.reverse('37.3337572, -122.0113815')

Location(Apple Park, 1, Apple Park Way, Monta Vista, Cupertino, Santa Clara County, California, 94087, United States of America, (37.3348469, -122.01139215737962, 0.0))
You can use many map services with Geopy, only you will have to provide an API key or user credentials. For example, using Google Maps API:
geolocator = GoogleV3(api_key=AUTH_KEY)
Requesting the geolocation uses the same code as above:
[In]: geolocator.geocode("1 Apple Park Way, Cupertino, CA").point[Out]: Point(37.3337572, -122.0113815, 0.0)
Though the structure of the response JSON will reflect the service used. Lat and Lon in the Google response are stored in the "geometry"."location" . But Geopy did the work for you and you just read the .point attribute.
Before you can display a dataset with addresses on a map you need to find the geolocations of these points. Let’s have a look at this shortlist of addresses:
data = """Name,AddressEU,"Rue de la Loi/Wetstraat 175, Brussel, Belgium"Apple,"1 Apple Park Way, Cupertino, CA"Google,"1600 Amphitheatre Parkway Mountain View, CA 94043"UN,"760 United Nations Plaza; Manhattan, New York City""""
We know that we can use GeoPy’s geolocator.geocode but what is the most effective way of turning a pandas dataFrame to coordinates. I like the approach of Abdishakur describes in this article
towardsdatascience.com
I’ll create a pandas dataFrame from our list
I’ll create a pandas dataFrame from our list
df = pd.read_csv(io.StringIO(data))df
2. Apply geolocator.geocode to the address column
df["loc"] = df["Address"].apply(geolocator.geocode)
3. Get .point containing latitude and longitude from the geocode’s response, if it’s not None.
df["point"]= df["loc"].apply(lambda loc: tuple(loc.point) if loc else None)
4. Split the .point into separate columns 'lat' 'lon' and 'altitude'
df[['lat', 'lon', 'altitude']] = pd.DataFrame(df['point'].to_list(), index=df.index)
Having 'lat' and 'loc' columns in the dataset is sufficient to display the points on any background map.
The geographical coordinates are useless unless you display them on a map. Python offers several libraries to make this task easy enough as well.
folium library, using leaflet.js to create a typical map on the top of OpenStreetMap or any other map data provider
plotly using its inherent map data
Folium maps are interactive, they contain markers with popups that can be clustered in areas with a high density of points, you can use layers and pick different source maps. And it’s all pretty easy to code:
# import the library and its Marker clusterization serviceimport foliumfrom folium.plugins import MarkerCluster# Create a map object and center it to the avarage coordinates to mm = folium.Map(location=df[["lat", "lon"]].mean().to_list(), zoom_start=2)# if the points are too close to each other, cluster them, create a cluster overlay with MarkerCluster, add to mmarker_cluster = MarkerCluster().add_to(m)# draw the markers and assign popup and hover texts# add the markers the the cluster layers so that they are automatically clusteredfor i,r in df.iterrows():    location = (r["lat"], r["lon"])    folium.Marker(location=location,                      popup = r['Name'],                      tooltip=r['Name'])\    .add_to(marker_cluster)# display the mapm
Plotly is gaining more and more popularity. Since it has introduced the Plotly express interface, creating a chart is a single liner and that applies to map charts as well. Let’s have a look at how to create a similar map with Plotly.
# import the plotly expressimport plotly.express as px# set up the chart from the df dataFramefig = px.scatter_geo(df,                      # longitude is taken from the df["lon"] columns and latitude from df["lat"]                     lon="lon",                      lat="lat",                      # choose the map chart's projection                     projection="natural earth",                     # columns which is in bold in the pop up                     hover_name = "Name",                     # format of the popup not to display these columns' data                     hover_data = {"Name":False,                                   "lon": False,                                   "lat": False                                     }                     )
.scatter_geo(df) creates the points on the map based on the data in the df dataFrame, like position or popups. We can set different size or color for each data point, but we didn’t do that in df dataFrame, so we can specify the same for all using .update_traces() .
fig.update_traces(marker=dict(size=25, color="red"))
You don’t have to determine the ideal zoom like in folium, but you just set fitbounds to ‘locations’. You can also show countries, rivers, oceans, lakes and specify their color with .update_geos .
fig.update_geos(fitbounds="locations", showcountries = True)
Finally, let’s add a title using .update_layout and display the final chart-map by fig.show() .
fig.update_layout(title = "Your customers")fig.show()
Admiring the map in the notebook can be satisfying, but occasionally you want to share your work with people who don't have python installed. Luckily exporting the map into .html is only one line of code.
Folium’s .save method creates a file that gets all the necessary resources like jquery, bootstrap, leaflet from CDN and the file has only 8 KB.
m.save("folium_map.html")
Plotly works with .write_html command which has a few optional parameters. The most important is include_plotlyjs . You can set if the output file will contain over 3 MB of plotly library which allows those beautiful chart effects or whether you want only your data points which can be included in another plotly project. In the second case, the size will be 9 KB.
fig.write_html("plotly_map.html", include_plotlyjs=True)
Displaying data on a map can be an effective way of how to present the company’s data. Python’s libraries help us to quickly gather the necessary geospatial data and use them to plot the maps. You can use open-source data or paid services to achieve this task.
Do you search for more inspiration, try plotly and folium map galleries
plotly.com
nbviewer.jupyter.org
Did you like this tutorial, check my other articles dealing with various data-related topics: