Package: usgeocoder
OverviewThank you for your interest in USGeoCoder package!
USGeoCoder is an easy and free-to-use package for geocoding US addresses with the US Census Geocoder API.
This package was created to solve two problems I encountered while trying to geocode data in my data pipelines:Geocode thousands of addresses in a reasonable amount of time without caps on total requests.Do it for free.TheUS Census Geocoder APIwas the best solution I found to meet these requirements.
There are limitations, of course (the main one being that this API only works for US addresses), but by sending requests in parallel, this package can geocode around 2,000 - 4,000 addresses per minute without ever hitting a rate limit or a total request cap.This package is designed to help anyone, from an individual data scientist or developer working on small projects to a business managing large data pipelines.
If this package helps you, I would love to hear from you! And I would love it even more if you give feedback or contribute to the package 😊Note:This package is in a Beta state, so please be aware that there may be bugs or issues. Thank you for your patience.Table of ContentsInstallationUsageAPI Request FunctionsBatch Geocoder FunctionGeocoder ClassContributeLicenseInstallationMake sure you have Python 3 installed, along with the pandas library.pipinstallusgeocoderUsageThis package consists of three main sets of functions and classes.API Request Functions (Forward and Reverse)Batch Geocoder Function (Parallelize API Request Functions)Geocoder Class (Data Manager for Batch Geocoder)The components will be detailed below in order.API Request Functionsfromusgeocoderimportgeocode_address,geocode_coordinatesIt is very simple to run a single request to geocode an address or a pair of coordinates.Addresses should look like this:123 Main St, City, State Zip.Coordinates should look like this:(Longitude, Latitude).# Forwardaddress='123 Main St, City, State Zip'response=geocode_address(address)# Reversecoordinates=(-70.207895,43.623068)response=geocode_coordinates(coordinates)Note:Notice coordinate pairs are stored as (Longitude, Latitude) or (x, y).
If results are not as expected, try switching the order of the coordinates.
For instance, Google Maps shows points as (Latitude, Longitude) or (y, x).
The order of (Longitude, Latitude) was chosen because it is consistent with the mathematical convention of plotting points on a Cartesian plane, and it is how many GIS systems order coordinate points.Batch Geocoder Functionfromusgeocoderimportbatch_geocoderThebatch_geocoderfunction will allow you to parallelize the requests in thegeocode_addressandgeocode_coordinatesfunctions.# Forwardaddresses=['123 Main St, City, State Zip','456 Main St, City, State Zip']located,failed=batch_geocoder(addresses,direction='forward',n_threads=100)# Reversecoordinates=[(-70.207895,43.623068),(-71.469826,43.014701)]located,failed=batch_geocoder(coordinates,direction='reverse',n_threads=100)Note:Thebatch_geocoderfunction has been optimized to run at a max of 100 forn_threads.
Increasingn_threadsbeyond 100 will increase the likelihood of hitting a rate limit error.Geocoder ClassfromusgeocoderimportGeocoderTheGeocoderclass aims to organize the geocoding process in a data pipeline.
When theGeocoderclass is initialized, it will create a directory calledgeocoderin the current working directory.
This new directory will store each address or set of coordinates seen by theGeocoderclass.
If this directory already exists, theGeocoderclass will instead load in the data from the directory.
A directory is created to avoid making duplicate requests to the API for the same address or set of coordinates, whether the request was successful or not.Using the Process MethodThe recommended way to use theGeocoderclass is to initialize it and then use theprocess()method to manage what actions to take in the geocoding process.
Theprocess()method will take a dataframe that has a column with complete addresses or sets of coordinates.
This column should be calledAddressorCoordinatesand be formatted the same as required by the API request functions.
By default, theprocess()method will perform the following:Add the data from the pipeline to theGeocoderclass.Forward geocode the addresses.Reverse geocode the coordinates from the forward geocoding step.Merge the geocoded data back to a copy of the original dataframe.Return the merged dataframe.Here is an example of using theprocess()method.geo=Geocoder()geocoded_df=geo.process(data=df)# orgeo=Geocoder(df)geocoded_df=geo.process()If you want to customize the geocoding process, you can flip certain steps toTrueorFalsein theprocess()method.
Here is an example of the defaults.geo.process(data=df,forward=True,reverse=True,merge=True,verbose=False)Note:TheGeocoderclass was designed assuming that most users will be geocoding addresses.
Therefore, the default behavior is to forward geocode addresses and then reverse geocode the coordinates from the forward geocoding step.
If you are strictly reverse geocoding coordinates, you can setforward=Falsein theprocess()method to skip the forward geocoding step.Using Separate MethodsIf you want to use theGeocoderclass to manage the geocoding process but would like to use separate methods for each step, you can do so.
Here is an example of the separate methods utilized in theprocess()method.geo=Geocoder(df)geo.forward()geo.reverse()geo.merge_data()geocoded_df=geo.dataNote:When adding data to theGeocoderclass, it is designed to add theAddressorCoordinatesas an un-duplicated list to itsaddressesandcoordinatesattributes.
When theforward()orreverse()methods are called, they look to these attributes for the data to geocode.
If you add a dataframe with bothAddressandCoordinatescolumns, theGeocoderclass will only populate thecoordinatesattribute as there is no need to forward geocode the addresses.
If theforward()method is called, it will raise an error.Using Helper FunctionsIf you have a dataframe with separate columns forStreet Address,City,State, andZip, and named accordingly, you can use a helper function to create a newAddresscolumn, or create the column yourself.
The below example illustrates a simple step to rename theexisting_colsto the required column names.# Create a new column with complete address using helper functionfromusgeocoderimportconcatenate_addressexisting_cols=['address 1','address 2','city','state','zip code','important feature']df=pd.DataFrame(columns=existing_cols)df.rename(columns={'address 1':'Street Address','city':'City','state':'State','zip code':'Zip'},inplace=True)df['Address']=concatenate_address(df)These steps work just the same for reverse geocoding to create a newCoordinatescolumn with separateLongitudeandLatitudecolumns.# Create a new column with complete coordinates using helper functionfromusgeocoderimportconcatenate_coordinatesexisting_cols=['x','y','important feature']df=pd.DataFrame(columns=existing_cols)df.rename(columns={'x':'Longitude','y':'Latitude'},inplace=True)df['Coordinates']=concatenate_coordinates(df)ContributeIf you would like to make this package better, please consider contributing 😊LicenseMIT