Using a free API to get geolocation information from a public IP address | by Abid Ebna Saif Utsha | Towards Data Science

This article is a demonstration of a project I have created using a free API. Initially, the purpose was to understand and learn more about API calls. I am working as an intern in a company which works with geospatial information. I got the inspiration to work on this from working there. I will try to explain the code and method I used in brief and along with a potential business perspective. Again, this is free and no additional verifications are needed. Let’s begin!
The business perspective of this project is to identify an area where people are accessing websites, basically audience segmentation. For example, an e-commerce website wants to know how many people in New York are browsing their website. So, imagining I am the owner of that e-commerce company, I am selling gaming gadgets online and want to know roughly how many people are accessing my website from the cities or all over the world. I only have the consumer’s public IP address. Now, I want to know from which state and countries my consumers belong to. This API will give me latitude and longitude from my consumer’s public IP Address. From there I can use geopy (Nominatim) to get address using that derived latitude and longitude. Quite easy, right. Now that the thinking and identifying the resources are done, let’s drive into some python code!
Wait a minute, I don’t have all the consumers public IP address, I was just imagining owning a gaming gadgets company. No worries, Python can support my imagination, let’s see how.
import randomimport socketimport structsocket.inet_ntoa(struct.pack('>I', random.randint(1, 0xffffffff)))
If you run the above lines in a Jupiter Notebook, you will see the following output.
Alright, all of these are great, but looking for one user at a time would be time-consuming and difficult, let’s make an automated python script which will generate 100 random IP addresses like these and look for their location and save the results in a CSV file. I have used VScode while developing this project, but for simplicity and ease of use, I will provide a colab notebook at the end which will contain source code. Also, I will give the GitHub link at the end as well.
To do that I would need the following libraries to be imported. requests, json are needed for the API calls, global_land_mask is used to verify the latitude and longitude(will be discussed later).
import requestsimport pandas as pdimport numpy as npimport jsonimport randomimport socketimport structfrom geopy.geocoders import Nominatimimport osfrom global_land_mask import globe
If you face any error while importing the libraries, just use the following commands in your terminal.
pip install <library>  #(pip install global-land-mask)orconda install <library>
The following is the function that generates 100 random IP addresses.
def getting_ip_address():    """This function returns a list of random IP address"""    new, explored=[],[]    i=0    while i<100:        ip = socket.inet_ntoa(struct.pack('>I', random.randint(1, 0xffffffff)))        if ip in explored:            continue        else:            new.append(ip)        i+=1    new = pd.DataFrame(new, columns=['ip'])    return new
I also created a list which stores generated IP Address which will be later used to check whether that IP Address already created before or not. I have converted the final list into a pandas DataFrame as I am more comfortable in working with pandas. Now, the next step would be to identify and get their latitude and longitude information.
def getting_ip(row):    """This function calls the api and return the response"""    url = f"https://freegeoip.app/json/{row}"       # getting records from getting ip address    headers = {        'accept': "application/json",        'content-type': "application/json"        }    response = requests.request("GET", url, headers=headers)    respond = json.loads(response.text)    return respond
This function takes each IP address and sends the request to the API to get latitude and longitude information along with other information. A sample output for one location is given below.
The output is in JSON format and is saved as a column in a pandas datagram. After that from that column, information like latitude, longitude, and tim_zone are deduced, which is shown below:
def get_information():    """This function calls both api and add information to the pandas dataframe column"""    new = getting_ip_address()    new['info'] = new['ip'].apply(lambda row: getting_ip(row))    new['time_zone'] = new['info'].apply(lambda row: row['time_zone'])    new['latitude'] = new['info'].apply(lambda row: row['latitude'])    new['longitude'] = new['info'].apply(lambda row: row['longitude'])    new['on_land'] = new.apply(lambda row: globe.is_land(row['latitude'],row['longitude']),axis=1)    new = new[new['latitude']!=0]    new = new[new['on_land']==True]    new['address'] = new.apply(lambda row: getting_city_nominatim(row),axis=1)    return new
This is the main information extracting function which calls two API and processes their information. First, remember generating 100 random IP address function shown above, that function is called and a dataframe containing IP addresses are returned. After that using lambda function to call API for each IP addresses and store that result in a column named ‘info’. After that, from that column for each IP address, time_zone, latitude, longitude is derived. Now, it comes to before mentioned global_land_mask library. I want to check whether the latitude and longitude fall under land areas or in the ocean. This globe.is_land(latitude, longitude) returns boolean True or False for each row which is saved in another column named on_land, later a simple filter has been done on dataframe to remove if any latitude is exactly 0 and if the latitude and longitude are in the ocean. After that calling the geopy Nominatim API to get a deeper address. The function that calls the geopy API is shown below:
def getting_city_nominatim(row):    """This function calls the geopy api and return the json address output"""    try:        lat = row['latitude']        lon = row['longitude']        geolocator = Nominatim(user_agent="my-application")        location = geolocator.reverse(f"""{lat,lon}""")        address = location.raw['address']        return address    except:        print('timeout')
This function calls geopy Nominatim using latitude and longitude derived from earlier API calls. For the same IP address is shown as a sample before ‘192.168.10.111’ returned latitude and longitude, the below image is showing a sample nominatim call using that latitude and longitude.
Now, I have all the building blocks ready, I need to save the results in CSV. the function for that is given below:
def append_to_existing_df(new):    """This function appends the new ip addresses to the dataframe"""    if os.path.isfile(f'{os.path.abspath("")}\location_of_ip_address.csv'):        new.to_csv('location_of_ip_address.csv', mode='a', header=False,index=False)    else:        new.to_csv('location_of_ip_address.csv', mode='w', header=True,\                   columns=['ip','info','time_zone','latitude','longitude','address'],index=False)def deleting_duplicate_entries():    """This function makes sure there are no duplicate ip addresses saved in the csv file"""    df = pd.read_csv('location_of_ip_address.csv')    df.sort_values('ip',inplace=True)    df.drop_duplicates(subset='ip',keep='first',inplace=True)    df.to_csv('location_of_ip_address.csv',index=False)
The first function above looks for whether there is a CSV file existing or not, if the file exists then the results are appended to that existing file, if the file does not exist in that directory then a new CSV file is created. os.path.abspath(“”) returns the path for the current directory where the python file is. and os.path.isfile() check if the file exists or not. After saving them and running the program multiple times, I noticed several IP addresses contains the same because the initial duplicate check was only for 100 random generate IP addresses. But now, I have run the program multiple times and the same IP address is found in multiple places. To remove that, an add on function was added later which read the whole CSV in a pandas dataframe and drop duplicates value and save the file back.
Now, that we have all the functions ready, the below block is showing the main function which calls all the functions created before.
def main():    """main function"""    new = get_information()    append_to_existing_df(new)    deleting_duplicate_entries()if __name__ == '__main__':    main()
After running, this creates a CSV which results are given below:
So, this was the project I created. I just gave an overview of how I came up with this idea, created materials, and automated a process. I didn’t delve into too many technical details. I am planning to write separately for that. This can be further improved and modified, for example, further automated the process so that it runs every day at certain times. The source code shown here is given in colab. Also, it is available in GitHub.
Thank you very much for reading the article. This is my first time writing for Medium, I will try to improve my writing and publish more often with some projects and ideas.