Analyzing earthquakes in USA to determine the possibly risky to quakes hotels.
By: Swastik Nath
Tags: Clustering, Machine Learning, Earthquake, Data Science, AI

Photo by Sunyu Kim on Unsplash

Ever wondered if there was a prediction system that could accurately predict the earthquakes far before it could hit the grounds beneath our feet, the fatalities and the destruction of several skyscrapers, homes, hotels would have drastically decrease and so we could have escaped from one of the greatest calamity of nature and the quakes would no longer become a reason behind the demise of a single people.

Suppose, you went to a beach city in the pacific coastal part of USA and checked in a hotel. Now, you’re enjoying the sea view and suddenly you feel a vulnerable tremor and the hotel, possibly a skyscraper collapsed like a toy and now you’re wondering if you could knew which hotels in these earthquake heavy places needs to be scrutinized and upgraded with anti-quake devices, you wouldn’t have chosen this hotel for staying by only being influenced by the user ratings.

But it’s possible by the advancement of today’s very powerful Data Science tools to identify the places where the chances of the tremors are highly possible and by that place, one could easily fetch the data of the nearby business venues including business venues, apartments, or just anything to measure the risk factors and the structural strengths to equip them with the quake-resistant devices and to upgrade them, leading to securing the residents and the visitors.

So, What we are going to do? A High Level Overview:

So, right now, the problem is clear to us. We need to analyse earthquake records and have to determine the places of it’s higher occurrence as well as to determine the hotels which are possibly at risk due to their location. With this problem in mind, let’s take a methodical approach to solve this problem. We will be following the methodology as prescribed by Dr. John Rollins to solve this problem.

First, let’s fetch a data-set containing earthquake data for United States from a certain data provider to start working with the data. After the fetching stage is completed, let’s proceed and clean the fetched data-set by dropping the unnecessary columns and replacing the null or invalid values with something that makes sense and thus making all the columns equally significant by normalizing the values with irregular range.

Now we analyse the data-frame’s different columns and their relationship with the other one. In simple words, we now try to find a relationship among the column values for analytical purposes. In this process we use several Python visualization libraries and packages to support our findings.

Now, we group the entire data frame to search for unique places with their frequency of occurrence.

Next we will go ahead and collect the the geocoded latitude and longitude of the most frequent places where the quake events were recorded. We will pass this latitude and longitude data to our street level information provider to fetch a result-set containing all the information about different business places information including information regarding nearby hotels around the recorded places.

Selecting the Earthquake and Street Level Data Providers and requesting data:

Before, we can dive any deeper into the analysis, we need loads of data and information. Selecting reliable yet affordable data providers are the primary things which, in data science can make a difference. We have to make sure that data we are provided with are efficient, reliable, flexible to use, modification and customization. As a matter of fact, if it’s human reviewed, instead of being automatically generated, the accuracy and the reliability will increase.

So, keeping all the things in mind, I have decided to go with the United States Geological Survey(USGS) for earthquake data. USGS provides a very intuitive, easy-to-use, reliable API and web portal service, which provides flexibility in output format, specifying regions of interest and more. USGS is a government operated research center and the data they provide are free of cost and are very reliable because most of them are reviewed by human before their registration. The API request link does not require any authentication.

So, we specify our region of interest to be USA, select our period of interest to be 2010 to 2019 and output format to Comma Separated Value(CSV) and hit search to obtain the result. We can use the API for requesting the data. The format of the request URL is shown as follows.

I have already downloaded a formatted the CSV dataframe to use. You can follow the link : https://s3.amazonaws.com/swastiknathgroup/Earthquake_results.csv to download and use the file on your own notebook or any other python environment.

Once the data is fetched, we create a Pandas Dataframe from the CSV file by using the pd.read_csv() function and drop all the unnecessary columns by using the pd.drop() function providing the columns to remove as a list object as an input parameter of the drop function.

I have also decided to go with Foursquare API for street level information about business places around USA including places of my interest. Their API is very affordable to use, is well documented and reliable. You will have to authenticate your API request URL every time you connect to the Foursquare servers and request data. The format of authentication is as follows:

#credentials for communicating with Foursquare API:

CLIENT_ID= 'your client id'

CLIENT_SECRET = 'your client secret'

VERSION = integer date of version.

Once we are finished with fetching the earthquakes data, we will define a python function to take as input the latitude and longitude list returned by the USGS earthquake data-set and use the data to search for nearby hotels around the vulnerable to earthquake places. The custom function whats_nearby also retrieves data from the JSON response and from a Pandas Data-frame to better represent the values.

#custom function for requesting street-level business information #from foursquare

pbar = ProgressBar(widgets=[Percentage(), Bar()], maxval=300).start()



import requests

def whats_nearby(latitudes, longitudes, radius = 15000):

nearby_places_list1 = []



status = 'EXECUTION STARTED, PLEASE WAIT ->'

for lat, lng in zip(latitudes, longitudes):

for i in range(len(latitudes)):

pbar.update(i+1)

pbar.finish()

query_url = '

CLIENT_ID,

CLIENT_SECRET,

VERSION,

lat, lng,

15000,

100)

response = requests.get(query_url).json()['response']['venues']



nearby_places_list1.append([(r['name'],

r['location']['lat'],

r['location']['lng'],

r['location']['formattedAddress']) for r in response])

vulnerable_places = pd.DataFrame(item for nearby in nearby_places_list1 for item in nearby)

vulnerable_places.columns = ['Venue', 'Venue Latitude', 'Venue Longitude', 'Address']



return vulnerable_places from progressbar import ProgressBar, Bar, Percentagepbar = ProgressBar(widgets=[Percentage(), Bar()], maxval=300).start()import requestsdef whats_nearby(latitudes, longitudes, radius = 15000):nearby_places_list1 = []status = 'EXECUTION STARTED, PLEASE WAIT ->'for lat, lng in zip(latitudes, longitudes):for i in range(len(latitudes)):pbar.update(i+1)pbar.finish()query_url = ' https://api.foursquare.com/v2/venues/search?&query=hotel&client_id={}&client_secret={}&v={}&ll={},{}&radius={}&limit={}'.format( CLIENT_ID,CLIENT_SECRET,VERSION,lat, lng,15000,100)response = requests.get(query_url).json()['response']['venues']nearby_places_list1.append([(r['name'],r['location']['lat'],r['location']['lng'],r['location']['formattedAddress']) for r in response])vulnerable_places = pd.DataFrame(item for nearby in nearby_places_list1 for item in nearby)vulnerable_places.columns = ['Venue', 'Venue Latitude', 'Venue Longitude', 'Address']return vulnerable_places

Analyzing the variables and the curve between their relation:

Let’s now try to understand the relationship between the variables of the feature set to understand the impact of individual variable upon the measurement of the earthquakes.

Ever imagined despite this very advancement of Machine Learning, supervised, unsupervised and deep reinforcement learning process what keeps building a hindrance between machine learning models and completely accurate prediction of earthquakes? Nature and data generated by the nature are simply very irregular and can not be accurately mapped into a mathematical relation, although every element of nature is driven by algorithms, they are very very complex to comprehend and irregular, unreliable for a machine learning model to understand. Every adeptly trained machine learning model tries to map the feature set with the target cell in the simplest way possible. We built a deep neural network to correctly predict earthquake, but that is still not as good as expected.

Let’s visualize the relationship between Root Mean Square Speed of the earthquake Energy Waves versus the Magnitude. We use the Seaborn package, which is built upon Matplotlib to automatically generate linear regression plots with the highest confidence possible. A great feature of this, it shows the deviation of the plots in terms of the regression line. In case of the RMS Vs. Magnitude, we see a linear relationship between the two, in spite of having minimal deviations on both sides of the line. These deviations can be ignored due to the natural irregularities.

sns.regplot(x= earth_quake_encoded['rms'], y= earth_quake_encoded['mag'], data = earth_quake_encoded, color = 'purple')

Regression plot between Root Mean Square Speed and Magnitude

Now, let us visualize the relation between Depth in which the earthquake wave ruptured versus the magnitude. The regression line with ignoble amount of deviation due to natural irregularities.

import seaborn as sns

reg_plot_depth_mag = pd.DataFrame(columns =['Depth', 'Magnitude'])

reg_plot_depth_mag['Depth'] = earth_quake_encoded['depth'].values

reg_plot_depth_mag['Magnitude'] = earth_quake_encoded['mag'].values sns.regplot(x=reg_plot_depth_mag['Depth'], y = reg_plot_depth_mag['Magnitude'], data = reg_plot_depth_mag, color = 'blue')

Regression plot between Magnitude and Depth.

It’s time now to visualize the curve of relationship between horizontal distance to the epicenter from the recorded even versus magnitude. The regression plot shows a linear regression line with high deviations, basically implying non-linearity between the two variables.

sns.regplot(x= earth_quake_encoded['dmin'], y = earth_quake_encoded['mag'], data = earth_quake_encoded, color = 'crimson')

Regression plot between Horizontal distance vs. Magnitude

Let’s now validate the data-set through visualization. Let’s take a look at the scatter plot between error in measuring magnitude versus magnitude to gain a confidence over the data-set we decide to plot the regression line as well. The result is what we expected. The regression line is negative in terms of its slope and is with minimal deviation, truly implying, if the error in measuring the magnitude increases, the actual magnitude decreases.

sns.regplot(x = earth_quake_encoded['magError'], y=earth_quake_encoded['mag'], color = 'orange')

Negative Regression plot between Magnitude and error in measuring magnitude

We now want to have a look at the time-frame, basically plotting the magnitudes continuously with respect to time. We use the Pandas Series.from_csv to create a series object and plot it directly using the pyplot scripting layer of Matplotlib. Before plotting, we ensure that the datatype of each of columns are correctly apprehended. The scripting layer is intelligent enough to plot the magnitudes in a smaller scale(for the sake of better visualization) with respect to the selected period of time.

time_vs_mag_bar = pd.DataFrame(columns = {'time', 'mag'})

time_vs_mag_bar['time'] = usa_earthquakes['time']

time_vs_mag_bar['mag'] = usa_earthquakes['mag']

time_vs_mag_bar.to_csv('time_vs_mag.csv') dates = mpl.dates.date2num(time_vs_mag_bar['time'].values)

plt.plot_date(dates, usa_earthquakes['mag'])

Time-frame representation of magnitude vs. time

from pandas import Series

series = Series.from_csv('usa_earth_quakes.csv', header=0)

series.plot(style='-')

plt.show()

Pandas Series plotting of continuous time-frame plotting of magnitude(in a smaller scale) vs. time.

We now want to visualize the data-set in a three dimensional space to even better resonate the variations along each of the axis. So, we use the Matplotlib’s mpl_toolkits.mplot3d package’s Axes3D function to plot 3-dimensional representation of Depth Vs. RMS Vs. Magnitude and Angular Gap between recording stations Vs. Number of Stations required to record the event vs Magnitude.

from mpl_toolkits.mplot3d import Axes3D

fig = plt.figure(figsize = (8, 6))

ax = fig.add_subplot(111, projection = '3d')

ax.set_xlabel('Depth')

ax.set_ylabel('RMS')

ax.set_zlabel('Magnitude')

ax.scatter(usa_earthquakes['depth'], usa_earthquakes['rms'], usa_earthquakes['mag'])

plt.show()

3-dimensional representation of Depth vs. RMS vs. Magnitude

fig = plt.figure(figsize = (8, 6))

ax = fig.add_subplot(111, projection = '3d')

ax.scatter(usa_earthquakes['gap'], usa_earthquakes['nst'], usa_earthquakes['mag'])

ax.set_xlabel('Gap')

ax.set_ylabel('Nst')

ax.set_zlabel('Magnitude')

plt.show()

3-dimensional representation of angular gap vs. number of stations vs. magnitude

Taking a look at the different maps representing the vulnerable to earthquake places in USA:

From the retrieved earthquakes data, we fetch the latitude, longitude and the common names of the places (obtained using Regex pattern). We use a python mapping library called Folium. Folium is creates highly interactive maps using Leaflet.js. So, at this point we plot the latitudes and longitudes in a map centered around United States and use a Folium plugin called HeatMap to generate a heat map representing the vulnerability frequency across USA along the map. Folium, being interactive, the heat map changes with zoom levels. We can also use another function called FastMarkerCluster to visualize the earthquake heavy regions in an uncluttered way. FastMarkerCluster is interactive too. They collapse and expand with the zoom level.