What is Exploratory Spatial Data Analysis (ESDA)?
Tags: Geographicdatascience, Data Science, Spatial Analysis, Statistics, Exploratory Data Analysis

Well, with this choropleth map, we can see binned price ranges, but that does not give us any statistics we can determine if there is spatial autocorrelation (Positive or Negative, or even where the hotspots and coldspots are. That is what we do next.

Spatial Weights and Spatial Lag

Before we perform any spatial autocorrelation, we first need to determine the spatial weights and spatial lag.

Spatial weights are how we determine the area’s neighborhood. There are different statistical methods that are used for determining spatial weights, and it is beyond this to provide an in-depth explanation of each in this article. One of the most commonly used spatial weights methods is Queen Contiguity Matrix, which we use. Here is a diagram explaining how the Queen contiguity matrix works ( included also is the rook contiguity matrix)

Contiguity Matrix Source

To calculate Queen contiguity spatial weights, we use Pysal.

w = weights.Queen.from_dataframe(avrg_price_airbnb, idVariable=”LSOA_CODE” ) w.transform = "R"

Spatial Lag is, on the other hand, is the product of spatial weights matrix for a given variable (in our case, the price). The spatial leg standardizes the rows and takes the average result of the price in each weighted neighborhood.

avrg_price_airbnb[“w_price”] = weights.lag_spatial(w, avrg_price_airbnb[“price”])

Now, we created a new column in our table that holds the weighted price of each neighborhood.

Global Spatial Autocorrelation

Global spatial autocorrelation determines the overall pattern in the dataset. Here we can calculate if there is a trend and summarize the variable of interest. Moran’s I statistics is typically used to determine the global spatial autocorrelation, so let us calculate that.

y = avrg_price_airbnb[“price”]

moran = Moran(y, w)

moran.I

And we get this number for this dataset 0.54. What does this number mean? This number summarises the statistics of the dataset, just like the mean does for non-spatial data. Moran’s I values range from -1 to 1. In our case, this number provides information that there is a positive spatial autocorrelation in this dataset. Remember that we are determining only the global autocorrelation with Moran’s I statistics. It does not tell us where this positive spatial autocorrelation exists ( We do that next).

We use Moran’s I plot to visualize the global spatial autocorrelation, which is identical to other scatter plots, with a linear fit that shows the relationship between the two variables.

fig, ax = moran_scatterplot(moran, aspect_equal=True)

plt.show()

Moran’s I Scatter Plot

Both Moran’s I and Moran’s I Scatter plot show positively correlated observations by location in the dataset. Let us see where we have spatial variations in the dataset.

Local Spatial Autocorrelation

So far, we have only determined that there is a positive spatial autocorrelation between the price of properties in neighborhoods and their locations. But we have not detected where clusters are. Local Indicators of Spatial Association (LISA) is used to do that. LISA classifies areas into four groups: high values near to high values (HH), Low values with nearby low values (LL), Low values with high values in its neighborhood, and vice-versa.

We had already calculated the weights (w) and determined the price as our variable of interest(y). To calculate Moran Local, we use Pysal’s functionality.

# calculate Moran Local

m_local = Moran_Local(y, w)

And plot Moran’s Local Scatter Plot.

# Plot

fig, ax = moran_scatterplot(m_local, p=0.05)

ax.set_xlabel(‘Price’)

ax.set_ylabel(‘Spatial Lag of Price’)

plt.text(1.95, 0.5, “HH”, fontsize=25)

plt.text(1.95, -1.5, “HL”, fontsize=25)

plt.text(-2, 1, “LH”, fontsize=25)

plt.text(-1, -1, “LL”, fontsize=25)

plt.show()

The scatter plot divides the areas into the four groups, as we mentioned.

Moran Local Scatter Plot — LISA

Now, this is cool, and we can see all values classified into four groups, but the exciting part is to see where these values cluster together in a map. Again, there is a function in Pysal (splot) to plot a map of the LISA results.

LISA Cluster Map -Airbnb Average price per neighborhood.

The map above shows the variation in the average price of Airbnb Properties. The red colors indicate neighborhoods clustered together, which have high prices surrounded by high prices as well (mostly the center of the city). The blue areas indicate where prices are low, also surrounded by areas with low-value prices (Mostly peripheries). Equally interesting is also Low-high and High-low area concentration.

Compared to the Choropleth map we started with this tutorial, the LISA is much more decluttered and provides a clear picture of the dataset. Exploratory Spatial Data Analysis (ESDA) techniques are powerful tools that help you identify spatial autocorrelation and local clusters that you can apply in any given variable.

Conclusion

In this tutorial, we have explored how we can perform Exploratory Data Analysis (EDA) for spatial data. The code for this tutorial is available in this GitHub with the notebooks and the data.

You can also directly run a Google Colab Notebook from here: