Time Series Modeling with ARIMA to Predict Future House Price | by Bonnie Ma | Towards Data Science

Time series used to be a topic that I tried to avoid when I was in graduate school because my peers shared the classes were very theoretical and all the cases discussed in class are related to Finance, and unfortunately, I was not into Finance at that time. Thanks to my Data Science bootcamp, I have another chance to encounter time series and I found it very practical and useful in many contexts. This time, I used time series analysis and models to predict the 5 best zip codes to invest in Brooklyn, where my husband and I were looking to buy an apartment. In this blog post, I will share the basic knowledge you need to know about time series and how I predicted the house price using ARIMA models step by step.
First of all, I want to lay down the structure of a time series modeling project. I will explain in detail in later sections.
Step 1: Data processing
Step 2: Data exploration and visualization
Step 3: Decide on model approach and KPI
Step 4: Develop the model on training set and validate using the test set
Step 5: Fine-tune the model and make the prediction
Keep in mind that time series is just a sequence of well-defined data points measured at consistent time intervals over a period of time. Time series analysis helps us understand the hidden pattern, meaningful characteristics and statistics about the data.
Dealing with the dataset from Zillow.com, I have to first select the city and reshape the data frame from wide to long format using pd.melt and then transform to time series data.
import pandas as pdimport numpy as npimport matplotlib.pyplot as plt%matplotlib inlinedfm.set_index('Month', inplace = True)
With visualization, we can identify underlying trends and stories in the data. Let’s take a look at the changes in housing prices in Brooklyn over time.
An overall upward trend can be observed from the years 1996 to 2007, followed by a fluctuation from 2008 to mid-2010. Starting in 2011, the house price became more stable and continued to rise again. We can say pretty confidently that the house market crash in 2008 was the cause of this fluctuation and we do want to skip this period to have a more accurate prediction of the future.
There are three important characteristics we care about: Stationarity, Seasonality, and Autocorrelation.
Most time series models work on the assumption that the time series are stationary which means its statistical properties such as mean, variance, etc. remain constant over time. Ideally, we want to have a stationary time series for modeling. Dickey-Fuller test can be used to test if a time series is stationary or not. Note that the null hypothesis is: time series is not stationary.
from statsmodels.tsa.stattools import adfullerdftest = adfuller(ts)
Seasonality refers to the periodic changes and patterns that repeat within a fixed period. Sometimes, seasonality can be combined with an increasing or decreasing trend. Seasonal decomposition is always helpful to detect seasonality, trend and any noise in the dataset. Take Brooklyn housing data for example:
from statsmodels.tsa.seasonal import seasonal_decomposedecomposition = sm.tsa.seasonal_decompose(month_avg, model='additive')trend = decomposition.trendseasonal = decomposition.seasonalresidual = decomposition.resid# Plot gathered statisticsplt.figure(figsize=(12,8))plt.subplot(411)plt.plot(month_avg, label='Original', color='blue')plt.legend(loc='best')plt.subplot(412)plt.plot(trend, label='Trend', color='blue')plt.legend(loc='best')plt.subplot(413)plt.plot(seasonal,label='Seasonality', color='blue')plt.legend(loc='best')plt.subplot(414)plt.plot(residual, label='Residuals', color='blue')plt.legend(loc='best')plt.tight_layout()
We can clearly see from the chart above that there is an upward trend along with yearly seasonality. The next step is to test our residuals with Dickey-Fuller test.
Autocorrelation helps us study how each time series observation is related to its recent (or not so recent) past. It is easy to get that tomorrow’s house price is very likely to be related to today’s house price. ACF (Autocorrelation Function) and PACF (Partial Autocorrelation Function) are two powerful tools. ACF represents autocorrelation of a time series as a function of the time lag. PACF seeks to remove the indirect correlations that exist in the autocorrelation for an observation and an observation at a prior time step.m
Check the ACF and PACF of the monthly average price of all zip codes.
from statsmodels.graphics.tsaplots import plot_acf, plot_pacffrom matplotlib.pylab import rcParamsrcParams['figure.figsize']=7,5plot_acf(month_avg); plt.xlim(0,24); plt.show()plot_pacf(month_avg); plt.xlim(0,24); plt.ylim(-1,1);plt.show()
The ACF shows the time series has autocorrelation with the previous time period, however, PACF does not show significant partial correlation.
If we subtract 3 months ago’s value from the current month’s value, which in other words is taking a lag-3 difference. We can see in PACF plot when lag=2, there is a negative partial autocorrelation and that means the lag-1 difference is significant in the time series data.
plot_acf(month_avg.diff(periods=3).bfill()); plt.xlim(0,24); plt.show()plot_pacf(month_avg.diff(periods=3).bfill()); plt.xlim(0,24); plt.ylim(-1,1);plt.show()
Since our dataset is not stationary, and there is a seasonal component, it would be reasonable to use SARIMA model — Seasonal ARIMA (Seasonal Autoregressive Integrated Moving Averages with exogenous regressors). Without diving too deep on the methodology, I will focus on the important parameters now.
Per the formula SARIMA(p,d,q)x(P,D,Q,s), the parameters for these types of models are as follows:
p and seasonal P: indicate the number of autoregressive terms (lags of the stationarized series)
d and seasonal D: indicate differencing that must be done to stationarize series
q and seasonal Q: indicate number of moving average terms (lags of the forecast errors)
s: indicates periodicity of the time series (4 for quarterly, 12 for yearly)
KPI: use AIC to select best set of parameters
Since there are 29 zip codes in the Zillow dataset for Brooklyn, I decided to build SARIMA model on 3 sample zip codes first and then iterate through all other zip codes.
I first create a list of data frames and each data frame has the info of one zip code.
zip_dfs = []zip_list = dfm.RegionName.unique()for x in zip_list:    zip_dfs.append(pd.DataFrame(dfm[dfm['RegionName']==x][['MeanValue']].copy()))
Then define the p,d,q and P,D,Q,s to take any value between 0 and 2
p = d = q = range(0,2)# Generate all different combinations of p, d and q tripletspdq = list(itertools.product(p,d,q))# Generate all different combinations of seasonal p, d and q tripletspdqs = [(x[0], x[1], x[2], 12) for x in list(itertools.product(p, d, q))]
The SARIMA model
ans = []for df, name in zip(zip_dfs, zip_list):    for para1 in pdq:        for para2 in pdqs:            try:                mod = sm.tsa.statespace.SARIMAX(df,                                                order = para1,                                                seasonal_order = para2,                                                enforce_stationarity = False,                                                enforce_invertibility = False)                output = mod.fit()                ans.append([name, para1, para2, output.aic])                print('Result for {}'.format(name) + ' ARIMA {} x {}12 : AIC Calculated = {}'.format(para1, para2, output.aic))            except:                continue
Then store all results to a data frame
result = pd.DataFrame(ans, columns = ['name','pdq','pdqs','AIC'])
Sort by lowest AIC to find the best parameters for each zip code
best_para = result.loc[result.groupby("name")["AIC"].idxmin()]
#Make Prediction and compare with real valuessummary_table = pd.DataFrame()Zipcode = []MSE_Value = []models = []for name, pdq, pdqs, df in zip(best_para['name'], best_para['pdq'], best_para['pdqs'], zip_dfs):ARIMA_MODEL = sm.tsa.SARIMAX(df,                                 order = pdq,                                 seasonal_order = pdqs,                                 enforce_stationarity = False,                                 enforce_invertibility = False,                                 )                                        output = ARIMA_MODEL.fit()    models.append(output)        #get dynamic predictions starting 2017-06-01    pred_dynamic = output.get_prediction(start=pd.to_datetime('2017-06-01'), dynamic = True, full_results = True)    pred_dynamic_conf = pred_dynamic.conf_int()    zip_forecasted = pred_dynamic.predicted_mean    zip_truth = df['2017-06-01':]['MeanValue']        sqrt_mse = np.sqrt(((zip_forecasted - zip_truth)**2).mean())        Zipcode.append(name)    MSE_Value.append(sqrt_mse)    summary_table['Zipcode'] = Zipcodesummary_table['Sqrt_MSE'] = MSE_Value
Next step will be to use the full data set to predict future values. I used the 3 years as an example.
#Final Model forecast_table = pd.DataFrame()current = []forecast_3Yr = []for zipcode, output, df in zip(Zipcode, models, zip_dfs):    pred_3 = output.get_forecast(steps = 36)    pred_conf_3 = pred_3.conf_int()    forecast_3 = pred_3.predicted_mean.to_numpy()[-1]    current.append(df['2018-04']['MeanValue'][0])    forecast_3Yr.append(forecast_3)    forecast_table['Zipcode'] = Zipcodeforecast_table['Current Value'] = currentforecast_table['3 Years Value'] = forecast_3Yrforecast_table['3Yr-ROI']=(forecast_table['3 Years Value'] - forecast_table['Current Value'])/forecast_table['Current Value']
This is my final result:
11220: South Sunset Park (3Yr ROI: 17%-87%)
11205: Clinton Hill (3Yr ROI: 16%-78%)
11203: East Flatbush (3Yr ROI: 8%-78%)
11224: Coney Island (3Yr ROI: -0.5%-76%)
11217: Boerum Hill (3Yr ROI: 6%-61%)
This model is purely based on time series so the predictions may not be very accurate because apparently there are many other factors that impact the house price such as economics, interest rate, house market safety score, etc.
A linear model would be more ideal if we want to take other factors into consideration.
Thanks for reading! Let me know your thoughts.
Useful Resources:
Using Python and Auto ARIMA to Forecast Seasonal Time Series
Everything you need to know about Time Series