Common Time Series Data Analysis Methods and Forecasting Models in Python | by Yuefeng Zhang, PhD | Towards Data Science

A time series is a sequence of data samples taken in time order with equal time intervals. Time series include many kinds of real experimental data taken from various domains such as finance, medicine, scientific research (e.g., global warming, speech analysis, earthquakes), etc. [1][2]. Time series forecasting has many real applications in various areas such as forecasting of business (e.g., sales, stock), weather, decease, and others [1].
Given a traditional (time order independent) dataset for supervised machine learning for prediction, data exploration and preprocessing are required before features engineering can be performed, and the features engineering needs to be done before a machine learning model can be chosen and applied to the engineered features for prediction.
Similarly to traditional dataset, given a time series dataset, data exploration and preprocessing are required before the time series data can be analyzed, and time series data analysis is required before a time series forecasting model can be chosen and applied to the analyzed dataset for forecasting.
In this article, I use a global warming dataset from Kaggle [2] to demonstrate some of the common time series data preprocessing/analysis methods and time series forecasting models in Python. The demonstration consists of the following:
Time series data preprocessing
Time series data analysis
Time series forecasting
As described before, for a time series data, data preprocessing is required before data analysis can be performed.
The first step towards data preprocessing is to load data from a csv file.
Time order plays a critical role in time series data analysis and forecasting. In particular, each data sample in a time series must be associated with a unique point in time. This can be achieved in Pandas DataFrame/Series by using values of DatetimeIndex type as its index values.
Once the earth surface temperature dataset in Kaggle [2] has been downloaded onto a local machine, the dataset csv file can be loaded into a Pandas DataFrame as follows:
df_raw = pd.read_csv('./data/GlobalTemperatures.csv', parse_dates=['dt'], index_col='dt')df_raw.head()
The option parse_dates is to tell Pandas to parse the string values in the dt column into Python datatime values, while the option index_col is to tell Pandas to convert the parsed values of the dt column into DatatimeIndex type and then use them as indices.
For simplicity, I extract the LandAverageTemperature column as a Pandas Series for demonstration purpose in this article:
df = df_raw['LandAverageTemperature']
Similarly to a traditional dataset, missing data frequently occurs in time series data, which must be handled before the data can be further preprocessed and analyzed.
The following code can check how many data entries are missing:
df.isnull().value_counts()
There are 12 missing data entries in the earth surface temperature time series. Those missing values cannot simply be removed or set to zero without breaking the past time dependency. There are multiple ways of handling missing data in time series appropriately [3]:
Forward fill
Backward fill
Linear interpolation
Quadratic interpolation
Mean of nearest neighbors
Mean of seasonal couterparts
I used forward fill to fill up the missing data entries in this article:
df = df.ffill()
Once the data preprocessing is done, the next step is to analyze data.
As a common practice [1][3][4], the first step towards time series data analysis is to visualize the data.
The code below uses the Pandas DataFrame/Series built-in plot method to plot the earth surface temperature time series:
ax = df.plot(figsize=(16,5), title='Earth Surface Temperature')ax.set_xlabel("Date")ax.set_ylabel("Temperature")
The above plot shows that the average temperature of the earth’s surface is around the range of [5, 12] and the overall trend is increasing slowly. No other obvious patterns show up in the plot due to mixture of different time series components such as base level, trend, seasonality, and other components such as error and random noise [1][3]. The time series can be decomposed into individual components for further analysis.
To decompose a time series into components for further analysis, the time series can be modeled as an additive or multiplicative of base level, trend, seasonality, and error (including random noise) [3]:
Additive time series:value = base level + trend + seasonality + error
Multiplicative time series:value = base level x trend x seasonality x error
The earth surface temperature time series is modeled as an additive time series in this article:
additive = seasonal_decompose(df, model='additive', extrapolate_trend='freq')
The option extrapolate_trend='freq' is to handle any missing values in the trend and residuals at the beginning of the time series [3].
In theory, the same dataset can be easily modeled as a multiplicative time series by replacing the option model=’additive’ with model=’multiplicative’. However, the multiplicative model cannot be applied to this particular dataset because the dataset contains zero and/or negative values, which are not allowed in multiplicative seasonality decomposition.
The resulting components of the additive decomposition can be extracted to form a Pandas DataFrame as follows:
additive_df = pd.concat([additive.seasonal, additive.trend, additive.resid, additive.observed], axis=1)additive_df.columns = ['seasonal', 'trend', 'resid', 'actual_values']additive_df.head()
The code below is to visualize the additive decomposed components: trend, seasonal, and residual (i.e., base level + error).
plt.rcParams.update({'figure.figsize': (10,10)})additive.plot().suptitle('Additive Decompose')
For the earth surface temperature time series data, we are interested in its long term trend the most, which can be extracted as follows:
trend = additive.trend
Once the data preprocessing and analysis are done, time series forecasting can begin.
This section presents the results of applying two common time series forecasting models to the earth surface temperature trend data:
ARIMA (AutoRegressive Integrated Moving Average)
LSTM (Long Short-Term Memory)
An ARIMA model [1][4] is determined by three parameters:
p: the autoregressive order
d: the order of differencing to make the time series stationary
q: the moving average order
ARIMA model consists of three parts [4]: AutoRegression (AR), Moving Average (MA), and a constant:
ARIMA = constant + AR + MA
where
AR = a linear combination of p consecutive values in the past time points (i.e., lags)
MA = a linear combination of q consecutive forecast errors in the past time points (i.e., lagged forecast errors)
Both AR and MA can only be applied to a stationary time series, which is achieved by differencing in ARIMA.
3.1.1 Determining the order of differencing d
A time series is (weakly) stationary if its mean is constant (independent of time) and its autocovariance function between two different time points s and t of the time series only depends on the time interval |s - t| (i.e., lag), not a specific time point [1].
Time series forecasting works only for a stationary time series since only the behavior of a stationary time series is predictable.
We can use the ADF test (Augmented Dickey Fuller test) [4] to check whether or not a time series is stationary. As an example, the code below is to check the earth surface temperature time series for stationarity:
from statsmodels.tsa.stattools import adfullerresult = adfuller(trend.values)print('ADF Statistic: %f' % result[0])print('p-value: %f' % result[1])
The p-value of the test is 0.012992.
The default null hypothesis of the ADF test is that the time series is non-stationary. Since the p-value of the ADF test above is less than the significance level of 0.05, we reject the null hypothesis and conclude that the time series is stationary (only trend stationary in this case).
Generally speaking, the following need to be done to make a time series stationary:
remove non-regular behaviors that can change mean and/or covariance over time
remove regular behaviors such as trend and seasonality that can change mean and/or covariance over time
Differencing is a popular data transform method for removing non-stationary behaviors (especially trend).
The following code is to perform 1st and 2nd order of differencing against the earth surface temperature time series:
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf# Original Seriesfig, axes = plt.subplots(3, 2, sharex=True)axes[0, 0].plot(trend.values); axes[0, 0].set_title('Original Series')plot_acf(trend.values, ax=axes[0, 1]).suptitle('Original Series', fontsize=0)# 1st Differencingdiff1 = trend.diff().dropna()axes[1, 0].plot(diff1.values)axes[1, 0].set_title('1st Order Differencing')plot_acf(diff1.values, ax=axes[1, 1]).suptitle('1st Order Differencing', fontsize=0)# 2nd Differencingdiff2 = trend.diff().diff().dropna()axes[2, 0].plot(diff2.values)axes[2, 0].set_title('2nd Order Differencing')plot_acf(diff2.values, ax=axes[2, 1]).suptitle('2nd Order Differencing', fontsize=0)
The figure below shows that 1st order of differencing is enough to remove the trend. 2nd order of differencing does not make any improvement. Thus the order of differencing d is set to 1 in this article.
3.1.2 Determining the autoregressive order p
The autoregressive order p can be determined by analyzing the results of PACF (Partial Autocorrelation Function) on the 1st order of differencing of the time series data [1][4]:
plt.rcParams.update({'figure.figsize':(9,3), 'figure.dpi':120})size = 100fig, axes = plt.subplots(1, 2, sharex=True)axes[0].plot(diff1.values[:size])axes[0].set_title('1st Order Differencing')axes[1].set(ylim=(0,5))plot_pacf(diff1.values[:size], lags=50, ax=axes[1]).suptitle('1st Order Differencing', fontsize=0)
We can observe that the PACF lag 1 is well above the significance line (gray area). Thus the autoregressive order p is set to 1 in this article.
3.1.3 Determining the moving average order q
The moving average order q can be determined by analyzing the results of ACF (Autocorrelation Function) on the 1st order of differencing of the time series data [1][4]:
plt.rcParams.update({'figure.figsize':(9,3), 'figure.dpi':120})size = 100fig, axes = plt.subplots(1, 2, sharex=True)axes[0].plot(diff1.values[:size])axes[0].set_title('1st Order Differencing')axes[1].set(ylim=(0,1.2))plot_acf(diff1.values[:size], lags=50, ax=axes[1]).suptitle('1st Order Differencing', fontsize=0)
We can observe that the ACF lag 1 is well above the significance line (gray area). Thus the moving average order q is set to 1 as well in this article.
3.1.4 Training ARIMA model
The following code divides the earth surface temperature trend time series into training and testing sub-series first and then uses the training data to train an ARIMA model with the determined values of p =1, d =1, q = 1.
A traditional dataset is typically randomly divided into training and testing subsets. However, this does not work for time series because it breaks the sequential time dependency. To avoid this problem, the temperature trend time series data is divided by keeping its original sequential order.
from statsmodels.tsa.arima_model import ARIMAtrain = trend[:3000]test  = trend[3000:]# order = (p=1, d=1, q=1)model = ARIMA(train, order=(1, 1, 1))  model = model.fit(disp=0)  print(model.summary())
It can be seen from the above model training results that the P Values of the AR1 and MA1 in the p>|z| column are highly significant (<< 0.05). This indicates that the choices of p =1 and q =1 are appropriate.
The code below is to plot the residuals.
# Plot residual errorsresiduals = pd.DataFrame(model.resid)fig, ax = plt.subplots(1,2)residuals.plot(title="Residuals", ax=ax[0])residuals.plot(kind='kde', title='Density', ax=ax[1])
The plot of the residuals shows no patterns (i.e., with constant mean and variance) except for the first 20% of the time series. This indicates that the trained ARIMA model behaves appropriately.
3.1.5 Forecasting using the trained ARIMA model
The code below is to use the trained ARIMA model to forecast 192 (this can be any positive integer) temperature values and then compare them with the testing time series:
# Forecast: 192 forecasting values with 95% confidencefc, se, conf = model.forecast(192, alpha=0.05)# Make as pandas seriesfc_series = pd.Series(fc, index=test.index)lower_series = pd.Series(conf[:, 0], index=test.index)upper_series = pd.Series(conf[:, 1], index=test.index)# Plotplt.figure(figsize=(12,5), dpi=100)plt.plot(train, label='training')plt.plot(test, label='actual')plt.plot(fc_series, label='forecast')plt.fill_between(lower_series.index, lower_series, upper_series,                  color='k', alpha=.15)plt.title('Forecast vs Actuals')plt.legend(loc='upper left', fontsize=8)
The forecasting results above show that the trained ARIMA model tends to forecast temperatures below the actual ones.
This section presents the results of applying the well-known LSTM model to the earth surface temperature trend time series.
3.2.1 Preparing dataset
Similarly to [6], the following code is to generate pairs of feature vector (a sequence of temperature values in the past time points) and label (target temperature at current time point) from the temperature time series for LSTM model training and evaluation.
from numpy import arrayfrom keras.models import Sequentialfrom keras.layers import LSTMfrom keras.layers import Densedef split_sequence(sequence, n_steps):    X, y = list(), list()    for i in range(len(sequence)):        # find the end of this pattern        end_ix = i + n_steps        # check if we are beyond the sequence        if end_ix > len(sequence)-1:            break        # gather input and output parts of the pattern        seq_x, seq_y = sequence[i:end_ix], sequence[end_ix]        X.append(seq_x)        y.append(seq_y)    return array(X), array(y)# define input sequenceraw_seq = trend.tolist()# choose a number of time stepsn_steps = 12# split into samplesX, y = split_sequence(raw_seq, n_steps)
For simplicity, I used the temperatures in the past 12 months to predict the temperature in next month in this article. The following are two samples of the generated dataset:
The generated dataset is divided into two parts: the first 3,000 for model training and the rest of dataset for model testing:
X_train = X[:3000]y_train = y[:3000]X_test  = X[3000:]y_test  = y[3000:]
3.2.2 Selecting LSTM model
The following LSTM model [6] takes a sequence of temperature values as input and generates one target temperate as output. Since the temperature forecasting is a regression issue, the output of the LSTM model can take any value and thus there is no associated activation function.
n_features = 1X_train = X_train.reshape((X_train.shape[0], X_train.shape[1], n_features))# define modelmodel = Sequential()model.add(LSTM(50, activation='relu', input_shape=(n_steps, n_features)))model.add(Dense(1))model.compile(optimizer='adam', loss='mse')# fit modelmodel.fit(X_train, y_train, epochs=200, verbose=1)
3.2.3 Training LSTM model
The following are the results of model training:
3.2.4 Forecasting using the trained LSTM model
Once the model training is done, the trained LSTM model can then be applied to the testing time series to forecast temperatures:
X_test = X_test.reshape((X_test.shape[0], X_test.shape[1], n_features))y_pred = model.predict(X_test, verbose=0)
The code below plots the predicted temperatures against the actual temperatures in the testing time series:
def plot_forecosting(df1, df2, line_stype1='b-', line_stype2='r--', title="", xlabel='Date', ylabel='Temperature', dpi=100):    plt.figure(figsize=(16,5), dpi=dpi)    plt.plot(df1.index, df1, line_stype1, label='actual')     plt.plot(df2.index, df2, line_stype2, label='forecast')    plt.gca().set(title=title, xlabel=xlabel, ylabel=ylabel)    plt.title('Forecast vs Actuals')    plt.legend(loc='upper left', fontsize=8)    plt.show()y_pred_1 = y_pred.reshape((y_pred.shape[0]))y_pred_series = pd.Series(y_pred_1)y_test_1 = y_test.reshape((y_test.shape[0]))y_test_series = pd.Series(y_test_1)plot_forecosting(y_test_series, y_pred_series, title='Land Average Temperature')
The plot of the predicted temperatures against the entire temperature trend time series is done as follows:
X_all = X.reshape((X.shape[0], X.shape[1], n_features))y_pred_all = model.predict(X_all, verbose=0)y_pred_all_1 = y_pred_all.reshape((y_pred_all.shape[0]))y_pred_all_series = pd.Series(y_pred_all_1)y_all = y.reshape((y.shape[0]))y_all_series = pd.Series(y_all)plot_forecosting(y_all_series, y_pred_all_series, title='Land Average Temperature')
The forecasting results above shows that the forecasted temperatures followed the actual temperatures closely.
In this article, I used a global warming dataset from Kaggle [2] to demonstrate some of the common time series data preprocessing/analysis practices and two widely adopted time series forecasting models ARIMA and LSTM in Python.
As can be seen in Section 3, the performance of the ARIMA model relies heavily on data preprocessing and analysis to make a time series stationary, while LSTM can work on a time series with minimum data preprocessing and analysis (e.g., no need to remove trend by differencing in LSTM model).
All of the source code used in this article is available in Github [7].
R. Shumway and D. Stoffer, Time Series Analysis and Its Applications, Springer, 4th Edition, 2017Climate Change: Earth Surface Temperature DataS. Prabhakaran, Time Series Analysis in Python — A Comprehensive Guide with ExamplesS. Prabhakaran, ARIMA Model — Complete Guide to Time Series Forecasting in PythonJ. Brownlee, How to Remove Trends and Seasonality with a Difference Transform in PythonJ. Brownlee, How to Develop LSTM Models for Time Series ForecastingY. Zhang, Jupyter notebook in Github
R. Shumway and D. Stoffer, Time Series Analysis and Its Applications, Springer, 4th Edition, 2017
Climate Change: Earth Surface Temperature Data
S. Prabhakaran, Time Series Analysis in Python — A Comprehensive Guide with Examples
S. Prabhakaran, ARIMA Model — Complete Guide to Time Series Forecasting in Python
J. Brownlee, How to Remove Trends and Seasonality with a Difference Transform in Python
J. Brownlee, How to Develop LSTM Models for Time Series Forecasting
Y. Zhang, Jupyter notebook in Github