Preprocessing Time Series Data for Supervised Machine Learning
By: Sijuade Oguntayo
Tags: Data Preprocess, Machine Learning, Time Series Analysis, Tsextract, Solar Energy

Build Features

pip install tsextract

tsExtract

The code above imports some functions from tsExtract that we’ll find useful to preprocess the data for supervised learning. The features_request defines the features we wish to build from the time-series data —

window — We take a window of size 48. Since the observations are recorded at 30-minute intervals, this is a window of one day. This can typically be chosen by looking at repeating patterns in the data, or by taking a grid-search approach and trying out different values of window sizes for which works best.

window_statistic — This function also takes a sliding window of size 48, a mean operation is then applied to collapse the data to 1-d. This is especially useful when all we want is simply a statistical summary of the windowed data rather than the raw values.

difference_statistic — The first value also refers to window size, the second value refers to the order of differencing to be applied while the third input is the statistic we wish to apply. To summarize, this takes a window of size 48, applies first-order differencing to that window, and then collapses the matrix to 1d using standard deviation along the second axis.

The arguments passed into the build_features function are the data to preprocess and the features_request dictionary defining the features to build.

Also passed in is the number of time steps in the future we wish to predict. A specified target_lag of 48 means that we would be training the model to predict 24 hours into the future.

Finally, we specify whether or not we want t_zero. t_zero refers to the present time. This is from the moment we both look back for explanatory variables and look forward to the target variable.

Another benefit to having t_zero is that this may be used to calculate the difference between the target variable and t_zero. This variable could then be used as the new target variable. In this situation, we would be training our model to predict the difference between the energy output at the present time and the target time as opposed to predicting the raw energy values.

build_df output

Scaling and train-test split

We apply standard scaler and split the data 70–30 into train and test set. This standardizes the features to have 0 mean and unit variance.

Modeling with Keras

We set up a two-layer NN architecture using Keras with dropout to reduce overfitting. The model is trained with a batch size of 32 and an adam optimizer with MSE as the loss function. Training is done for 100 epochs with test loss —

493/493 [==============================] - 0s 781us/step - loss: 0.1598

Actual & Prediction Line Plot

A line plot of both the true and predicted values shows similar movements.

Actual vs Prediction Scatter Plot

The scatterplot shows a positive correlation between the actual and predicted values. This could be tighter.

Lagged Correlation

With time series, we don’t only consider error metrics like MSE, MAE, etc. This is because such metrics don’t capture lag. To get a better picture of how well our model is performing, we also look at Lag Correlation (Cross-Correlation) between the true and predicted values.

A stock price predictor model that correctly predicts the general movement of a stock price for example, but predicts sharp movements much later than the actual values would not be a useful model since it would lead to missed opportunities.

The plot below shows an example of this. A model that produces the results below might return a relatively low error. Such a model, however, would show a high lag correlation coefficient at a lag of 5.

Example — Lag Correlation

To check for this, we can compute the lagged correlation between the true and predicted values. Specifically, we’ll be using Spearman’s correlation coefficient. This is calculated as —

Lag Correlation

where —

∂ represents the lag value

cov() represents the covariance of two vectors

σ — standard deviation

y(t+∂) is the vector y shifted by ∂ time periods

Lag Correlation Coefficient

We notice a high correlation coefficient at lag 0 which is ideal. We also notice a gradual reduction as the lag value increases. Here, we would ideally like to see a sharp drop after lag 0 as this would signify high confidence. More could be done to improve on this model.