Single and Multi-Step Temperature Time Series Forecasting for Vilnius Using LSTM Deep Learning Model | by Eligijus Bujokas | Towards Data Science

The aim of this article is to provide code examples and explain the intuition behind modeling time series data using python and TensorFlow.
All the code and data is public and can be seen in the following repository:
github.com
This article is a generalization of one of my previous articles:
towardsdatascience.com
The article above explains how to forecast time-series data for a single step forward. This article shows how to do multi-step forecasting and use more than 1 feature in the model.
The very brief version of this article is that using the past 48 hours of data and forecasting 1 hour into the future (single step) I have achieved a mean absolute error of 0.48 (0.34 median) degrees of Celcius error.
Using the past 168 hours of data and forecasting 24 hours ahead, the mean absolute error was 1.69 (1.27 median) degrees of Celcius.
The features used were the hourly past data of temperature, the daily and yearly cyclical signals, pressure, and wind speed.
The data is obtained using an API from https://openweathermap.org/. The data was gathered every hour from 1990.01.01 to 2020.11.30 near the TV tower of Vilnius. Vilnius is not a large metropolis and the TV tower is in the city itself so the temperature that is near the tower should be very similar to what is the temperature in all parts of the city.
Here and throughout the article the main data object is called d. It is created by reading the raw data:
d = pd.read_csv(‘data/weather.csv’)# Converting the dt column to datetime object d[‘dt’] = [datetime.datetime.utcfromtimestamp(x) for x in d[‘dt’]]# Sorting by the date d.sort_values(‘dt’, inplace=True)
There are 271008 total data points in the dataset.
The data seems to be stationary with a clear cyclical pattern.
The graph above shows that there is a clear day cycle of temperatures — the median temperature is the highest around noon and the lowest around midnight.
The cyclical pattern is even more evident in the temperatures grouped by month — the hottest months are June up to August and the coldest ones are December to February.
The problem with the data as it is now, we only have the date column. If would convert it to numeric (let us say, extract the timestamp in seconds) and add it as a feature in modeling the cyclical nature would be lost. Thus, the first thing we need to do is to engineer some features that catch the cyclical trend.
We want the machine to know that hours 23 and 0 are closer to each other than hours 0 and 4. We know that the period of the cycle is 24 hours. We can use the cos(x) and sin(x) functions. The x in the functions is the hour of the day.
# Extracting the hour of dayd["hour"] = [x.hour for x in d["dt"]]# Creating the cyclical daily feature d["day_cos"] = [np.cos(x * (2 * np.pi / 24)) for x in d["hour"]]d["day_sin"] = [np.sin(x * (2 * np.pi / 24)) for x in d["hour"]]
The head of the resulting data frame:
The newly created features catch the cyclical pattern. The question may arise, why do we use both sin and cos functions?
Drawing a horizontal line in the above graph and analyzing just one of the curves we would get that, for example, cos(7.5h) = cos(17.5h) and so on. This may lead to some errors when learning and predicting so in order to make every point unique, we add another cyclical function. Using the two features together, all times can be distinguished from each other.
To create the same cyclical logic for the time of the year, we will use the timestamp functionality. Timestamp in python is a value that calculates how many seconds have passed since 1970.01.01 0H:0m:0s. Every date object in python has the function timestamp().
# Extracting the timestamp from the datetime object d["timestamp"] = [x.timestamp() for x in d["dt"]]# Seconds in day s = 24 * 60 * 60# Seconds in year year = (365.25) * sd["month_cos"] = [np.cos((x) * (2 * np.pi / year)) for x in d["timestamp"]]d["month_sin"] = [np.sin((x) * (2 * np.pi / year)) for x in d["timestamp"]]
In this section, we have created 4 additional features from the datetime column: day_sin, day_cos, month_sin and month_cos.
In the weather dataset, there are two additional columns: wind_speed and pressure. Wind speed is measured in meters per second (m/s) and the pressure is measured in hectopascals (hPa).
To see any relationship between the temperature and the two features we can plot the 2d histograms:
The more intense the color the bigger the relationship in certain bin values of the two distributions. For example, the temperatures tend to be higher when the pressure is around 1010 and 1020 hPa.
We will use these two features in modeling as well.
The data that we have with all the feature engineering is this:
The function f that we want to approximate is:
The goal is to use past values to predict the future. The data is a time series or a sequence. For sequence modeling, we will choose tensorflow implementation of recurrent neural network with an LSTM layer.
The input to an LSTM network is a 3D array:
(samples, timesteps, features)
samples — total number of sequences constructed for training.
timesteps — the length of the samples.
features — number of features used.
The first thing before modeling is from the data that is in a 2D format convert into a 3D array. The following function does that:
For example, if we assume that the whole data is the first 10 rows of the data, we use 3 past hours as features and we want to forecast 1 step ahead:
ts = d[‘temp’, ‘day_cos’, ‘day_sin’, ‘month_sin’, ‘month_cos’, ‘pressure’, ‘wind_speed’].head(10).valuesX, Y = create_X_Y(ts, lag=3, n_ahead=1)
As we can see, the shape of the X matrix is 6 samples, 3 timesteps and 7 features. In other words, we have 6 observations each with 3 rows of data and 7 columns. There are 6 observations because the first 3 lags are dropped and used only as X data and we are forecasting 1 step ahead thus the last observation is lost as well.
The first value pairs of X and Y are presented in the above picture.
The hyperparameter list for the final model:
# Number of lags (hours back) to use for modelslag = 48# Steps ahead to forecast n_ahead = 1# Share of obs in testing test_share = 0.1# Epochs for trainingepochs = 20# Batch size batch_size = 512# Learning ratelr = 0.001# Number of neurons in LSTM layern_layer = 10# The features used in the modeling features_final = [‘temp’, ‘day_cos’, ‘day_sin’, ‘month_sin’, ‘month_cos’, ‘pressure’, ‘wind_speed’]
The model class:
The last step before creating all the matrices for the modeling is to scale the data.
# Subseting only the needed columns ts = d[features_final]nrows = ts.shape[0]# Spliting into train and test setstrain = ts[0:int(nrows * (1 — test_share))]test = ts[int(nrows * (1 — test_share)):]# Scaling the data train_mean = train.mean()train_std = train.std()train = (train — train_mean) / train_stdtest = (test — train_mean) / train_std# Creating the final scaled frame ts_s = pd.concat([train, test])# Creating the X and Y for trainingX, Y = create_X_Y(ts_s.values, lag=lag, n_ahead=n_ahead)n_ft = X.shape[2]
Now we split the matrices into training and validation matrices
# Spliting into train and test sets Xtrain, Ytrain = X[0:int(X.shape[0] * (1 — test_share))], Y[0:int(X.shape[0] * (1 — test_share))]Xval, Yval = X[int(X.shape[0] * (1 — test_share)):], Y[int(X.shape[0] * (1 — test_share)):]
The final shapes of the matrices:
Shape of training data: (243863, 48, 7)Shape of the target data: (243863, 1)Shape of validation data: (27096, 48, 7)Shape of the validation target data: (27096, 1)
All that is left is to create the object using the model class, train the model, and inspect the results on the validation set.
# Initiating the model objectmodel = NNMultistepModel( X=Xtrain, Y=Ytrain, n_outputs=n_ahead, n_lag=lag, n_ft=n_ft, n_layer=n_layer, batch=batch_size, epochs=epochs,  lr=lr, Xval=Xval, Yval=Yval,)# Training of the model history = model.train()
With the trained model we can forecast and compare the values with the original ones.
# Comparing the forecasts with the actual valuesyhat = [x[0] for x in model.predict(Xval)]y = [y[0] for y in Yval]# Creating the frame to store both predictionsdays = d[‘dt’].values[-len(y):]frame = pd.concat([ pd.DataFrame({‘day’: days, ‘temp’: y, ‘type’: ‘original’}), pd.DataFrame({‘day’: days, ‘temp’: yhat, ‘type’: ‘forecast’})])# Creating the unscaled values columnframe[‘temp_absolute’] = [(x * train_std[‘temp’]) + train_mean[‘temp’] for x in frame[‘temp’]]# Pivotingpivoted = frame.pivot_table(index=’day’, columns=’type’)pivoted.columns = [‘_’.join(x).strip() for x in pivoted.columns.values]pivoted[‘res’] = pivoted[‘temp_absolute_original’] — pivoted[‘temp_absolute_forecast’]pivoted[‘res_abs’] = [abs(x) for x in pivoted[‘res’]]
Plotting the results:
plt.figure(figsize=(12, 12))plt.plot(pivoted.index, pivoted.temp_absolute_original, color=’blue’, label=’original’)plt.plot(pivoted.index, pivoted.temp_absolute_forecast, color=’red’, label=’forecast’, alpha=0.6)plt.title(‘Temperature forecasts — absolute data’)plt.legend()plt.show()
The two lines are very close to each other, they basically overlap. The distribution of absolute errors:
The median absolute error is 0.34 degrees Celcius and the average is 0.48 degrees Celcius.
To predict 24 hours ahead the only thing needed is to change the hyperparameters. Specifically, the n_ahead variable. The model will try to predict the next 24 hour values using 168 hours from before (one week).
# Number of lags (hours back) to use for modelslag = 168# Steps ahead to forecast n_ahead = 24# Share of obs in testing test_share = 0.1# Epochs for trainingepochs = 20# Batch size batch_size = 512# Learning ratelr = 0.001# Number of neurons in LSTM layern_layer = 10# Creating the X and Y for trainingX, Y = create_X_Y(ts_s.values, lag=lag, n_ahead=n_ahead)n_ft = X.shape[2]Xtrain, Ytrain = X[0:int(X.shape[0] * (1 - test_share))], Y[0:int(X.shape[0] * (1 - test_share))]Xval, Yval = X[int(X.shape[0] * (1 - test_share)):], Y[int(X.shape[0] * (1 - test_share)):]# Creating the model object model = NNMultistepModel(    X=Xtrain,    Y=Ytrain,    n_outputs=n_ahead,    n_lag=lag,    n_ft=n_ft,    n_layer=n_layer,    batch=batch_size,    epochs=epochs,     lr=lr,    Xval=Xval,    Yval=Yval,)# Training the model history = model.train()
Inspecting some random chunks of 24 hours chunks:
Some 24 hours sequences seem to be close to one another while others are not.
The mean absolute error is 1.69 C and the median is 1.27 C.
In conclusion, this article presented a simple pipeline example when working with modeling and forecasting of the time series data:
Reading, cleaning, and augmenting the input data
Selecting the hyperparameters for the lag and n steps ahead
Selecting the hyperparameters for the deep learning model
Initiating the NNMultistepModel() class
Fitting the model
Forecasting n_steps ahead
I hope that the reader can use the code showcased in this article in his/her professional and academic work.