Time series transformations selection using EDAspy
By: Vicente P. Soloviev
Tags: Python, Time Series Forecasting, Python Packages, Time Series Analysis, Feature Engineering

Time series transformations selection using EDAspy

Example of optimisation in forecasting model

In Machine Learning (ML) projects, when we work with time series (TS)in the project we usually look for the ideal time series transformations in order to improve the implemented model performance.

This can be a tedious task if we work with a large dataset, and we want to try lots of different time series transformations. I have implemented an approach that can efficiently manage this task.

Estimation of Distribution Algorithms (EDAs) are a type of evolutionary algorithms that reproduce the next generation using a probabilistic model based on the best individuals selected in the previous generation. Some different EDAs implementations are implemented in the Python package EDAspy (https://github.com/VicentePerezSoloviev/EDAspy ; https://pypi.org/project/EDAspy/). To install the package just do:

pip install EDAspy

A very easy example is shown below. Note that for such an easy example the improvement obtained is very small compared with the improvement we could obtain if a larger dataset is used. Some easy time series transformations are implemented, but feel free to try more TS transformations.

First we load the needed libraries:

import pandas as pd

import statsmodels.api as sm

from statsmodels.tsa.api import VAR

import matplotlib.pyplot as plt

from sklearn.metrics import mean_absolute_error



# EDAspy libraries

from EDAspy.timeseries import EDA_ts_fts as EDA

from EDAspy.timeseries import TS_transformations

Then we use a little public dataset to use as an example (available in Pandas library). Visualize the data:

mdata = sm.datasets.macrodata.load_pandas().data

df = mdata.iloc[:, 2:12]

df.head()

Image by author

We list the variable without the variable we want to forecast (‘pop’), and build the dataset with the time series transformations of the rest of variables. More transformations can be added following the steps:

- Add the transformation postfix

- Add to the dataset the respective variable with name (name + postfix)

But some available time series transformations are available in TSTransformations.

variables = list(df.columns)

variable_y = 'pop' # pop is the variable we want to forecast

variables = list(set(variables) - {variable_y}) TSTransf = TSTransformations(df)

transformations = ['detrend', 'smooth', 'log'] # postfix to variables, to denote the transformation



# build the transformations

for var in variables:

transformation = TSTransf.de_trending(var)

df[var + 'detrend'] = transformation



for var in variables:

transformation = TSTransf.smoothing(var, window=10)

df[var + 'smooth'] = transformation



for var in variables:

transformation = TSTransf.log(var)

df[var + 'log'] = transformation

We must define a cost function. In this case is the following with hyper-parameters. The cost function returns a MAE and inputs a list of variables from the built dataset (with time series transformations):

def cost_function(variables_list, nobs=20, maxlags=15, forecastings=10):

"""

variables_list: list of variables without the variable_y

nobs: how many observations for validation

maxlags: previous lags used to predict

forecasting: number of observations to predict



return: MAE of the prediction with the real validation data

"""



data = df[variables_list + [variable_y]]



df_train, df_test = data[0:-nobs], data[-nobs:]



model = VAR(df_train)

results = model.fit(maxlags=maxlags, ic='aic')



lag_order = results.k_ar

array = results.forecast(df_train.values[-lag_order:], forecastings)



variables_ = list(data.columns)

position = variables_.index(variable_y)



validation = [array[i][position] for i in range(len(array))]

mae = mean_absolute_error(validation, df_test['pop'][-forecastings:])



return mae

We take the normal variables without any time series transformation and try to forecast the y variable using the same cost function defined. This value is stored to be compared with the optimum solution found.

mae_pre_eda = cost_function(variables)

print('MAE without using EDA:', mae_pre_eda)

# MAE without using EDA: 5.091478009948458

Initialization of the initial vector of statistics. Each variable has a 50% probability to be or not chosen

vector = pd.DataFrame(columns=list(variables))

vector.loc[0] = 0.5

Run the algorithm. The code will print some further information during execution

eda = EDA(max_it=50, dead_it=5, size_gen=15, alpha=0.7, vector=vector,

array_transformations=transformations, cost_function=cost_function)

best_ind, best_MAE = eda.run(output=True)

EDA execution. We see how along the iterations the algorithm decreases the optimized MAE of the forecasting model. Image by author

The results are the following. In the left side the best local costs, and in the right side the best global cost find until the respective iteration. To plot the results:

hist = eda.historic_best



relative_plot = []

mx = 999999999

for i in range(len(hist)):

if hist[i] < mx:

mx = hist[i]

relative_plot.append(mx)

else:

relative_plot.append(mx)



print('Solution:', best_ind, '

MAE post EDA: %.2f' % best_MAE, '

MAE pre EDA: %.2f' % mae_pre_eda)



plt.figure(figsize = (14,6))



ax = plt.subplot(121)

ax.plot(list(range(len(hist))), hist)

ax.title.set_text('Local cost found')

ax.set_xlabel('iteration')

ax.set_ylabel('MAE')



ax = plt.subplot(122)

ax.plot(list(range(len(relative_plot))), relative_plot)

ax.title.set_text('Best global cost found')

ax.set_xlabel('iteration')

ax.set_ylabel('MAE')



plt.show()

Image by author

Hope that this is useful for your future projects. Future medium posts will share more real examples showing how to use EDAspy functionalities. Feel free to look for more examples in the notebooks section of the package (https://github.com/VicentePerezSoloviev/EDAspy/tree/master/notebooks)