Getting started with giotto-time
By: Francesco Palma
Tags: Causality, Python, Time Series Analysis, Energy Demand, Forecasting

Generating accurate time series forecasts is essential in many business and research contexts. At L2F we found that there was limited tooling available to generate these forecasts in Python and the PyData stack. Although popular libraries like statsmodels and Facebook’s Prophet work well for certain applications, it was needed a flexible framework that would allow to combine tens, possibly hundreds of models and stack thousands of features.

To that end, we have open-sourced our time series tooling with the Python library giotto-time. The library is designed for large-scale machine learning applications, where creating custom models is important. giotto-time also provides a simple API to automate feature selection, perform causality tests, and generate new features.

In this blog post we present some of the functionalities of the Python time series library giotto-time. The accompanying notebook is available here.

The data and the problem

Electricity is crucial in today’s society. However, it is hard and expensive to store. This is especially true with the increasing importance of renewables in energy markets, which makes forecasting the energy demand and important task for energy providers.

To showcase the features available in giotto-time, we will focus on the problem of forecasting the Swiss energy demand up to 21 days in advance.

The data was downloaded from Swissgrid, a Swiss energy transmission grid operator and can be found here. The plot shows the daily average energy demand from 2016 to 2019.

The time series library giotto-time

Getting started with giotto-time is a simple as running:

pip install giotto-time

In the following sections we will go through the four most important features of giotto-time, which have also been used to model the energy demand:

Detrending and seasonality

Causality tests

Feature creation

Forecasting

The library is designed to make full use of the powerful time series and date functionality of pandas. In particular, the central object in giotto-time is a pandas DataFrame with a DateTimeIndex for timestamps and columns for each feature.

Another important aspect of the library is tight integration with scikit-learn. This enables consistent use of the fit , predict and transform paradigm of the popular machine learning framework.

1) Detrending and seasonality

The first step in the giotto-time pipeline is to model the underlying trend and seasonality in the time series. The Detrender in giotto-time makes it possible to do this by choosing either a polynomial or exponential model. Then simply calling fit_transform on the time series will remove the trend as shown in the code snippet below.

How to remove a trend with giotto-time

In order to encode seasonalities, e.g. summer and winter or weekly seasonalities, it’s possible to fit a sine wave with an appropriate phase shift as in the example below.

This can be done as follows:

How to add a periodic seasonal feature

2) Causality tests

Assessing causality is one of the most debated topics in time series analysis, and often requires an ad-hoc analysis. Due to the amount of data and external factors to take into account, this is often not possible. In giotto-time we provide a method to automate this process, while keeping the statistical rigour of ad-hoc approaches.

Correlation doesn’t mean causation. But, if statistically significant, it can be a helpful feature in a forecasting framework

The ShiftedLinearCoefficient class finds the optimal lag for which a target time series is most correlated to any feature time series. Once the shift is found, a statistical test is performed to assess the significance of the correlation. Finally, through the transform method, all the feature time series are automatically shifted forward to align them with the target time series at the optimal lag. In this way the output is ready to be used for predictions!

Data scientist using Giotto for the first time

It’s possible to use either linear fit or Pearson correlation coefficients. Either bootstrap tests or permutation tests can be performed to assess significance. In the code example below you can see how this is done in practice. In the end, we can shift the columns in the data frame by simply using the transform method.

How to perform a causality test and shift the time series

3) Feature creation

Arguably, one of the most important functionalities of the library is the feature creation step.

The following functions are available:

Shift feature: used to shift a time series by a given number of steps

Moving average: for a given window size

Constant feature: a constant value for all the time steps

Polynomial feature: for a given degree

Exogenous feature: makes it possible to add another time series

Calendar feature: public holidays for a given country

Periodic seasonal feature: can be used to create a sine wave with a given start date and a certain period

Custom feature: use your own function here

How to create features

Once the features have been created, we need to split the X and y data frames. In particular we have to define a time horizon which corresponds to the number of time steps we would like to look ahead. For the energy demand example, this is 21 days which results in a data frame with 21 columns and NaN values in the lower right triangle of the test values of y. This is due to there being no known values there. The label creation is done using the following transformer:

How to create the labels

An example is shown in the figure below. The first column corresponds to looking one time step ahead, the second two and so on.