Time Series Data Introduction
By: Inbar Oz
Tags: Data Pre Processing, Time Series Analysis, Python, Data Science, Time Series Forecasting

Time series data introduction:

Central Park, NYC by Fabio Fistarol on Unsplash

From weather forecasting to stock market forecasting or medical diagnosis, solving many of the world’s greatest challenges relies on the ability to model time series data. Time series can be defined as a collection of random variables indexed by the order they are obtained in time. The modeling goal is to acquire an understanding of the underlying structure that produced the observed data while considering its unique time dependencies and characteristics.

So… How do we start?

The most important step is to define the research data attributes by the common literature terms. Using accurate terms is very helpful in literature examination and allows you to retrieve past similar works and relevant methods to tackle your challenges. Identifying the data characteristics is important for model selection since some models have shown greater success when used on certain data types. In addition, some models make assumptions about the data that require data preparation.

In the following sections, I will demonstrate these concepts using weather data measured at the Central Park of New York between the years 1800–2020. The full code & data are available here. Let’s begin!

Time series data attributes terminology:

Is the data univariate/ multivariate?

Univariate time series consists of a single sequence of observations recorded over time increments. A multivariate time series data has more than one time-dependent series.

What is the sampling frequency of the time entries?

Is the time series containing a value per year? month? second?

For example, the temperature is a continuous signal and therefore temperature measurements can be sampled once an hour, minute, second and more. If the data is multivariate, it can also have a multi-frequency structure, meaning each time series can be sampled at a different time frequency. Multi-frequency data requires special attention in feature engineering, to be explained below.

Is the data Stationary?

In a stationary time series, the series summary statistics such as the mean don’t change significantly over time. When performing statistical time series modeling we will usually want to approximate the series summary statistics such as expected value, variance, and the correlation between the history values to the next values. By working with stationary series we can utilize that summary statistics don’t change over time and assume they will not change at the prediction series values. Therefore, there are fewer model parameters to approximate making stationary data easier to analyze and predict.

To check stationarity, we can calculate statistics such as the mean and variance of the series values at different time intervals, and make sure there are no significant differences between the results over time.

We can use a statistical test to check whether the data is stationary or not, such as the ADF test (Augmented Dickey-Fuller test).

The null hypothesis: The data is not stationary. The alternative hypothesis: The data is stationary. When examining the max temperature series, the p-value received is above 0.05, so we can’t reject the null hypothesis, which means we can’t assume the data is stationary.

Does the data have a seasonality pattern?

seasonality is the presence of alterations that occur at specific regular time intervals, such as weekly, monthly or more.

How to detect the seasonality patterns of the series?

we can check the autocorrelation, the linear correlation of a series with a lagged version of itself as a function of the time lag between them.

The ACF (autocorrelation function) values can be calculated for a range of lags, lags with extreme autocorrelation values define the seasonality pattern. Non-linear autocorrelation can be checked by other serial dependency tests such as the Durbin-Watson test. In the following ACF plot, we can see that the highest correlation is achieved by a lag of 1, meaning that the observation at time t is mostly explained by the observation at time t-1. We can also see an yearly high correlation, demonstrating the yearly weather seasonality, and a half year pattern of negative correlation, demonstrating the seasons changing.

Are there missing values?

After we understand the signal sampling frequency, some time entries may be missing. For example, a series of hourly temperature measurements may not include some records due to a temporary malfunction of the recording system.

Does the time series have Auxiliary features?

Auxiliary features are any attributes that provide additional information of the whole time series. for example, for a time series of sequential blood test results, auxiliary features can be the patient’s sex, age, and weight. In this weather data, we can see an example for an auxiliary feature — the location of the weather station.

Wait, the model I want to use assumes certain input attributes, that don’t exist in my current data. What can I do?

Data preparation:

Some models require several data preparation steps, the most common are described below.

How to make the data stationary?

To make the data stationary, our main goal is to clean the trend and seasonality if exists.

1. Difference the series values: Differencing means that for a Time series X, values differencing generates a new series Z where z(k) = x(t)-x(t-1). One can conduct multiple differencing steps, the first differencing step removes a linear trend, a second differencing step removes a quadratic trend, and so on.

data['max_temperature_roll_by_1'] = np.roll(data.max_temperature, 1)

data['diff_max_temperature'] = data.max_temperature - data.max_temperature_roll_by_1

Now let’s run the ADF stationarity test again: Now the P-value received is less than 0.05, which indicates that the Differenced series is stationary!

2. Difference the time entries: We can difference the time entries from a certain time occasion and receive a new series that is scaled to this event.

For instance, if we model a series of alerts after a cyber attack was detected, we may want to scale the time of alerts to the detection time, since this transformation will enable one to conduct a comparison between reaction times at different cyber attack events.

3. Remove trend: Removing trend can be done by fitting a linear regression to model the trend and continue in modeling the residuals, which are now clean of the trend. As we can see, there is an increasing trend in the series of daily maximum temperatures:

After fitting an LR model, there is no trend observed at the residuals series:

reg = LinearRegression().fit(data[['jul_date']], data['max_temperature'])

data['reg_pred'] = reg.predict(data[['jul_date']])

data['LR_residuals'] = data.max_temperature - data.reg_pred

4. Remove seasonality:

4.1. Once the seasonal lag was detected we can use it to difference the time series values by creating a new series Z where:

Z(k) = x(t)-x(t-seasonality_lag)

data['max_temperature_roll_by_year'] = np.roll(data.max_temperature, 365)

data['max_temperature_removed_seasonality'] = data.max_temperature - data.max_temperature_roll_by_year

4.2. We can also use the seasonality pattern detected and calculate the average pattern values. For example, We can calculate the average max temperature per day of year (range : [1,365]), and create a new series:

Z(t) = x(t)-x avg_of_dayofyear

avg_by_dayofyear = data.groupby('dayofyear').max_temperature.mean().reset_index().rename(

columns = {'max_temperature':'avg_max_temperature'})

data = data.merge(avg_by_dayofyear, on=['dayofyear'],how = 'left').set_index(data.index)

data['max_temperature_removed_avg'] = data.max_temperature - data.avg_max_temperature

5. Conduct a logarithmic or exponential transformation:

This kind of transformation may help in stabilizing the variance of the series across time.

How to treat missing values in time series?

Some models have a built-in mechanism for it, but other models require full time series input. To handle time series missing values one can:

1.Impute the missing entries:

Interpolate the missing data by fitting a curve (for example — by using linear regression) to each time series and complete the missing time entries by calculating the curve value.

(for example — by using linear regression) to each time series and complete the missing time entries by calculating the curve value. Spline interpolation: Instead of fitting a single high dimensional curve, one can fit multiple curves to subsets of the series.

Instead of fitting a single high dimensional curve, one can fit multiple curves to subsets of the series. Impute the missing values by an ML model. In multivariate data, the model may use other features to impute the missing values at a certain feature.

2.Remove samples or features with missing information

3. Let the model handle it — some models only require a declaration of the missing entries indices and overcome the missing information in a way that is optimized to the model.

Please remember: The common interpolation technique of filling the missing values by using summary statistics may not work well in time series data.

Completing missing values examples:

Completing missing values using linear interpolation:

data['interpolate_max_temperature_linear'] = pd.DataFrame.interpolate(data.missing_max_temperature, method = 'linear')

Completing missing values using the next valid observation:

data['backfill_max_temperature'] = data.missing_max_temperature.fillna(value=None, method='backfill', axis=None, limit=None, downcast=None)

How to transform multi-frequency data to uni frequency data?

When working with multivariate time series it may include series with higher frequency than others. For example, a low frequency series can describe a price of a sweater per day, and a high frequency series can describe the hourly temperature. Some models assume that all time series data follows the same frequency. To provide that, we can choose to:

Work in low frequency: summarize over the highest frequency series to achieve a new low frequency series. For example, calculate the average temperature per day. The main drawback of this method is that we lose information by this resolution reduction.

Work in high frequency: Expand the low frequency series to high frequency. For example, expand the daily price series to hourly series, by duplicating the daily value 24 times. The main drawback of this method is that the enhanced resolution may be a misleading description of the high frequency actual observations. In the previous example, the temperature has varied during the day and didn’t have a constant value across time.

Great! Our data is now ready for modeling and evaluation.

To be discussed in the following posts, stay tuned!