Time Series Data and Machine Learning-Part 2: Time series as inputs to a model
By: Yixuan Zhou
Tags: Python, Feature Engineering, Machine Learning, Time Series Analysis, Scikit Learn

The easiest way to incorporate time-series into your machine learning pipeline is to use them as features in a model. This chapter covers common features that are extracted from time series in order to do machine learning.

Classification and Feature Engineering

Using raw timeseries data is too noisy for classification

We need to calculate features. An easy start is to summarize statistics of your audio data, removes the time dimension and give a more traditional classification dataset.

Here is the process: for each timeseries, we calculate several summary statistics. These then can be used as features for a model.

print(audio,shape)

→#(n_files, time)

→(20, 7000)

means = np.mean(audio, axis=-1) #last column/dimension maxs = np.max(audio, axis=-1) stds = np.std(audio, axis=-1) print(means.shape)

#(n_files,) array of numbers, one per time series

→(20,)

In summary, we’ve just collapsed a 2-D dataset (samples x time) into several features of a 1-D dataset(samples)

We can combine each feature, and use it as an input to a model

If we have a label for each sample, we can use scikit-learn to create and fit a classifier.

from sklearn.svm import LinearSVC

#Note that means are reshaped to work with scikit-learn

X = np.column_stack([means, maxs, stds]) y = labels.reshape([-1, 1]) model = LinearSVC() model.fit(X,y)

Scoring your scikit-learn model

from sklearn.metrics import accuracy_score

#Different input data

predictions = model.predict(X_test)

#Score our model with % correct manually

percent_score = sum(predictions == labels_test) / len(labels_test)

#Using a sklearn scorer

percent_score = accuracy_score(labels_test, predictions)

Heartbeat data Example

Some recordings are normal heartbeat activity, while others are abnormal activity.

Two dataframes, normal and abnormal, each with the shape of (n_times_points, n_audio_files) containing the audio for several heartbeats. Also, the sampling frequency is loaded into a variable called sfreq. A convenience plotting function show_plot_and_make_titles() is also available in workplace

fig, axs = plt.subplots(3, 2, figsize=(15, 7), sharex=True, sharey=True) # Calculate the time array

time = np.arange(len(normal)) / sfreq # Stack the normal/abnormal audio so you can loop and plot

stacked_audio = np.hstack([normal, abnormal]).T # Loop through each audio file / ax object and plot

# .T.ravel() transposes the array, then unravels it into a 1-D vector for looping.

for iaudio, ax in zip(stacked_audio, axs.T.ravel()):

ax.plot(time, iaudio)

show_plot_and_make_titles()

As you can see, there is a lot of variability in the raw data, let’s average out some of that noise to notice a difference.

A common technique to find simple differences between two sets of data is to average across multiple instances of the same class. This may remove noise and reveal underlying patterns (or it may not).

# Average across the audio files of each DataFrame

mean_normal = np.mean(normal, axis=1)

mean_abnormal = np.mean(abnormal, axis=1) # Plot each average over time

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 3), sharey=True)

ax1.plot(time, mean_normal)

ax1.set(title=”Normal Data”)

ax2.plot(time, mean_abnormal)

ax2.set(title=”Abnormal Data”)

plt.show()

There we could see a noticeable difference between the two. Maybe, it’s quite noisy. Let’s see how we could dig into the data a bit further.

Build a classification model with raw data

from sklearn.svm import LinearSVC # Initialize and fit the model

model = LinearSVC()

model.fit(X_train,y_train) # Generate predictions and score them manually

predictions = model.predict(X_test)

print(sum(predictions == y_test.squeeze()) / len(y_test))

→0.5555555

Note that the predictions didn’t do so well. That’s because the features you’re using as inputs to the model (raw data) aren’t very good at differentiating classes.

Next, we’ll explore how to calculate some more complex features that may improve the results.

we’ll smooth the data to calculate the auditory envelope. The envelope throws away information about the fine-grained changes in signal, focusing on the general shape of the audio waveform.

To do this, we’ll need to calculate the audio’s amplitude, then smooth it over time. First, we’ll remove noise in timeseries data by smoothing it with a rolling window. Instead of averaging over all time, we can do a local average. It removes short-term noise, while retaining the general pattern.

Calculating a rolling window statistic

#Audio is a pandas dataframe print(audio.shape) #(n_times, n_audio_files) →(5000, 20) #Smooth data by taking the rolling mean in a window of 50 samples window_size = 50 windowed = audio.rolling(window=window_size) audio_smooth = windowed.mean()

Calculating the auditory envelope

audio_rectified = audio.apply(np.abs) #so all data are positive audio_envelope = audio_rectified.rolling(50).mean()

The result is a smooth representation of how the audio energy changes over time.

Feature engineering the envelope

#calculate several features of the envelope, one per sound envelop_mean = np.mean(audio_envelope, axis = 0) envelop_std = np.std(audio_envelope, axis=0) envelop_max = np.max(audio_envelope, axis=0) #create training data for a classifier X = np.column_stack([envelope_mean, envelope_std, envelope_max]) y = labels.reshape([-1,1])

Cross Validation for Classification

cross_val_score automates the process of

1) splitting data into training/validation sets

2) fitting the model on training data

3)scoring it on validation data

4) repeating this process

from sklearn.model_selection import cross_val_score model = LinearSVC() scores = cross_val_score(model, X, y, cv=5) #split data into 5 print(scores)

→[0.60911642 0.59975305 0.61404035 … …]

print(np.mean(scores))

→ 0.598476

Auditory features: The Tempogram

We can summarize more complex temporal information with timeseries-specific functions. librosa is a great library for auditory and timeseries feature engineering. The tempogram estimates the tempo of a sound over time. We can calculate summary statistics of tempo in the same way that we can for the envelopment.

import librosa as lr audio_tempo = lr.beat.tempo(audio, sr=sfreq, hop_length=2**6, aggreate=None)

Note that librosa functions tend to only operate on numpy arrays instead of DataFrames, so we’ll access our Pandas data as a Numpy array with the .values attribute.

# Calculate the tempo of the sounds

tempos = []

for col, i_audio in audio.items():

tempos.append(lr.beat.tempo(i_audio.values, sr=sfreq, hop_length=2**6, aggregate=None)) # Convert the list to an array so you can manipulate it more easily

tempos = np.array(tempos) #（60，138） 60 lists and each list has 138 data # Calculate statistics of each tempo

tempos_mean = tempos.mean(axis=1)

tempos_std = tempos.std(axis=1)

tempos_max = tempos.max(axis=1)

#HERE IS AN EXAMPLE OF MEAN()

# Create the X and y arrays; the first three column are from the former step: for each timeseries, we calculate several summary statistics.

X = np.column_stack([means, stds, maxs, tempos_mean, tempos_std, tempos_max])

y = labels.reshape([-1, 1])

# Fit the model and score on testing data

percent_score = cross_val_score(model, X, y, cv=5)

print(np.mean(percent_score))

Note that your predictive power may not have gone up (because this dataset is quite small), but you now have a more rich feature representation of audio that your model can use.