Time Series Clustering and Classification on the Dow Jones.
By: Peijin Chen
Tags: Deep Learning, Time Series Analysis, Machine Learning, Time Series Forecasting, Dow

What I want to do is build a model for sequence-to-sequence type prediction on the Dow Jones Industrial Average (DJI). However, rather than treat this as a regression problem, I want to discretize the problem, so that what I am predicting is one of a set number of possible “types” of predictions. Using SAX and PAA, I can take the closing prices of the DJI on 50 consecutive days. I do this by using PAA to reduce the window of 50 points to 3 points, and use the letters ‘a’ and ‘b’ (below mean, above mean) to describe this windows as a three letter word such as ‘aab’ or ‘aba’. Since I am only using ‘a’ and ‘b’, I have 8 possibilities, though only 6 show up in the data (from 1970–present). Here’s how I did this: first, I used the nolitsa library to do some simple moving average smoothing on the data. Then I created the 50 time step windows with a stride of 1 (so lots of overlapping) using the pyentrp. Finally, I used saxpy to do PAA (reducing 50 to 3) and then SAX to create symbolic representation using 2 letters.

import numpy as np

from pyentrp import entropy as ent

from saxpy.znorm import znorm

from saxpy.paa import paa

from saxpy.sax import ts_to_string

from saxpy.alphabet import cuts_for_asize

words = []

dow_df = ent.util_pattern_space(hist_sma, lag = 1, dim = 50)

dow_df = dow_df[:]

for i in range(len(dow_df)):

dat_znorm = znorm(dow_df[i,:])

dat_paa= paa(dat_znorm, 3) #three letter words

word = ts_to_string(dat_paa, cuts_for_asize(2)) # 2 let alphabet

words.append(word)

Here are counts for each of the 6 words:

Counter({‘abb’: 3505, ‘aab’: 3263, ‘bba’: 1930, ‘baa’: 1776, ‘aba’: 978, ‘bab’: 928})

I used a standard Bidirectional LSTM with one layer and then a Dense/Softmax layer and a sequence of anything from 3–10 timesteps to make the prediction of the next step.

model = Sequential()

model.add(Bidirectional(LSTM(32, kernel_initializer = "he_normal",return_sequences=False),

input_shape=(n_timesteps, output))) #model.add(Bidirectional(LSTM(32, return_sequences = False)))

model.add(Dense(output, activation='softmax'))

model.summary()

model.compile(loss='categorical_crossentropy', optimizer=Adam(lr=.001), metrics=['acc'])

With early stopping, you can get around 84% F1-score, so that’s not too bad, though the sequence itself is not that varied. Of the 6 “labels” you will find that it stays this way for quite a number of steps (at least more than 10) before changing into another label. When you look at the predictions and the ground-truth, you will find that the transitions are always off — that’s where the model makes a bunch of mistakes. It tends to “figure out” the change 1 or 2 steps after it happens in ground-truth.

Euclidean K-means on the DJI

Next, I decided to use the tslearn package to cluster the 50 time step windows; using the Euclidean metric (the fastest one by a long shot). I wanted to see what I would get if I demanded 6 clusters.

km = TimeSeriesKMeans(n_clusters=6,verbose=True, random_state=seed)

y_pred = km.fit_predict(df)

This is what came out:

6 mean “paths” for 50 time-step windows of the Dow Jones Industrial Average.

There is some level of resemblance between the ‘words’ and the ‘clusters’. If you were just dealing with clusters; you could either train a model to learn the sequence of clusters, or else just create a classification model that takes maybe 20–30 points of your current window, and classify with one of the 6 labels so that you had some idea of where it might go in the near future.

Lastly, I’ll just say that it would be interesting to see what would happen if you either wanted more clusters or if you changed the time period…and also what would happen if you used a stride > 1, thereby downsampling and using less of the data. Maybe some other time.