2019 RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing (NCSP2019)
Honolulu, Hawaii, USA, March 4-7, 2019

Time Series Classification with Time Delay Coordinate System
Using Convolutional Neural Network
Tomoya Takata1 , Ken Kamiyotsumoto1 , Kazuki Nagao1 ,Yoko Uwate1,3 , Yoshifumi Nishio1 and Hidekuni Takao2,31

Department of Electrical and Electronic Engineering, Tokushima University
2-1 Minami-Josanjima, Tokushima 770-8506, Japan
Email:{tomoya, kamiyotsumoto, k-nagao, uwate, nishio}@ee.tokushima-u.ac.jp
2
Department of Intelligent Mechanical System Engineering, Kagawa University
2217-20, Hayashi-cho, Takamatsu, Kagawa 761-0396, Japan
Email:takao@eng.kagawa-u.ac.jp
3JST-CREST

Abstract
Time series data are often classified by using Recurrent Neural Network and 1-Dimensional Convolutional Neural Network. In this study, we used 2-Dimensional Convolutional
Neural Network for 1-Dimensional data using the chaos theory. We used the time delay coordinate system for inputsof Convolutional Neural Network. By constructing a figure
called an attractor and analyzing it with 2-Dimensional Convolutional Neural Network, we could improve some of thetest accuracy compared with conventional methods.
1. Introduction
Time series data are used everywhere in society. Classification of time series data has been actively conducted in recentyears. Its characteristics are varied and those with regularity and those with irregular chaotic properties can be mentioned. Chaos is a phenomenon showing a complicated appearance. The appearance is considered to be unpredictable
due to numerical error and found in a part of a dynamicalsystem. Chaos is unlike any unpredictable random. Although
its behavior follows the deterministic rule, it cannot obtainthe solution by the integral method. Therefore, we need tofind other methods. The one of methods is converting 1Dimensional data into 2-Dimensional data (2D-data). By using this method, we can observe the behavior [1]. Furthermore, we can classify time series data to analyze this 2D-data.
Thereupon, we are required to discover how to analyze it.
In recent years, the method which analyze 2D-data by Convolutional Neural Networks (CNN) attracts attention. CNN is
one of Neural Networks and a mathematical model similar tosome properties found in brain functions. In this study, we expect to improve test accuracy of classification by combining
chaos theory and CNN.

many times and now reaches the present. Currently, CNN isdiverse in medical field, car field, home electronics field andso on. The beginning of these booms was image recognition.
CNN is inspired from the biological process and conceived
from the arrangement of the visual cortex of animals. In thefield of image recognition, CNN has achieved tremendous
performance with many tasks. In addtion, CNN is attracting attention. In particular, the intermediate layer of CNN
extracts high versatility and splendid feature quantities.
The network structure of CNN is divided into an inputlayer, an intermediate layer and an output layer. The intermediate layer includes convolution layers, pooling layers andfully connected layers. Features of inputs are extracted in theconvolution layer, and position invariance is acquired in thepooling layer. Next, it becomes the 1-Dimensional array infully connected layers and it changes to probability. Finally,
CNN outputs classification results by the probability.
In recent years, Recurrent Neural Network (RNN) which
has time series structure and 1-Dimensional CNN (1D-CNN)
which perform feature extraction of 1-Dimensional data (1Ddata) are often used. However, in this study, we convert
the 1D data to graphics and try to extract features of 2Dimensional data (2D data).
3. Dataset
We carry out time series classification of the surface shapeof fabrics. We prepare three types of fabrics. In this study, wecall them fabric1, 2 and 3. Table 1 shows the number of thetime series data each length.

2. Convolutional Neural Network
The research on CNN was established as an academic fieldin 1956. Since then, it has repeated the ice ages and booms

442

Table 1: The number of the time series data.
length of 500 (ms)length of 2000 (ms)train data
100
25

test data
205

2019 RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing (NCSP2019)
Honolulu, Hawaii, USA, March 4-7, 2019

Figures 1 shows the examples of the time series data about
the amplitude of the surface shape with the time series datawith the length of 500 (ms). Figures 2 shows the examples of
the time series data about the amplitude of the surface shapewith the time series data with the length of 2000 (ms).

(a) fabric1.

In this study, we set the number of the dimension to 2 to use
2D-CNN. Let the horizontal axis be x (n) and the vertical axis
be x (n + τ ). We analyze the image made by this operation
with 2-Dimensional CNN. Figures 3 and 4 show the examplesof the structure of the delay coordinate system each lengthof the fabric1. Figures 5 and 6 show the examples of thestructure of the delay coordinate system each length of thefabric2. Figures 7 and 8 show the examples of the structureof the delay coordinate system each length of the fabric3.

(b) fabric2.

(a) τ =5.

(b) τ =15.

(c) τ =25.

(d) τ =35.

(c) fabric3.
Figure 1: The amplitude of the surface shape of each fabric when
we use time series data with the length of 500 (ms).

Figure 3: Structure of the delay coordinate system when we usetime series data with the length of 500(ms) of fabric1.

(a) fabric1.

(b) fabric2.

(a) τ =5.

(b) τ =15.

(c) τ =25.

(d) τ =35.

(c) fabric3.
Figure 2: The amplitude of the surface shape of each fabric when
we use time series data with the length of 2000 (ms).

Figure 4: Structure of the delay coordinate system when we use
4. Proposed Method
We convert the 1D data into 2D data using time delay co- time series data with the length of 2000(ms) of fabric1.
ordinate system. Let the value of data at a certain time be
x (n). Furthermore, if the time delay value is τ , this system isrepresented by Eq. (1).
f (x) = [x(n), x(n + τ ), x(n + 2τ ), ...]

(1)

4432019 RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing (NCSP2019)
Honolulu, Hawaii, USA, March 4-7, 2019(a) τ =5.

(b) τ =15.

(a) τ =5.

(b) τ =15.

(c) τ =25.

(d) τ =35.

(c) τ =25.

(d) τ =35.

Figure 5: Structure of the delay coordinate system when we usetime series data with the length of 500(ms) of fabric2.

Figure 8: Structure of the delay coordinate system when we usetime series data with the length of 2000(ms) of fabric3.

5. Architecture
We used 1D-CNN and RNN for convention architecture.
The structures of 1D-CNN and 2D-CNN were the same. Figure 9 shows structure of CNN which we used in this study.
(a) τ =5.

(b) τ =15.

(c) τ =25.

(d) τ =35.

Figure 6: Structure of the delay coordinate system when we usetime series data with the length of 2000(ms) of fabric2.

Figure 9: Structure of CNN.

(a) τ =5.

(c) τ =25.

We used 2 convolutional layers. The pooling layers taketime to process. Therefore, we did not use the pooling layers
and we could reduce the time to process. We used the activation function after convolution called prelu [3]. The equationof prelu is expressed as follows.
{x (x ≥ 0)
f (x) =(2)ax (x < 0)(b) τ =15.

The negative slope a (0 < a < 1) is adjusted by network
learning. We derive the probability to calculate classification
results by the softmax activation function.

(d) τ =35.

Figure 7: Structure of the delay coordinate system when we use

exp(xj )ρ(x) = ∑ni=0 exp(xi )time series data with the length of 500(ms) of fabric3.

(3)

ρ(x) is the probability of being classified as j. n is the total
number of classes.

444

2019 RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing (NCSP2019)
Honolulu, Hawaii, USA, March 4-7, 2019

RNN is one of Neural Networks where connections beTable 4: Test accuracy with 2D-CNN changing τ when wetween nodes form a directed graph along a sequence. Thisuse 2000 length of time series data.
allows it to exhibit temporal dynamic behavior for a time sequence. Figure 10 shows structure of RNN we used in thisτ = 5 τ = 15 τ = 25 τ = 35study.
fabric1,20.5
1.0
0.8
1.0
In this study, we set the weight collection to 0.001 and set
fabric2,3
0.4
0.5
0.78
0.92
epochs to 150. In every dataset, train accuracy reached maxifabric3,10.65
0.8
0.56
0.9
mum value 1.
fabric1,2,3
0.34
0.43
0.53
0.67

Table 5: Comparison of test accuracy using 2D-CNN with
1D-CNN and RNN.
fabric1,2
fabric2,3
fabric3,1
fabric1,2,3

Figure 10: Structure of RNN.

RNN0.96
1.0
0.7
0.8

1D-CNN
1.0
1.0
0.3
0.67

2D-CNN1.00.92
0.9
0.67

In the future, we will construct the time delay coordinate
system with changing parameter τ . In addition, we will adjust
the width of dividing the time series data.

6. Simulation results
We show the effect of classification by time delay coordinate system. At first, we show the classification results when
we use time series data with lengths of 500 (ms) and 2000 Acknowledgment(ms) changing τ . At second, we compared the results with This research has been partially supported by JST-CREST rethe highest test accuracy using 2D-CNN with 1D-CNN and search funding program (Grant Number JPMJCR1531).
RNN.
References
Table 2: Test accuracy with 2D-CNN changing τ when weuse 500 length of time series data.
fabric1,2
fabric2,3
fabric3,1
fabric1,2,3

τ =5
0.95
0.95
0.6
0.65

τ = 15
0.93
0.75
0.78
0.61

τ = 25
0.93
0.93
0.55
0.65

τ = 35
0.95
0.9
0.45
0.62

Table 3: Comparison of test accuracy using 2D-CNN with
1D-CNN and RNN.
fabric1,2
fabric2,3
fabric3,1
fabric1,2,3

RNN
0.85
0.85
0.78
0.72

1D-CNN0.84
0.83
0.780.73

2D-CNN
0.95
0.93
0.78
0.65

[1] Holger Kantz and Thomas Schreiber, “Nonlinear Time
Series Analysis”, by the press syndicate of the university of cambridge, first published in 2000, second published in 2003.
[2] Aaron van den Oord, Sander Dieleman, Heiga Zen,
Karen Simonyan, Oriol Vinyals, Alex Graves, Nal
Kalchbrenner, Andrew Senior and Koray Kavukcuoglu,“WaveNet: A Generative Model for Raw Audio”, Submitted on 12 Sep 2016 (v1), last revised 19 Sep 2016
(this version, v2).
[3] Kaiming He, Xiangyu Zhang, Shaoqing Ren andJian Sun,“Delving Deep into Rectifiers: Surpassing
Human-Level Performance on ImageNet Classification”, arXiv:1502.01852, Submitted on 6 Feb 2015.

7. Conclusion
In this study, we used the time delay coordinate system.
This method showed any of test accuracy was improved compared with the conventional method. Sufficient accuracy wasobtained with binary classification. However, sufficient accuracy was not obtained with multiple classification.

445[4] Alex Sherstinsky, “Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory(LSTM) Network”, arXiv:1808.03314, Submitted on 9Aug 2018 (v1), last revised 4 Nov 2018.