IEEE Workshop on Nonlinear Circuit Networks
December 4-5, 2020

Influence of Surrogate Dataon Time Series Classification by Neural Network
Tomiyuki Furugaki† , Tomoya Takata† ,Yoko Uwate† , Yoshifumi Nishio† and Hidekuni Takao‡

Tokushima University, 2-1 Minami-Josanjima, Tokushima-shi, Tokushima-ken 770-8506, Japan†
Email: {furugaki, tomoya, uwate, nishio}@ee.tokushima-u.ac.jp†
Kagawa University, 2217-20 Hayashi-Cho, Takamatsu-shi, Kagawa-ken 761-0396, Japan‡
Email: takao@eng.kagawa-u.ac.jp‡
Abstract—Neural Networks (NN) can classify time series data.
One of the challenges is understanding what the NN depend
on for classification. In this study, 1-Dimensional Convolutional
Neural Network (1D-CNN) is used as a classification models
of surface shape of material. In this study, we focus on therelationship between autocorrelation, frequency distribution andtest accuracy for using 1D-CNN with surrogate data.

I. I NTRODUCTION
Neural Networks (NN) are a system modeled on neurons ofthe human brain nervous system. Among them, 1-Dimensional
Convolutional Neural Network (1D-CNN) is used for timeseries classification. 1D-CNN needs to learn time series datain advance to classify. There are various characteristics depending on each time series source. 1D-CNN classifies byfinding them. However, the specific judgment part is unknown.
Therefore we need to find these.
In this study, surrogate data method is used. The surrogate
data method is creating surrogate data. Surrogate data isdata that preserves some of the statistical properties of timeseries data and destroys other properties. After that, it isindicated that there is a significant difference between thestatistical properties of time series data and the surrogate data.
In this way, the method proves the importance of destroyed
properties.
In this study, the data that the 1D-CNN learns is replaced
from the original data with surrogate data. In this way, we canfind out which part of the time series data is important.
II. C ONVOLUTIONAL N EURAL N ETWORKThe research on CNN was established as an academic field
in 1956. Since then, it has repeated the ice ages and booms
many times and now reaches the present. Currently, CNN isdiverse in medical fields, car fields, home electronics fields andso on. The beginning of these booms was image recognition.
CNN is inspired from the biological process and conceived
from the arrangement of the visual cortex of animals. In thefield of image recognition, CNN has achieved tremendous
performance with many tasks. In addition, CNN is attractingattention. In particular, the intermediate layer of CNN extracts
high versatility and splendid feature quantities. The network
structure of CNN is divided into an input layer, an intermediate
layer and an output layer. The intermediate layer includes

convolution layers, pooling layers and fully connected layers.
Features of inputs are extracted in the convolution layer, andposition invariance is acquired in the pooling layer. Next, it
becomes the 1-Dimensional array in fully connected layers andit changes to probability. Finally, CNN outputs classification
results by the probability. However, in recent years, CNN has
also been use to time series data. It is 1D-CNN. To date,
Recurrent Neural Network (RNN) has been the mainstream
for learning time series data. In this study, CNN is used fortime series data that is one-dimensional data.
III. S URROGATE DATA METHOD
The surrogate data method was proposed in 1992 for chaostime series analysis. There are no necessary and sufficientconditions for chaos. Therefore, the only way to determine
chaos is to determine that there is chaos. In many cases,chaos is determined by spectral continuity, strange attractors,
Lyapunov exponents, bifurcations, and so on. However, it
has been pointed out that even with random noise alone.
The Lyapunov exponent is positive and noise and chaos
cannot be distinguished. Therefore, the surrogate method wasproposed to test whether it is noise or data generated froma deterministic system. hypothesis test, it is difficult to saythat it is noise if a data passes the test. However, it cannot beasserted that it is chaos because, the surrogate method is basedon hypothesis testing in statistics. In this study, surrogate data
was created and compared the accuracy of learning surrogate
data with the accuracy of learning original data. In this way,
we researched that characteristics of the original data wereimportant.
IV. DATASETIn this study, time series data of surface profile of thematerial were classified. Three types of surface profile wereprepared. In this study, they were called data 1, 2 and 3.
Fifteen pieces of data for 20 second were prepared. Each timeseries data was sampled at a sampling frequency 3000 [Hz].
Next, the data was agumentationed. There were two types ofaugmentation. Figure 1 shows the examples of the originaltime series data. Figure 2 and 3 show the examples of thetime series data about the data was added time shift and timestretch.

- 108 -(b) Fourier Transform Surrogates Data (FTSD)n∑2πkn
X(ω) =x(n)e−i N(1)

n=1
2πkn
1 ∑X(ω)ei N
N n=1nx(n) =Fig. 1.

Original data.

Fig. 2.

Time shift data.

Equations (1) and (2) show discrete Fourier Transform (DFT)and Inverse Discrete Fourier Transform (IDFT). k meansfrequency. N (= 5000) means the number of the samples.
Step 1. Calculate DFT X(ω) of x(n).
Step 2. Randomize the phase of X(ω).
Step 3. Calculate IDFT randomized X(ω).
FTSD was implemented using the previous steps. FTSD has
broken the frequency distribution of the data. Figure 5 shows
FTSD of the Fig. 1.

Fig. 5.

Fig. 3.

Time stretch data.

V. P ROPOSED M ETHOD
Four types of surrogate data were created. Surrogate data
was destroyed some information. The following explanations(a), (b), (c) and (d) describe how to create four types ofsurrogate data. (a) Random Shuffle Surrogate Data (RSSD)
x(n) means time function. n means time. The order of n ischanged randomly by the RSSD data. RSSD has broken theautocorrelation of the data. Figure 4 shows RSSD of the Fig. 1

Fig. 4.

(2)

FTSD.

(c) Amplitude Adjusted Fourier Transform Surrogates Data(AAFTSD)
Step 1. Prepare random numbers R(n) according to thestandard normal distribution.
Step 2. Sorting R(n) in the same size relation as x(n).
Step 3. Create R′ (n) which is FTSD of sorted R(n).
Step 4. Sorting x(n) in the same size relation as R′ (n).
AAFTSD was implemented using the previous steps.
AAFTSD has broken the autocorrelation. However AAFTSD
has similar autocorrelation than that of original. Figure 6
shows AAFTSD of the Fig. 1.

Fig. 6.

RSSD.

- 109 -AAFTSD.

(d) Iterated Amplitude Adjusted Fourier Transform Surrogates Data (IAAFTSD)

exp(xj )ρ(x) = ∑ni=1 exp(xi )(0)

Step 1. Prepare s which is RSSD of original data as theinitial value.
Step 2. Calculate DFT S (i) of s(i) .
Step 3. Replace amplitude of S (i) with amplitude of orig(i)inal. Put it as S(i)Step 4. Calculate IDFT s(i) of S .
(i)
Step 5. Sorting s in the same size relation as originaldata.
Step 6. Add 1 to i.
Step 7. Increase the number of i.
IAAFTSD was implemented using the previous steps.
IAAFTSD saves the frequency distribution and has similar
autocorrelation than that of original than that of AAFTSD.
The more i, the closer the autocorrelation of the IAAFTSD
approaches that of the original. Figure 7 shows IAAFTSD ofthe Fig. 1.

{f (α, x) =αx (x < 0)x(x ≥ 0)(3)

(4)

ρ(x) was the probability of being classified as j. n was thetotal number of classes. α was a parameter that is determinedby learning. The PReLU function learns values during training.
Therefore, it can better adapt to weights and biases. Figure 9shows PReLU function when α = 0.02.

Fig. 9.

PReLU function.

VII. S IMULATION RESULTS
Fig. 7.

Fifteen pieces of data were prepared. After sampling them,each data was divided into four pieces. Furthermore, surrogate
data was created. Table 1 shows the number of the train data
and test data. Table 2 shows the results of our research.

IAAFTSD.

VI. A RCHITECTURE
TABLE IT HE NUMBER OF THE TRAIN DATA AND TEST DATA

1D-CNN was used for convention architecture. Figure 8
shows the structure of 1D-CNN we used.

data 1data 2
data 3train data
336
336
336

test data
84
84
84

TABLE IIT EST ACCURACY
original data
RSSD
FTSD
AAFTSD
IAAFTSD(i = 7)
Fig. 8.

test accuracy
47.77778
39.72210
54.49840
52.14286
54.12698

Structure of 1D-CNN.

Two convolutional layers were used. Drop out layer was prepared to prevent over learning. Classification results is derived
the probability by the softmax activation function. Parametric
Rectified Linear Unit (PReLU) function was prepared forthe activation function. Equation (3) shows softmax function.
Equation (4) shows PReLU function.

Table 2 shows ten-times-averaged test accuracy results.
Table 2 shows test accuracy of RSSD is lower than that of
the FTSD. Test accuracies of IAAFTSD, AAFTSD and FTSD
are higher than that of original.
Figure 10 shows the relationship between test results andautocorrelation relative error of surrogate data.

- 110 -[4] James Theiler. ”Some Comments on the correlation dimension of 1/f α
noise Physics LetterS A, Vol. 155, No. 8,9, pp. 480-493, May 1991.
[5] James Theiler, Stephen Eubank, Andre Longtin, Bryan Galdrikian, andJ. Doyne Farmer. ”Testing for nonlinearity in time series: the method ofsurrogate data” Physica D, Vol. 58, pp. 77-94, 1992.
[6] Dean Prichard and James Theiler. ”Generating Surrogate Data for with
Several Simultaneously Measured Variables”. Physical Review Letters,Vol. 73, No. 7, pp. 951-954, August 1994.
[7] Thomas Schreiber and Andreas Schmitz.”Improved Surrogate Data for
Nonlin- earity Tests”. Physical Review Letters, Vol. 77, No. 4, pp. 635638, 1996.
Fig. 10.

Relative autocorrelation and test result.

∑N/2
Dr =2
f =0 (Y2 (f ) − Y1 (f ))∑N/22
f =0 Y1 (f )(5)Equation (5) denotes autocorrelation relative error of surrogate
data. Y1 (f ) means fourier transform of original data. Y2 (f )means fourier transform of surrogate data. First, create surrogate data of the original data. Next, create the surrogate data of
the previous surrogate data. In this way, the data was createdwith only the autocorrelation gradually destroyed. Figure 10
shows test accuracy will increase when the autocorrelation of
the original data is broken just a little. However, test accuracy
will decrease when the autocorrelation of the original data isbroken larger.
VIII. C ONCLUSION
In this study, three types classification was carried out withsurrogate data. Then, we understood that test accuracy of
RSSD is lower than that of the FTSD. RSSD has broken theautocorrelation of the data. FTSD has broken the frequency
distribution of the data. Therefore, it was understandable
that autocorrelation is more important than than frequency
distribution for 1D-CNN. Furthermore, test accuracy will
increase when the autocorrelation of the original data is broken
just a little. However, test accuracy will decrease when theautocorrelation of the original data is broken larger.
The future challenge is to find new indicators that relate to
the test accuracy. Test accuracy will increase when the autocorrelation of the original data is broken just a little. However,
it is not obvious that why has test accuracy improved.
R EFERENCES[1] James Theiler, Paul Linsay, and David M.Rubin. ”Detecting Nonlinearity
in Data with Long Coherence Times”. In Andreas S.Weigend andNeil A.ershenfeld, editors, ”Time Sereies Prediction Forecasting theFuture and Understandung the Past”, pp.429-445. Addison Wesley,1993.
Proceedings Volume In The Santa Fe Institute Studies In The Science
Of Complexity.
[2] James Theiler, Bryan Galdrikian, Andr Longtin, Stephen Eubank, andJ. Doyne Farmer. ”Using Surrogate Data to Detect Nonlinearity in TimeSeries”. In Martin Casdagli and Stephen Eubank, editors, ”Onlinear
Modeling And Forecasting”, pp. 163-188. Addison Wesley, 1992. AProceed- Ings Volume In The Santa Fe Institute Studies In The Science
Of Complexity.
[3] Daniel T. Kaplan and Richard J. Cohen. ”Is Fibrillation Chaos?”.
Circulation Research, Vol. 67, No. 4, pp. 886-892, October 1990.

- 111 -