IEEE Workshop on Nonlinear Circuit Networks
December 13-14, 2019

Voice Recognition Using Surrogate Methodwith 1D-Convolutional Neural Network
Tomiyuki Furugaki, Tomoya Takata,
Yoko Uwate and Yoshifumi Nishio
Dept. of Electrical and Electronic Engineering, Tokushima University
2-1 Minami-Josanjima, Tokushima 770–8506, Japan
Email:{furugaki, tomoya, uwate, nishio}@ee.tokushima-u.ac.jp

Abstract—Voice classification are often classified by using
Recurrent Neural Network and 1-Dimensional Convolutional
Neural Network (1D-CNN). 1D-CNN is used for classification
model. In general, 1D-CNN learns original data. However, in
this study, original data is replaced with surrogate data. Test
accuracies at that time are compared. In this way, we searchwhich part of time series data is effective for using 1D-CNN.

I. I NTRODUCTION
Neural Network (NN) is a system model on neurons of thehuman brain nervous system. Among them, 1-Dimensional
Convolutional Neural Network (1D-CNN) is used for voiceclassification. 1D-CNN needs to learn voice waveform inadvance to classify voice. There are various characteristics
depending on each sound source. 1D-CNN classifies voice byfinding them. However, the specific judgment part is unknown
because we need to find these.
In this study, the surrogate data method is used. Thesurrogate data method is creating surrogate data. Surrogate
data is preserves some of the statistical properties of timeseries data and destroys other properties. After that, it isindicated that there is a significant difference between thestatistical properties of time series data and the surrogate data.
In this way, the method proves the importance of destroyed
properties.
In this study, the data that the CNN learns is replaced fromthe original data with surrogate data. It can be seen that whichpart of the voice waveform is important by comparing the test
accuracy at that time.

II. C ONVOLUTIONAL N EURAL N ETWORKThe research on CNN was established as an academic field
in 1956. Since then, it has repeated the ice ages and booms
many times and now reaches the present. Currently, CNN isdiverse in medical field, car field, home electronics field andso on. The beginning of these booms is image recognition.
CNN is inspired from the biological process and conceived
from the arrangement of the visual cortex of animals. In thefield of image recognition, CNN has achieved tremendous
performance with many tasks. In addition, CNN is attractingattention. In particular, the intermediate layer of CNN extracts
high versatility and splendid feature quantities. The network

structure of CNN is divided into an input layer, an intermediatelayer and an output layer. The intermediate layer includes
convolution layers, pooling layers and fully connected layers.
Features of inputs are extracted in the convolution layer, andposition invariance is acquired in the pooling layer. Next, it
becomes the 1-Dimensional array in fully connected layers andit changes to probability. Finally, CNN outputs classification
results by the probability. In recent years, CNN has also been
use to audio signal processing. In this study, CNN is used fortime series data that is one-dimensional data to classify voice.
III. S URROGATE DATA METHOD
The surrogate data method was proposed in 1992 for chaostime series analysis. There are no necessary and sufficientconditions for chaos. Therefore, the only way to determine
chaos is to find out that there is chaoticity. In many cases,chaos is determined by spectral continuity, strange attractors,
Lyapunov exponents, bifurcations, and so on. However, it has
been pointed out that even with random noise alone. The
Lyapunov exponent is positive and noise and chaos cannot
be distinguished. Therefore, the surrogate data method isproposed to test whether it is noise or data generated froma deterministic system. With hypothesis testing, it is difficult
to say that it is noise if a data passes the test. However, itcannot be asserted that it is chaos because the surrogate data
method is based on hypothesis testing in statistics. In thisstudy, surrogate data is created and compared the accuracy oflearning surrogate data with the accuracy of learning originaldata. In this way, what characteristics of the original data areimportant can be found.
IV. DATASETIn this study, time series data of voices are classified. Threetypes of voice are prepared. In this study, they are called voice
1, 2 and 3. Forty pieces of data for 6 second each are prepared.
Each time series data is sampled at a sampling frequency 3000
[Hz]. Next, the data is augmentationed. There are three typesof augmentation. Figure 1 shows the examples of the originaltime series data. Figures 2, 3 and 4 show the examples of thetime series data about the data is added white noise, time shift
and time stretch for augmentation.

-5-V. P ROPOSED M ETHOD
Four types of surrogate data are created. Surrogate data is
destroyed some information. The following explanations (a),(b) , (c) and (d) describe how to create four types of surrogate
data.

Fig. 1: Original data(a) Random Shuffle Surrogate Data (RSSD)
x(n) means time function. n means time. It is RSSD data thatchanges the order of n at random. The correlation of original
data is broken by converting the data into RSSD. Figure 5
shows RSSD of the original data in Fig. 1.

Fig. 5: RSSD
Fig. 2: White noise data(b) Fourier Transform Surrogates Data (FTSD)

x(n) =n∑x(n)e−i N

(1)

2πkn
1X(ω)ei N
N n=1(2)X(ω) =

2πkn

n=1n
∑

Equations (1) and (2) show discrete Fourier Transform (DFT)and Inverse Discrete Fourier Transform (IDFT). k means
frequency. N (= 15000) means the number of the samples.
Step 1. Calculate DFT X(ω) of x(n).
Step 2. Randomize the phase of X(ω).
Fig. 3: Time shift data

Step 3. Calculate IDFT randomized X(ω).
FTSD is made in this way. The frequency distribution oforiginal data is broken by converting the data into FTSD.
Figure 6 shows FTSD of the original data in Fig. 1.

Fig. 4: Time stretch data

Fig. 6: FTSD

-6-

VI. A RCHITECTURE(c) Amplitude Adjusted Fourier Transform Surrogates Data(AAFTSD)
Step 1. Prepare random numbers R(n) according to thestandard normal distribution.

1D-CNN is used for convention architecture. Figure 8 shows
the structure of 1D-CNN we used.

Step 2. Sorting R(n) in the same size relation as x(n).
Step 3. Create R′ (n) which is FTSD of sorted R(n).
Step 4. Sorting x(n) in the same size relation as R′ (n).
AAFTSD is made in this way. The correlation of original
data is broken by converting the data into AAFTSD. However
AAFTSD has similar correlation than that of RSSD. Figure 7
shows AAFTSD of the original data in Fig. 1.

Fig. 9: structure of 1D-CNN

Fig. 7: AAFTSD(d) Iterated Amplitude Adjusted Fourier Transform Surrogates Data (IAAFTSD)
Step 1. Prepare s(0) which is RSSD of original data as theinitial value.
Step 2. Calculate DFT S (i) of s(i) .
Step 3. Replace amplitude of S (i) with amplitude of origi(i)nal. Put it as S
Step 4. Calculate IDFT s(i) of S(i)Step 5. Sorting s(i).

in the same size relation as original data.

Step 6. Add 1 to i.
Step 7. Repeat until i = 7.

A convolutional layer is composed with two learnable
parameters weight and bias. This is called a filter. The filter
performs a convolution calculation along the entire input.
Equation (3) shows convolution formula.

(I ∗ K)(i, t) =∑∑m

I(m, n)K(i − m, j − n)(3)n

I is the input data and K is the filter. Maxpooling compresses
information to transform input data into a more manageable
form Two convolutional layers and two maxpooling layers areused. Drop out layer is prepared to prevent over learning. By
using the relu function, gradient disappearance hardly occurs
and easy to calculate. The probability is derived to calculateclassification results by the softmax activation function. Equations (4) and (5) show relu function and softmax function.

IAAFTSD is made in this way. IAAFTSD saves the frequency distribution and has similar correlation than that ofAAFTSD. Figure 8 shows IAAFTSD of the original data inFig. 1.

f (x) = x+ = max(0, x)(4)

exp(xl )ρ(x) = ∑ni=1 exp(xl )(5)

ρ(x) is the probability of being classified as l. n is the total
number of classes.

Fig. 8: IAAFTSD-7-

VII. S IMULATION RESULTS
Forty pieces of data are prepared. Each data was augmentationed to three types. After sampling them, surrogate data iscreated. Furthermore, each data is divided into three pieces.
In this way, the number of data became four hundred. Table
I shows the number of the train data and test data. Table IIshows the results of our research.
TABLE I: The number of the train data and test data
voice 1voice 2
voice 3train data
360360360

test data
120120120

TABLE II: Test accuracies
original data
RSSD
FTSD
AAFTSD
IAAFTSD[2] James Theiler, Bryan Galdrikian, Andr Longtin, Stephen Eubank, and J. Doyne Farmer, “Using Surrogate Data to Detect
Nonlinearity in Time Series”. In Martin Casdagli and Stephen
Eubank, editors, “Onlinear Modeling and Forecasting”, pp. 163188. Addiison Wesley, 1992. a Proceeding Volume in the Santa
Fe Institute Studies in the Science of Complexity.
[3] Daniel T. Kaplan and Richard J. Cohen, “Is Fibrillation Chaos?”.
Circulation Research, Vol. 67, No. 4, pp. 886-892, October 1990.
[4] James Theiler. “Some Comments on the correlation dimensionof 1/f α noise Physics LetterS A, Vol. 155, No. 8, 9, pp. 480493, May 1991.
[5] James Theiler, Stephen Eubank, Andre Longtin, Bryan
Galdrikian, and J. Doyne Farmer, “Testing for nonlinearity intime series: the method of surrogate data” Physica D, Vol. 58,
pp. 77-94, 1992.
[6] Dean Prichard and James Theiler. “Generating Surrogate Data
for with Several Simultaneously Measured Variables”. PhysicalReview Letters, Vol. 73, No. 7, pp. 951-954, August 1994.
[7] Thomas Schreiber and Andreas Schmitz. “Improved Surrogate
Data for Nonlin- earity Tests”. Physical Review Letters, Vol. 77,
No. 4, pp. 635-638, 1996.

test accuracy [%]
87.9
56.9
75.9
57.2
78.7

We investigate the average of ten times of test accuracy.
Table II shows each test accuracy of surrogate data are lower
than that of the original time series data. The rates of test accuracies decline of FTSD and IAAFTSD are lower than thoseof RSSD and AAFTSD. Test accuracy of IAAFTSD is higher
than that of FTSD. These results show that CNN classifies thevoices using frequency distribution and correlation.
VIII. C ONCLUSION
In this study, three types classification were carried out withsurrogate data. Then, we understood that each test accuracyof surrogate data were lower than that of the original data.
Moreover, the declined rates of test accuracies of FTSD andIAAFTSD were lower than those of RSSD and AAFTSD.
FTSD did not store frequency distribution of original data.
Therefore, it was understandable that frequency distribution
was more important than correlation for 1D-CNN. Furthermore, IAAFTSD has correlation and the most similar frequency distribution of original data. However, test accuracyof IAAFTSD was lower than that of original data. From theabove, 1D-CNN may recognize the chaoticity of the originaldata.
In the future, we will find that relationship between similarity
of correlation and test accuracy.
R EFERENCES[1] James Theiler, Paul Linsay, and David M.Rubin, “Detecting
Nonlinearity in Data with Long Coherence Times”. In Andreas
S. Weigend and Neil A. ershenfeld, editors, “Time Sereis
Prediction Forecasting the Future and Understandung the Past”,
pp.429-445. Addison Wesley, 1993. a Proceeding Volume in theSanta Fe Institute Studies in the Science of Complexity.

-8-