2023 International Symposium on Nonlinear Theory and Its Applications
NOLTA2023, September 26-29, 2023, Catania and Online

Chaotic Data Classification Methods for Residual Neural Network
Hiromu Hamanaka† Yoko Uwate Yoshifumi Nishio†Dept. of Electrical and Electronic Engineering, Tokushima University
2-1 Minami Josanjima, Tokushima, Japan
Tel: +81-88-656-7662, Fax: +81-88-656-7471
Email: {hamanaka, uwate, nishio}@ee.tokushima-u.ac.jp
Abstract– Chaotic data can be seen in a variety of data, including biological signals. Therefore, there is a need in various fields to study the classification of chaotic data. In
chaos theory, there is a method of embedding data using 3D
spatial coordinates called attractors. In this study, an attractor image was created using chaotic data, and its classification accuracy was examined using a neural network. In thisstudy, ResNet, one of the neural networks, was used to classify the attractor images of chaotic data. We proposed twotypes of attractor images: a squared attractor, which devised
a way to make the original attractor features more prominent by squaring the original chaotic data values, and an attractor with ve-locity information, which reflects the attractor's velocity information. As a result, it was confirmed thatthe new attractor image achieves higher classification accuracy than the conventional attractor image.
1.

Introduction

Neural networks have attracted a great deal of attention inrecent years and have achieved significant results. They areused in a variety of settings, including image recognition,prediction, and language processing. They are also used not
only in the academic field but also in the industrial field.
Currently, neural networks are outperforming the Humanerror rate. This allows neural networks to classify images
that are impossible for humans to classify [1]. Researchsuch as biological signal analysis is also conducted usingneural networks [2].
ResNet (Residual Neural Networks) [3] is one of the mostused neural networks in image classification. It solves thegradient loss problem, which has been a problem in the pastby reflecting information from the input in the output. In
this study, ResNet was used.
Chaos theory is used to analyze the chaotic data handled inthis study. Chaos is a signal that appears to be disordered
but is deterministic. Chaos theory is a theory that deals withphenomena in certain dynamical systems that exhibit complex aspects that are considered unpredictable due to numerical errors. Unpredictable does not mean random. It
means that the behavior of the system follows deterministic
laws, but since the integral method cannot provide a solution, the future, and past behavior must be determined bynumerical analysis. However, because of initial value
ORCID iDs Hiromu Hamanaka: 0009-0007-29573135,Yoko Uwate: 0000-0002-2992-8852,Yoshifumi
Nishio: 0000-0002-0247-0001
asensitivity (the property that if there is a very small difference in the initial state of the same system, that difference
grows exponentially with time), information at a certain
point in time must be obtained with infinite precision, anderrors in the process of numerical analysis amplify the difference between the value obtained and the true value. In
other words, prediction is virtually impossible.
Typical examples of chaotic data include biological signals[4], weather phenomena, and stock price fluctuations. Improving the accuracy of chaotic data classification is expected to make a variety of contributions. Chaotic data classification using atlases reconstructed by neural networks
has been studied [5]. Therefore, this research aims to makethe characteristics of chaotic attractors more prominent inorder to improve the accuracy of chaotic data classification.
In addition, the reason why we use chaotic data as attractor
images is that it is possible to show the differences in thebehavior of the data in a small amount of data in a noticeable way.
There are two types of attractors we propose: the first iscalled a squared attractor, which is created by squaring all
the original attractor data. The second is an attractor thatincludes velocity information. The conventional attractor
can reflect the trajectory of the chaos data, but it cannot reflect the process of the chaos data. Therefore, this attractor
allows the attractor to be displayed as a plot. This is to reflect one of the most important pieces of information, thevelocity information, in the attractor.
As a result, the two attractors proposed here show higher
classification accuracy than the conventional attractor.
2.

Residual Neural Network

ResNet (Residual Neural Networks) is one of the most usedneural networks in image classification. ResNet is theWinner network of ILSVRC2015. When multilayer neuralnetworks are used in traditional neural networks, there is aproblem called the gradient loss problem. This is a problem
in which the gradient becomes too small to find the optimalsolution, making learning impossible. Therefore, ResNet
proposed a method to learn deeper convolutional neuralnetworks (CNNs) by introducing residual blocks andlearning residuals. With this mechanism, even with aconsiderable number of deep convolutional layers (50~150layers), object recognition CNNs can be trained from largescale image data as highly accurate models without
degradation problems. Figure 1 shows a flowchart of theThis work is licensed under a Creative Commons
Attribution-NonCommercial-NoDerivatives 4.0 International.

– 285 –

CNN and ResNet. In this study, ResNet50 with 50
intermediate layers is used.

The chaotic nature of the data was confirmed at 𝜌 =28. A
high difficulty data set was created by selecting 𝜌 =27 and29, which do not show a clear difference from 𝜌 =28. The
initial values of X, Y, and Z were set from 1.0 to 2.0 inincrements of 0.1. For each combination, 1000 data wereset; 3000 data were created for the three types of 𝜌. Thesewere divided into 1800 training data and 1200 test data.
Figures 3, 4 and 5 show some of the three types of data fordifferent P. Equation 2 below shows the Lorenz equation.
(𝜎=10, 𝛽=8/3, 𝜌=27. 28. 29)ⅆ𝑥= 𝜎(𝑦 − 𝑥)ⅆ𝑡ⅆ𝑦ⅆ𝑡

= 𝑥(𝜌 − 𝑧) − 𝑦ⅆ𝑧{ⅆ𝑡= 𝑥𝑦 − 𝛽𝑧

Fig 1: CNN, ResNet.
3. Theorem of Embedding of Takens
Takens' Embedding Theorem can be used to embed 𝑥(𝑡)time-series data into a coordinate system with a timedifference [6]. Equation (1) shows the Takens' Embedding
Theorem. Figure 2 shows 1D data embedded in an attractor.
Fig 3: Part of Lorenz data (𝜌 =27).
𝑣(𝑡) = (𝑥(𝑡), 𝑥(𝑡 + 𝜏), … , 𝑥(𝑡 + (𝑛 − 1)𝜏))(1)

Fig 4: Part of Lorenz data (𝜌 =28).

Fig 2: 1D data embedded in an attractor.
4. Data-set
Equation (2) represents the Lorenz equation. It is one of
the most famous mathematical models in which chaotic
behavior is observed, was used as the chaos data in thisstudy. The three parameters of the Lorenz equation, 𝜎, 𝛽,and 𝜌 were set to 27, 28, and 29 for the 𝜌 parameter, andthe data were divided into three parts. The reason for settingthese values was to perform a high-difficulty classification.

– 286 –

Fig 5: Part of Lorenz data (𝜌 =29).

(2)

5. Proposed Attractor
Figures 6, 7, and 8 show some of the data from the traditional attractor image and proposed attractor image. Figure 6 shows the traditional attractor with the chaotic data
unmodified by applying Takens' embedding theorem. Figure 7 shows the squared attractor designed by squaring all
the original data values proposed in this study. Figure 8
shows the attractor with velocity information added to thetrajectory drawn by the attractor proposed in this study. thereason why the shape of the squared attractor is significantly different from the traditional attractor is that the negative elements of the data are replaced by positive ones. The
reason why the color of the attractor with velocity information differs from place to place is due to the plot settings.
The color of the attractor is designed to be darker at points
where the plotting speed is slow.

Fig 8: Proposed attractor (includes speed information).
6. Simulation Conditions and Results
This section presents the simulation conditions and results. Table 1 summarizes the simulation conditions. Table
2 shows the simulation results of classification accuracy forthe traditional attractor and the two attractors proposed inthis study. The simulation is terminated this time when
there is no decay in test accuracy during 150 epochs. As aresult, the squared attractor is 13% more accurate than theprevious one, and the attractor with speed information is8% more accurate than the previous one.
Table 1: Simulation Conditions
Condition Item
Application
Batch sizes
16
Epoch
300
Optimizations
Adam
Initial value of learning coefficients
0.0001Minimum value of learning coeffi5.0×10-7
cientsFig 6: Traditional attractor.

Table 2: Simulation Results
attractor
Classification
accuracy (%)
traditional Attractor
71.2
Squared attractor
84.3
Attractor with speed information
79.8

Fig 7: Proposed attractor (squared attractor).

7. Consideration
This section discusses why the attractor images proposed in this study show better classification accuracy thantraditional attractor images. First, a discussion of thesquared attractor is presented. Figures 9 to 12 show a comparison of the proposed attractor with the traditional attractor. As can be seen from all the figures, the squared attractor
has a wider line spacing than the traditional attractor. From
this, it can be said that we have succeeded in showing thefeatures of the attractor more prominently. As a result, we– 287 –believe that the feature extraction of attractor images has
become easier, leading to improved classification accuracy.
Second, we present a consideration of attractors with velocity information. As can be seen from all of Fig. 9-12, thevelocity information of the chaotic data, which has not beenshown by traditional attractors, can be shown by displaying
the attractor as a plot. In addition, it can be confirmed thatthe trajectory of the attractor is darker at locations wherethe velocity is slow due to the plot settings. From the above
points, we have succeeded in showing the velocity information, which is one of the important elements of the datathat has not been reflected by the traditional attractor, in attractor. As a result, the classification accuracy was improved. The two attractor images proposed in this study
were shown to be superior to traditional attractor images.

Fig 12: Attractor (Data④).
8. Conclusion
The objective of this study was to obtain highly accurate results by using the proposed attractor image to classify
chaotic data. Two attractors were proposed: a squared attractor designed by squaring the original attractor data andan attractor reflecting velocity information on the originalattractor. As a result, both two proposed attractors showed
more accurate classification results than those obtained using the traditional attractor.
References

Fig 9: Attractor (Data①).

[1] K. He, X. Zhang, S. Ren, and J. Sun “ Delving Deep
into Rectifiers: Surpassing Human-Level Performanceon ImageNet Classification.” 2015 IEEE International
Conference on Computer Vision, 2015, pp. 1026-1034.
[2] R. D. Labati, E. Muñoz, V. Piuri, R. Sassi, and F. Scotti,“ Deep-ECG: Convolutional Neural Networks for ECG
biometric recognition” Pattern Recognition Letters,
2019, pp.78-85.
[3] K. He, X. Zhang, S. Ren, and J. Sun “ Deep Residual
Learning for Image Recognition.” Computer Vision
and Pattern Recognition Conference, 2016, pp.770-778.

Fig 10: Attractor (Data②).

[4] O. H. Mayumi, T Miao, K. Sato, and K. Tanaka “Development of a Self-Check System for Mental Health Using a Pulse Wave Mouse.” International Conference on
Software Technologies, 2007, pp 239-248.
[5] T. Takata, Y. Uwate, H. Takao and Y. Nishio, “Time Series Classification of Vibration Data Using Attractors
Made from Multidimensional Space by Convolutional
Neural Network.” International Forum on Advanced
Technologies, 2020.
[6] K. Aihara, T. Ikeguchi, Y. Yamada, and M. Komuro,“Fundamentals and applications of chaotic time series
analysis.” Industrial book, p.23, 2000.

Fig 11: Attractor (Data③).

– 288 –