Clustering Feature Extraction of Chaotic Circuitswith Learning on Coupling WeightsϮϬϮϯ//ŶƚĞƌŶĂƚŝŽŶĂů^ǇŵƉŽƐŝƵŵŽŶŝƌĐƵŝƚƐĂŶĚ^ǇƐƚĞŵƐ;/^^ͿͮϵϳϴͲϭͲϲϲϱϰͲϱϭϬϵͲϯͬϮϯͬΨϯϭ͘ϬϬΞϮϬϮϯ/ͮK/͗ϭϬ͘ϭϭϬϵͬ/^^ϰϲϳϳϯ͘ϮϬϮϯ͘ϭϬϭϴϮϭϯϱ

†Yoko Uwate† , Thomas Ott‡ and Yoshifumi Nishio†
Dept. of Electrical and Electronic Engineering, Tokushima University
2-1 Minami-Josanjima, Tokushima, Japan
Email: {uwate, nishio}@ee.tokushima-u.ac.jp‡
Zurich University of Applied Sciences
8820 Waedenswil, Switzerland
Email: thomas.ott@zhaw.ch

Abstract— If clustering can be performed in a continuous
system, there is potential usefulness in processing speed forlarger scale clustering. We have proposed clustering method of
coupled chaotic circuit networks with learning. In this study,
we propose a method to extract the center of a cluster bystabilizing synchronization through learning in networks wheresynchronization is not stable due to chaos.

I. I NTRODUCTION
Memristors were proposed by Chua in 1971 [1]. Memristors are passive devices that store a passing charge andchange resistance accordingly. Later, in 2008, Hewlett-Packard
Laboratories developed a memristor using a thin ﬁlm oftitanium dioxide, which attracted attention as a fourth circuit
element. In recent years, the development of brain computers
using memristors has been actively studied in the ﬁeld of
neuromorphic engineering [2], [3]. Due to the characteristics
of memristors, many computing methods have been proposedthat use memristors as synapses [4], [5]. Synapses are connections between neurons that play an important role in signaltransmission.
One of the basic rules of learning in the brain is the Hebbianrule. This rule has proposed by Hebb in 1949 [6]. This is afundamental rule of learning and long-term memory basedon the hypothesis that synapses become more efﬁcient attransmitting electrical stimuli each time a neuron ﬁres, andconversely, less efﬁcient if they do not ﬁre for a long time.
The neurobiological mechanism of Hebb’s rule has also been
identiﬁed. The electrical signal enhances the coupling strengthand strengthens the part of the system that communicatesmore.
Our research group has been investigating synchronization
phenomena in coupled oscillators and coupled chaotic circuits.
While many studies of synchronization in networks have usedmathematical models for nodes, we have focused our researchon electronic circuit models. The reasons for this are, ﬁrst,
that we would like to deal with synchronization observed as aphysical phenomenon that actually exists, and second, that anelectronic circuit model was necessary when we considered
engineering applications in the future. Now that memristor
devices are in practical use, replacing the coupling in our previously proposed coupled oscillator network with memristors

is expected to have applications in modeling more complex,
higher-dimensional nonlinear systems. Possible engineering
applications include modeling of cyber-physical systems.
In recent years, much data analysis has been importantin marketing and other areas. One of the data analysis isclustering. Clustering is a method of classifying similar dataand is used in various ﬁelds such as data mining, image
processing and biological data analysis [7]-[9].
In previous studies, we have proposed a method of clustering chaotic circuits on a two-dimensional plane using synchronization and conﬁrmed its effectiveness [10]-[13]. As afurther improvement, we are also investigating clustering when
the coupling between circuits is changed as in Hebbian rulelearning [14], [15].
However, the dynamics of the proposed coupled chaotic
network with learning had not been investigated. Therefore,
in this study, we focused on how the coupled chaotic circuit
network with learning makes synchronization decisions, andinvestigated it in a network containing three clusters. By
using the computer simulations, we conﬁrm that the proposed
method is a suitable system for extracting cluster centers bylearning coupling weights.
II. C HAOTIC C IRCUITS N ETWORKS
The nodes of the network are represented by chaotic circuits.
The chaotic circuit is shown in Fig. 1. This chaotic circuit has
been studied in the references [16]-[18]. This circuit composedof three memory elements, one linear negative resistanceelement, and one nonlinear resistance element consisting of
two diodes.
The approximate I − V characteristics of the nonlinear
resistance element are indicated by the following equation,
where the parameter rd is the slope of the nonlinear resistance.
rdvd (i2 ) =
2

 


 


i 2 + V  − i 2 − V  .



rd
rd 

By changing the variables and using the parameters,

√
C
L1 C
i1 =
V x; i2 =V y; v = V z;
L1
L2

Authorized licensed use limited to: Yoshifumi Nishio. Downloaded on August 15,2023 at 12:05:08 UTC from IEEE Xplore. Restrictions apply.

(1)

2

1.5

1

y

0.5

0

-0.5

-1

(a) Chaotic circuit.

-1.5(b) Chaotic attractor.

"cluster1.dat""cluster2.dat""cluster3.dat"-2-2

Fig. 1.



Chaotic circuit model.

√



C
L1 C
L1
= α;
= β; rd= δ; t = L1 Cτ
(2)
L1
L2
L2
The normalized circuit equations are obtained as follows.
⎧
dxi
⎪
⎪
= αxi + zi
⎪
⎪
dτ
⎪
⎪
⎪
⎪
⎪
dyi
⎪
⎪
⎨
= zi + f (y)
dτ
(3)
⎪
⎪
N
⎪
⎪ dzi
⎪
⎪
= −xi − βyi −
γij (zi − zj )
⎪
⎪ dτ
⎪
⎪
j=1
⎪
⎩
(i, j = 1, 2, · · ·, N )rwhere f (y) is described as follows:δf (y) =
2

-1.5

 



 

y + 1  − y − 1  .



δ
δ

(4)

In the computer simulations, we set the parameters to be
α = 0.460, β = 3.0 and δ = 470. The characteristic function
f (y) can be described as a three-segment piecewise-linear
function. In this study, the value of γij reﬂects the distance
between the circuits in an inverse manner, as described using
the following equation:
γij =

g
.
(dij )2(5)

Here, dij denotes the Euclidean distance between the i − thcircuit and the j − th circuit, while g is a scaling parameterthat determines the coupling strengths.
III. C IRCUITS A RRANGEMENT AND L EARNING P ROCESS
A. Circuits Arrangement
Here, we consider the case of 100 chaotic circuits arrangedin 2-dimensional space. The number of clusters is set tothree, cluster-1, 2 and 3 consist of 50, 30, and 20 chaotic
circuits, respectively. The chaotic circuits are randomly placed
according to a normal distribution (σ) from the center of eachcluster. The circuit arrangement with three clusters is shownin Fig. 2.
The purpose of clustering for this circuit layout is not todivide it into three clusters, but to extract as many nodes as
possible from the central part of the cluster.

-1

-0.5

0
x

0.5

1

1.5

2

Fig. 2. Circuit arrangement with three clusters (N =100, the number ofclusters: 3, red points: 50 (cluster-1), green points: 30 (cluster-2), blue points:
20 (cluster-3)).

B. Learning Process
In the previous study, we have proposed the clustering
method that applies Hebbian rule as well as the determinationof clusters by synchronization of chaotic circuit networks. The
proposed clustering method is based on the concept of the
Hebb rule. As the clustering algorithm, it makes the couplingstrength of edges with high synchronization between circuits
stronger and the coupling of edges with weak synchronization weaker. The clustering method with learning of coupledchaotic circuits is explained below. (The conceptual diagramof the computer simulation is shown in Fig. 3.)[step-1] At the initial state, all nodes are fully connectedwith coupling strengths depending on distance. The scaling
parameter g is set to g = 0.00008.
[step-2] After a transient phase, we apply two rules for asequence of generations. Each generation has length τh =10, 000.
• (check synchronization:) In order to check whether two
nodes are alike, we calculate the synchronization ratio
for every pair of oscillators. If the synchronization ratio
is larger than 60%, the corresponding coupling strength
becomes stronger with Δγ = 0.00001.
In order to analyze the synchronization ratio, we deﬁne
a synchronization state as|xk − xn | < 0.3(k ∈ Sn )[step-3] Step-2 is repeated until 100 iterations are reached(H = 100).
[step-4] At the ﬁnal state (H = 100), we check the synchronization ratio for every pair of oscillators.
IV. S IMULATION R ESULTSFirst, we investigated changes in the number of synchronized edges of coupled chaotic circuit networks. Figure 4
shows the change in the total number of synchronized edges
between the clusters. From this ﬁgure, the proposed methodwith learning has less variation of synchronized edges with thesimulation time. In contrast, the conventional method shows alarge ﬂuctuation of synchronized edges.

Authorized licensed use limited to: Yoshifumi Nishio. Downloaded on August 15,2023 at 12:05:08 UTC from IEEE Xplore. Restrictions apply.

Ttransient state H1 H2 H3 H4

H100

450
Number of synchronized edges in Cluster-1

τh τh τh τh ... τhτtsτ (iteration)

Applying Hebbian Learning
voltage differences: fabs(xi-xj) < 0.3
synchronization ratio > 60%
synchronization
coupling: γ +=0.00001

un-synchronization
coupling: γ -=0.00001
Parameters:

400
350
300
250
200
150
Proposed method (w/ Learning)
100
Conventional method500τts=10,000τh=10,000
Hebbian Learning (H)=100

0

10

20

30

40

50

60

70

80

90

100

H

(a) cluster-1.
160

Learning process for clustering.

Number of synchronized edges in Cluster-2

Fig. 3.

The evolution of synchronized edges within the three clusters is shown in Fig. 5. It can be seen that for all clusters,
the proposed method with learning has less variation in thenumber of edges, as well as the variation of synchronizededges for the whole cluster.

140
120
100
80
60

0

0

10

20

30

40

50

60

70

80

90

100

H
500

(b) cluster-2.
400
80
Number of synchronized edges in Cluster-3

Total number of synchronized edges in cluster

Conventional method

20

600

300
Proposed method (w/ Learning)200

Conventional method
100

0

0

10

20

30

40

50

60

70

80

90

100

H

Fig. 4.

Proposed method (w/ Learning)

40

Total number of synchronized edges in the whole cluster.

70
60
50
40
30

Proposed method (w/ Learning)

20

Conventional method

10
0

0

10

20

30

40

50

60

70

80

90

100

H

(c) cluster-3.
Fig. 5.

Number of synchronized edges in the cluster.

450
Proposed method (w/ Learning)
Number of miss edges (between clusters)

Figure 6 shows the change in synchronization edges acrossclusters. This edge is considered ”miss edge” from the clustering concept. The large number of these missed edges is a
cause of poor clustering. From the results, it can be said thatboth methods have a large variation of missed edges, however
the proposed method clearly has a higher number of times withfewer missed edges. Therefore, we consider that learning of
the coupling weights inﬂuences the control of missed edges.
Average of number of synchronized edges in each category
is summarized in Tab. I. It can be seen that the number ofsynchronized edges within clusters is higher for the conventional method. On the other hand, the proposed method has
fewer synchronized edges between clusters (miss edges) andis more effective for clustering.
Figure 7 shows examples of the results of displaying synchronized edges for the proposed and conventional methods.
In the proposed method, there are synchronized edges in thecenter of each cluster and the number of missed edges between
clusters is small. In the conventional method, there are many

400
Conventional method
350
300
250
200
150
100
50
0

0

10

20

30

40

50

60

70

80

90

100

H

Fig. 6.

Number of synchronized edges between clusters (miss edges).

Authorized licensed use limited to: Yoshifumi Nishio. Downloaded on August 15,2023 at 12:05:08 UTC from IEEE Xplore. Restrictions apply.

TABLE I
AVERAGE OF NUMBER OF SYNCHRONIZED EDGES IN EACH CATEGORY
Type
Total
(in the cluster)
Cluster-1
Cluster-2
Cluster-3
Miss(between the clusters)

Proposed method
376.00Conventional method458.77225.63
99.74
50.63
76.75

290.38112.5455.85
181.51synchronized edges throughout, but the number of missed
edges between clusters is also high. These results suggestthat the proposed method with learning process is effective
in extracting cluster features because synchronized edges areconcentrated in the central part of the clusters.
(a) proposed method with learning.
2"cluster1.dat""cluster2.dat""cluster3.dat""edge_hebb0.8.dat"

1.5

1

Y

0.5

0

-0.5

-1

-1.5

-2
-2

-1.5

-1

-0.5

0
X

0.5

1

1.5

2

(a) proposed method with learning.
2"cluster1.dat""cluster2.dat""cluster3.dat""edge_nohebb.dat"

1.5

1

0.5

Y

(b) conventional method without learning.
0

Fig. 8.

-0.5

One example of phase difference between 1st circuit to the others.

-1

-1.5

V. C ONCLUSION

-2
-2

-1.5

-1

-0.5

0
X

0.5

1

1.5

2

(b) conventional method without learning.
Fig. 7.

One example of clustering result by displaying synchronized edges.

Finally, an example of a phase difference result is shownin Fig. 8. This ﬁgure shows the phase of the ﬁrst circuitwith reference to the other circuits in Lissajous. It can beseen that the proposed method synchronizes with only a few
circuits, while the conventional method synchronizes withmany circuits. Therefore, this is a supporting result that the
proposed method is effective in cluster extraction.

In this study, we focused on the dynamics of the previously
proposed clustering method using synchronization of coupled chaotic circuit networks with learning, and investigatedchanges in synchronized edges. Because of the chaotic circuit,synchronization in the network is not stable, but by learning
the coupling weights, we were able to reduce the variationin the number of synchronization edges. We also found thatthe proposed method has fewer synchronized edges between
clusters and is more likely to extract the center of the cluster.
Future work includes developing a method for accurate
clustering and investigating the case where the number of clusters is increased. Clustering using memristor-coupled chaotic
circuits is also one of our future research topics.

Authorized licensed use limited to: Yoshifumi Nishio. Downloaded on August 15,2023 at 12:05:08 UTC from IEEE Xplore. Restrictions apply.

R EFERENCES[1] L. O. Chua, “Memristor -The Missing Circuit Element,” IEEE Transactions on Circuit Theory, vol. ct-18, no. 5, 1971.
[2] Y. Li, Z. Wang, R. Midya, Q. Xia and J. J. Yang, “Review of memristor
devices in neuromorphic computing: materials sciences and device
challenges,” Physics D: Applied Physics, vo51, no. 50, 2018.
[3] Y. Zhang, Z. Wang, J. Zhu, Y. Yang, M. Rao, W. Song, Y. Zhuo, X.
Zhang, M. Cui, L. Shen, R. Huang and J. J. Yang, “Brain-inspired computing with memristors: Challenges in devices, circuits, and systems,”
Applied Physics Reviews, https://doi.org/10.1063/1.5124027, 2020.
[4] W. Huh, D. Lee and C-H. Lee, “Memristors Based on 2D Materials as an
Artiﬁcial Synapse for Neuromorphic Electronics,” Advanced Materials,
https://doi.org/10.1002/adma.202002092, 2020.
[5] K. Liao, P. Lei, M. Tu, S. Luo, T. Jiang, W. Jie and J Hao, “Memristor
Based on Inorganic and Organic Two-Dimensional Materials: Mechanisms, Performance, and Synaptic Applications,” ACS Appl. Mater.
Interfaces, https://doi.org/10.1021/acsami.1c07665, 2021.
[6] D. O. Hebb, “Organization of Behavior,” 1949.
[7] Daniel T. Larose & Chantal D. Larose “Discovering Knowledge in Data:
An Introduction to Data Mining,” John Wiley and Sons, 2014.
[8] N. Dhanachandra, K. Manglem & Y. J. Chanu, “Image Segmentation
Using K -means Clustering Algorithm and Subtractive Clustering Algorithm,” Procedia Computer Science, Vol. 54, pp. 764-771, 2015.
[9] G. Manogaran, V. Vijayakumar, R. Varatharajan, P. M. Kumar & R. Sundarasekar, “Machine Learning Based Big Data Processing Framework
for Cancer Diagnosis Using Hidden Markov Model and GM Clustering,”
Wireless Personal Communications, Vol. 102, Issue 3, pp. 2099–2116,
2018.
[10] Y. Takamaru, H. Kataoka, Y. Uwate & Y. Nishio, “Clustering Phenomena
in Complex Networks of Chaotic Circuits,” Proc. ISCAS’12, Vol. 22, pp.
320-334, 2012.
[11] Y. Takamaru, Y. Uwate, T. Ott & Y. Nishio, “Dependence of Clustering
Patterns on Density of Chaotic Circuits in Networks,” RISP Journal of
Signal Processing, vol. 17, no. 4, pp. 103-106, Jul. 2013.
[12] K. Ago, Y. Uwate & Y. Nishio, “Investigation of Clustering in Complex
Networks Using Coupled Chaotic Circuits,” Proc. of IFAT’15, pp. 101102, 2015.
[13] K. Oi, Y. Uwate & Y. Nishio, “Synchronization and Clustering in
Coupled Parametrically Excited Oscillators with Small Mismatch,” Proc.
of ISCAS’15, pp. 910-913, 2015.
[14] Y. Uwate, T. Ott and Y. Nishio, “Synchronization and Clustering of
Chaotic Circuit Networks with Hebbian Rule,” Proc. of NCSP’22, pp.
373-376, Feb. 2022.
[15] Y. Uwate and Y. Nishio, “Clustering in Globally Coupled Chaotic
Circuits with Changing Weights,” Proc. of ISOCC’22, 2022.
[16] Y. Nishio, N. Inaba, S. Mori & T. Saito, “Rigorous Analyses of Windows
in a Symmetric Circuit,” IEEE Transactions on Circuits and Systems,
vol. 37, no. 4, pp. 473-487, Apr. 1990.
[17] C. Bonatto & J. A. C. Gallas, “Periodicity Hub and Nested Spirals in
the Phase Diagram of a Simple Resistive Circuit,” Phys. Rev. Lett., 101,
054101, Aug. 2008.
[18] A. Koseska, E. Volkov, & J. Kurths, “Real-World Existence and Originsof the Spiral Organization of Shrimp-Shaped Domains,” Phys. Rev. Lett.,
105, 074102, Aug. 2010.

Authorized licensed use limited to: Yoshifumi Nishio. Downloaded on August 15,2023 at 12:05:08 UTC from IEEE Xplore. Restrictions apply.