IEICE TRANS. FUNDAMENTALS, VOL.E99–A, NO.3 MARCH 2016

742

PAPER

Multi-Layer Perceptron with Pulse Glial Chain
Chihiro IKUTA†a) , Nonmember, Yoko UWATE†b) , Yoshifumi NISHIO†c) , Members,
and Guoan YANG††d) , Nonmember

SUMMARY Glial cells include several types of cells such as astrocytes,
and oligodendrocytes apart from the neurons in the brain. In particular, astrocytes are known to be important in higher brain function and are therefore sometimes simply called glial cells. An astrocyte can transmit signals
to other astrocytes and neurons using ion concentrations. Thus, we expectthat the functions of an astrocyte can be applied to an artificial neural network. In this study, we propose a multi-layer perceptron (MLP) with apulse glial chain. The proposed MLP contains glia (astrocytes) in a hidden
layer. The glia are connected to neurons and are excited by the outputsof the neurons. The excited glia generate pulses that aﬀect the excitationthresholds of the neurons and their neighboring glia. The glial network provides a type of positional relationship between the neurons in the hidden
layer, which can enhance the performance of MLP learning. We confirm
through computer simulations that the proposed MLP has better learning
performance than a conventional MLP.
key words: multi-layer perceptron, glial chain, classification

1. Introduction
The adult human brain and the rest of the central nervous
system comprise up to one trillion nerve cells, including excitable nerve cells and synapses. The cells in the central nervous system are classified into two types, namely, neurons
and glia, including astrocytes and oligodendrocytes. Many
studies have investigated the biological features of neurons
and their functions. Neurons can transmit and gather electrochemical signals from each other, thereby accomplishing such brain tasks as thinking and memory. So far the
astrocytes of the glia had not been investigated deeply because the functions of astrocytes in the brain were diﬃcult
to investigate. Hence, these cells were long considered to be
merely support cells for the neurons. Recently, researchers
discovered that an astrocyte can transmit signals by adjusting the concentration of ions in the glia [1]–[3]. In addition,
an astrocyte has many receptors for ions such as adenosine
triphosphate (ATP), glutamic acid (Glu), and calcium ions
(Ca2+ ). These ions are important for brain function because
neurons also use ATP and Glu in synapses, and astrocytes
Manuscript received February 19, 2015.
Manuscript revised August 28, 2015.
†
The authors are with Tokushima University, Tokushima-shi,
770-8506 Japan.
††
The author is with Xi’an Jiaotong University, Shaanxi
710049, China.
a) E-mail: ikuta@ee.tokushima-u.ac.jpb) E-mail: uwate@ee.tokushima-u.ac.jpc) E-mail: nishio@ee.tokushima-u.ac.jpd) E-mail: gayang@mail.xjtu.edu.cn
DOI: 10.1587/transfun.E99.A.742

generates Ca2+ concentration wave [4], [5]. Among them,
we have focused on Ca2+ because astrocytes induce ATP, Dserine, etc. in accordance with changes in Ca2+ concentration [6]–[9]. ATP and D-serine directly influence the membrane potentials of neurons. A Ca2+ concentration wave istransmitted to the astrocytes in a wide range of the brain.
The neurons are also influenced by the Ca2+ concentration
wave through the ions emitted from the astrocytes. Thus, astrocyte functions relate closely to brain functioning. Therefore, on the basis of the astrocyte functions, we consider themodeling of astrocyte functions in an artificial neural network (ANN).
Recently, many kinds of ANN models have been proposed and widely applied in modern technology. A multilayer perceptron (MLP) is a feed forward ANN comprising layers of neurons. In an MLP, the neurons comprise theneuron layers, and a neuron in one layer connects only withneurons in other layers. Then, we can obtain the ideal inputoutput relation of an ANN by setting up weights of network
connections. In general, a backpropagation (BP) algorithm
is used for determining connection weights [10]. Using the
BP algorithm, an MLP can be used for various tasks suchas pattern recognition, machine learning, and data mining.
However, MLPs involve two problems. First, an MLP istrapped in a local minimum because the BP algorithm uses
the steepest decent method. Second, an MLP does not haveconnections between neurons in the same layer. Furthermore, the neurons in a layer do not have position dependency in an MLP. Simulated annealing (SA) [11] can beused to improve the BP algorithm [12], [13]. In SA, the network searches a wide range with the objective function, andthe searching range decreases with time. In this way, the SA
can find a global solution. Moreover, a modeling of glial
functions in an ANN was reported by A.B. Porto-Pazos, A.
Alvarellos, and others [14]–[16]. These studies showed thatglia improved the performance of a feed forward neural network. That work drew attention to the relationship between
glia and neurons. In our model, we focus on the relationshipamong glia. We consider that a glial network can provide
position relationships to neurons.
In this study, we propose a new MLP with a pulse glial
chain∗ (PGC) inspired by the functions of astrocytes. All∗
We reported the concept and the basic investigation of a random glial network in an IEICE letter [18]. In the previous study, a
glial cell generated a uniform random noise, whose amplitude waschanged by the output of the connected neuron.

c 2016 The Institute of Electronics, Information and Communication Engineers
Copyright IKUTA et al.: MULTI-LAYER PERCEPTRON WITH PULSE GLIAL CHAIN

743

glia are individually connected to the neurons in the hidden layer of the MLP and have an interactive eﬀect in the
glial network. A neuron output is greater than a glial excitation threshold, hence it can excite a glial cell. The excited
glial cell generates a pulse, and the pulse influences the excitation threshold of the neuron and the states of the neighboring glia. The glial pulse is attenuated exponentially andinduces a pulse chain. The neighboring glia are excited andgenerate pulses subsequent to those of the first excited glial
cell, thereby propagating the pulse in the glial network. The
generated pulse is transmitted to the connected neuron andinfluences the excitation threshold of this neuron. This pulse
accelerates the convergence of the weight update of the connected neuron because the pulse retains the large value ofthe neuron. A neuron output greater than the glial excitationthreshold almost converges to a local optimum value, whichis therefore retained by the pulse. The neighboring neurons
obtain energy from the pulses propagated by other glia. The
influence of a pulse is independent of the states of the neighboring neurons. Therefore, the neighboring neurons deviate
from the local optimum values of their neighboring neurons.
We consider that the pulse glial chain (PGC) accelerates thelearning of the entire MLP network. This study confirmsthat the MLP with PGC has better performance than thestandard MLP in learning time series, solving the Proben1[17], and solving the two-spiral problem (TSP).
2. Proposed Method
In this study, we propose a PGC inspired by the features ofastrocytes. The glia are individually connected to neurons
in the hidden layer and influence neighboring glia. We connect glia only to the neurons in the hidden layer, because thenumber of neurons in the input and output layers depends
on the simulation tasks. In addition, we consider that having connections within the same layer is important for the
MLP. The conventional MLP already has connections between diﬀerent layers. For the reasons stated above, we connect glia to neurons only to compare the glial eﬀects in several diﬀerent tasks. The proposed MLP with PGC is shownin Fig. 1.

Fig. 1

MLP with PGC.

2.1 Glial Pulse Chain
All glia are connected to the nearest neurons in the hidden
layer of an MLP and generate pulses according to the outputs of the connected neurons. Here, we define the glial
output function as in Eq. (1).
⎡⎢⎢⎢1, {θn < yi ∪ ψi+1 (t − D) = 1∪
ψi (t + 1) = ⎢⎢⎢⎢⎣ ψi−1 (t − D) = 1)} ∩ (t − τi > θg ) , (1)γψi (t), else,
where ψ is the output of a glial cell, i is the position number of a glial cell in the hidden layer, y is the output of theconnected neuron, θn is the excitation threshold of the glial
cell, D is the delay time of a glial eﬀect, τ is the time ofthe previous pulse generation, θg is the period of inactivity,and γ is the attenuation parameter. In the excitation condition of Eq. (1), yi > θn indicates that the neuron excites
the glial cell when the neuron output is greater than θn andhas a constant value; thus, it is not changed with iteration.
The glial cell is also excited by receiving a glial pulse fromits neighboring glial cell, because the glial pulse requires
time D to be transmitted to its neighboring glia, a condition described by ψi+1 (t − D) = 1 and ψi−1 (t − D) = 1.
The glial cell has an inactivity period θg ; hence, t − τi > θgmust be satisfied to generate a glial pulse. The glias do notlearn based on the BP algorithm, even though the neurons
are trained based on that algorithm. The generation patternof glial pulses changes dynamically during MLP learning;
an example pattern is shown in Fig. 2. In the figure, the first
glial cell is excited and generates its corresponding pulse
with the excitation condition of yi > θn as shown in Eq. (1).
The first glial cell receives neuron output greater than θn ,and a pulse is propagated to the neighboring glial cells suchthat the transmitted pulse of the first glial cell excites theneighboring glial cells according to ψi+1 (t − D). Thus, theneighboring glial cells generate the pulse with a delay fromthe first glial cell. Furthermore, the tenth glial cell is excited
at a time similar to that of the first glial cell, and the eﬀect of
the tenth glial cell is also propagated to the neighboring glia.
Finally, the seventh glial cell is excited independently of theinfluences of the tenth and first glial cells. The seventh glial
cell receives the pulse of the tenth glial cell; however, it is
not excited by using these pulses, because the seventh had
already begun the period of inactivity (i.e., to t − τi > θg .).
The change in the generation pattern of pulses depends on
the neurons’ outputs, which are changed by BP learning,
and the generation pattern Fig. 2(b) is also diﬀerent from
Fig. 2(a). From this example, we see that the generation pattern of pulses changes dynamically during learning.
Next, we focus on the responses of two glial cells. The
outputs of the connected neurons and the responses of twoneighboring glial cells are shown in Fig. 3. In this figure,glial cells one and two are connected to neurons one andtwo, respectively. Glial cell one neighbors glial cell two;
thus, they influence each other. During the data iterations,the input learning data are switched in sequence with the

IEICE TRANS. FUNDAMENTALS, VOL.E99–A, NO.3 MARCH 2016

744

2.2 Neuron Updating Rule
A neuron is multi-input and single-output, and we canchange its output by tuning connection weights between
neurons. The standard updating rule for a neuron is defined
as in Eq. (2).
⎛ n
⎞
⎜⎜⎜
⎟⎟⎟
yi (t + 1) = f ⎜⎜⎜⎝ wi j (t)x j (t) − θi (t)⎟⎟⎟⎠ ,(2)
j=1

Fig. 2

An example of glial pulses (D = 5).

where y is the output of the neuron, w is the weight of theconnection, x is the input of the neuron, and θ is the excitation threshold of the neuron. In this equation, the weight ofthe connection and the excitation threshold of the neuron arelearned based on the BP algorithm. Next, we show our proposed neuron updating rule. Because glia may increase themembrane potential of a neuron in a biological system, we
add the glial eﬀect ψ to the inner state of the neuron. (Glu,ATP, etc.) [19], [20], and these ions influence the membrane potential of the neuron. The inner state of the neuron
increases as a result of the glial eﬀect. In this study, this updating rule is used only for the neurons in the hidden layer.
Since we emphasize the position relationship of neurons inthe same layer, we arrange the glias in one dimension to enable observation of the position relationship of neurons. The
updating rule proposed in this study is described by Eq. (3).
⎛ n
⎞
⎜⎜⎜
⎟⎟⎟
⎜
(3)
yi (t + 1) = f ⎜⎜⎝ wi j (t)x j (t) − θi (t) + αψi (t)⎟⎟⎟⎠ ,j=1

where α is the weight of the glial eﬀect. The peak of thegenerated pulse changes according to α. In this study, wechoose an optimal value of α using heuristic search. In
this expression, the weight of connection and the excitationthreshold of the neuron are obtained based on the BP algorithm just as in the standard neuron updating rule. However,
the glial eﬀect is independent of learning. ψ is updated using
Eq. (1). Equations (2) and (3) are used as sigmoidal functions to a neuron activating function expressed as in Eq. (4).
Fig. 3 Diﬀerent patterns of the two glia excitations (θn = 0.9, D = 5,
γ = 0.8, and α = 0.5).

iterations; thus, the output of a neuron changes dynamicallywith the iterations. The output of neuron one is greater thanthe excitation threshold of the connected glial cell one, andmeanwhile, glial cell one is also excited by the output of
neuron one. Actually, we just observe the pulse generationin glial cell one, because the output of neuron two is lessthan the excitation threshold of the connected glial cell two.
However, glial cell one influences the state of glial cell two,and glial cell two is excited by glial cell one. Thus, glial cell
two generates the pulse with a delay.

f (a) =

1,
1 + e−a(4)where a is the inner state of the neuron. Several activating
functions, such as tangent hyperbolic, using absolute value
have been proposed for modeling MLP performance [21].
In this study, we use the sigmoidal function as the activating
function. This function is basic and is used frequently [22][23] because the derivative can easily be calculated, and it
can easily be applied to the BP algorithm. In our method, theactivation function does not have an essential role; therefore,
we can change the sigmoidal function to other functions.
3. Simulations
In this section, we show our experimental results based oncomputer simulation of three diﬀerent tasks. We use six

IKUTA et al.: MULTI-LAYER PERCEPTRON WITH PULSE GLIAL CHAIN

745

addition, we obtain the proper parameters of the PGC basedon a large number of simulation experiments with various
parameter values. Thus, the parameters are determined as
θn = 0.9, D = 1, θg = 45, and γ = 0.8.
Here we use the mean square error (MSE) described by
Eq. (5) as the error evaluation.
1 (T n − On )2 ,N n=1
N

MS E =(5)

where T is the target value, O is the output of MLP, andN is the number of learning data. In this study, we use theaverage, the minimum, the maximum, the standard deviation, and the accuracy of the MSE to evaluate the validation
accuracy of the experimental result. MLP performance isbetter when the MSE is smaller and the validation accuracy
is larger. We calculate the validation accuracy using a kfold cross-validation estimate. Here, we fix the value of k toten, and obtain the validation accuracy for each simulation,
and we also apply a Wilcoxon signed-rank test to the experimental results. The Wilcoxon signed-rank test is one of thenonparametric tests. We compare two results obtained from
diﬀerent methods and obtain a sampling probability. If thesampling probability is less than 0.05, we conclude that theresults have a broad distinction. In contrast, if the sampling
probability is greater than 0.05, we are not sure whether two
results are good or not.
Fig. 4

Network structures of five MLPs.

types of MLP for the performance comparison.
(1) Conventional MLP(2) MLP with random noise(3) MLP with SA noise(4) MLP with randomly timed pulses(5) MLP with glial pulse(6) MLP with PGC
The network structures for the six types of MLP models areshown in Fig. 4. A conventional MLP (1) does not have an
external unit. An MLP with random noise (2) has a uniform
random noise that influences the excitation threshold of neurons in the hidden layer. An MLP with SA noise (3) has
a normally distributed noise that influences the excitationthreshold of neurons in the hidden layer and whose amplitude decreases exponentially with iteration. An MLP withrandomly timed pulses (4) has pulses at random times in theneurons in the hidden layer. In such an MLP, the neurons
in the hidden layer have a pulse that influences the excitation threshold of the neuron, and this pulse is generated at
random. In an MLP with a glial pulse (5), the glia respondto the output of the connected neuron in the same manneras the proposed MLP with PGC (6); however, the generated
pulse is not propagated to neighboring glia. Thus, the generated pulse only increases the excitation threshold of theconnected neuron. In the simulations, the optimal noise andpulse amplitude of each method is decided heuristically. In

3.1 Task 1: Learning Time Series
For the first task, we use successive chaotic time series asdata sets, with the skew tent map for the generation of thetime series described by Eq. (6).
⎧ 1 φ (t) (0 ≤ φ (t) ≤ A )
i
i
i
⎪
⎪
⎨ Ai
,
φi (t + 1) = ⎪
⎪
⎩ 1
(1
−
φ
(t))
(A
<
φ
(t)
≤
1)
i
i
i
1−Ai

(6)

We use A1 = 0.45 and A2 = 0.55. The generated chaotic
time series vary with the value of A. The data set includes
the two successive chaotic time series obtained with A1 andA2 . An example of the successive chaotic time series isshown in Fig. 5. In this simulation, the MLP comprises threelayers (connected 4-40-1), and the simulation conditions are
as shown in Table 1. The column headings in the table indicate the following: the number of inputs is indicated for
one round of learning, the number of classifications is for
the Boolean classification of the input data, the training data
sets are for the number of data sets used for learning, the unlearned data sets are for the number of data sets used to show
the performance of unlearning data sets, and the validation
is for the number of data sets used for validation obtained
from ten-fold cross-validation.
We input four successive chaotic time series (φi (t),φi (t + 1), φi (t + 2) and φi (t + 3)) to the neurons in the input layer, and the MLP learns the correlating classification.
And also, φ1 (t)-φ1 (t + 3) and φ2 (t)-φ2 (t + 3) are switched

IEICE TRANS. FUNDAMENTALS, VOL.E99–A, NO.3 MARCH 2016

746
Table 3ries.

Wilcoxon signed-rank test of learning performance of time se(1)
0.000
0.009
0.000
0.000
0.000

(1)
(2)
(3)
(4)
(5)
(6)

Table 4

Fig. 5

Successive chaotic time series obtained by skew tent map.

Table 1 Conditions of chaotic time series.
Num. of
Num. ofTraining
Unlearned Validation
inputsclass.
data sets
data sets
4
2
200
200
20

(1)
(2)
(3)
(4)
(5)
(6)

Table 2
(1)
(2)
(3)
(4)
(5)
(6)

Average0.02868
0.02464
0.028410.02804
0.028400.00756

Maximum
0.140080.120160.140110.140030.140100.11087

Std. Dev.
0.02650
0.02172
0.025160.025480.026290.01347

Accuracy
89.83690.05488.75889.90689.82194.319with time. We prepare ten data sets with each data set including 200 successive chaotic time series, obtained fromten diﬀerent initial values. We use ten fold cross-validationfor validation accuracy; thus, 180 and 20 data are used forlearning and validation, respectively. We obtain the simulation results from 100 trials for each data set; thus, the total
number of trials is 1,000, with each trial having 50,000 iterations.
Table 2 shows the statistical results from 1,000 trials. In
this simulation, the validation accuracies are approximately
90 in the MLPs. From Table 2, we can see that the MLP
with PGC (6) has the best average. For the maximum, theMLP with PGC (6) obtains the best value of 0.11087, and its
standard deviation 0.01347, the smallest of all. However, forthe minimum, the MLP with random noise (2) obtains thebest value of 0.00000, though the other MLPs also obtainadequate results.
Table 3 shows the Wilcoxon signed-rank test for the
MLPs. The sampling probability values are less than 0.05;
thus, we can be sure of the diﬀerences in the average performance of the MLPs in Table 2.
Table 4 shows a classification performance of unlearning time series. We prepare the unlearning time series byusing diﬀerent initial values, with the number of data sets
for unlearning time series being the same as for the learning
data sets. We also provide the unlearning time series to thetrained MLPs and compare the output of the MLPs with thetrue classification of the chaos generated using Eq. (6).
In this result, the validation accuracy becomes approxi-

(4)
0.000
0.011
0.000
0.000
0.000

(5)
0.000
0.000
0.006
0.000
0.000

(6)
0.000
0.000
0.000
0.000
0.000
-

Minimum
0.000130.00000
0.000130.00004
0.000020.00009

Maximum
0.264080.281160.26408
0.26800
0.25382
0.24771

Std. Dev.
0.06479
0.06759
0.06479
0.067450.06386
0.05097

Accuracy
86.36187.76886.22587.63786.66889.683Wilcoxon signed-rank test of classification performance of time

Learning performance of time series.
Minimum
0.00008
0.00000
0.00009
0.00001
0.000070.00007

(3)
0.009
0.000
0.000
0.006
0.000

Classification performance of unlearning time series.

Average0.101640.099460.101640.10094
0.10179
0.05681

Table 5
series.

(2)
0.000
0.000
0.011
0.000
0.000

(1)
(2)
(3)
(4)
(5)
(6)

(1)
0.000
0.000
0.000
0.000
0.000

(2)
0.258
0.000
0.011
0.000
0.000

(3)
1.000
0.258
0.000
0.006
0.000

(4)
0.417
0.724
0.417
0.000
0.000

(5)
0.959
0.154
0.959
0.386
0.000

(6)
0.000
0.000
0.000
0.000
0.000
-

mately 87 for every MLP. The proposed MLP with PGC (6)has the best average, maximum, and standard deviation. The
MLP with random noise (2) has the best minimum. These
results are similar to those for learning performance.
Table 5 shows the Wilcoxon signed-rank test for theclassification performance. The value of the sampling probability of the MLP with PGC (6) is less than 0.05 as compared with the others. We can be sure our method has better
performance than the others, because average of the proposed MLP with PGC (6) in Table 4 is less than the others.
Although the value of the sampling probability of the MLP
with random noise (2) is smaller than the value of the conventional MLP, this value is larger than the others. Therefore, we cannot say that the MLP with random noise (2) isbetter than the others.
3.2 Task 2: Proben1
In this simulation, we use Proben1, the benchmark problems for ANNs [17]. We choose Cancer, Card, and Glassfrom the data sets of Proben1. Every data set has multivariable inputs and a Boolean supervised signal. The number of
input dimensions and classifications of each task is shownin Table 6.
Here, the numbers of input dimensions and classifications for each task depends on the number of input and output neurons, respectively. In addition, the number of neurons in the hidden layer is 40, and the MLP comprises 9-402, 51-40-2 and 9-40-6 for solving Cancer, Card, and Glass,respectively. The data sets are as in Table 6. Thus, we obtain

IKUTA et al.: MULTI-LAYER PERCEPTRON WITH PULSE GLIAL CHAIN

747

Data
CancerCard
Glass

Table 6 Data sets of Proben1.
Num. of
Num. ofTraining
Unlearn.
inputsclass.
data sets
data sets
9
2
350
174
51
2
345
172
9
6
107
53

Table 7
Cancer (1)
(2)
(3)
(4)
(5)
(6)
Card (1)
(2)
(3)
(4)
(5)
(6)
Glass (1)
(2)
(3)
(4)
(5)
(6)

Table 8Wilcoxon signed-rank test of learning performance of Cancer.

Validation
35
34
10

(1)
0.000
0.490
0.000
0.000
0.000

(1)
(2)
(3)
(4)
(5)
(6)

(2)
0.000
0.000
0.000
0.000
0.000

(3)
0.490
0.000
0.062
0.000
0.004

(4)
0.000
0.000
0.062
0.000
0.000

(5)
0.000
0.000
0.000
0.000
0.000

(6)
0.000
0.000
0.004
0.000
0.000
-

Learning performance of Proben1.

Average Minimum Maximum Std. Dev. Accuracy
0.00569 0.00286
0.014290.00132
93.8910.00592 0.00287
0.014290.00164
94.0950.00523 0.00001
0.00858
0.0012994.4040.00578 0.00286
0.014290.0014193.9630.00561 0.00286
0.02001
0.00207
93.9830.00492 0.00000
0.00858
0.00177
94.2550.01856 0.00581
0.03479
0.00653
83.5530.01886 0.00581
0.03479
0.00638
83.6140.01839 0.00581
0.03479
0.00657
89.1950.01864 0.00581
0.03479
0.00668
83.5560.01886 0.00294
0.03193
0.0061083.6060.01769 0.00002
0.03189
0.0070283.7410.01058 0.00019
0.029420.00625
91.1690.01091 0.00177
0.02373
0.00547
91.4520.00952 0.00010
0.022450.00552
92.3440.01081 0.00022
0.024110.00584
91.4760.00903 0.00031
0.059390.00755
91.5400.00892 0.00031
0.028140.00628
91.750

Table 9

(1)
0.026
0.086
0.981
0.022
0.795

(1)
(2)
(3)
(4)
(5)
(6)

Table 10

(2)
0.026
0.008
0.075
0.562
0.731

(3)
0.086
0.008
0.073
0.006
0.999

(4)
0.981
0.075
0.073
0.062
0.631

(5)
0.022
0.562
0.006
0.062
0.255

(6)
0.795
0.731
0.999
0.631
0.255
-

Wilcoxon signed-rank test of learning performance of Glass.
(1)
0.199
0.274
0.102
0.003
0.028

(1)
(2)
(3)
(4)
(5)
(6)

Table 11the simulation results from 100 trials for each benchmark
problem, and each trial has 50,000 iterations. Table 7 shows
the learning performance of the MLPs for each benchmark
problem.
In this simulation, the validation accuracies are approximately 94, 83, and 91, in Cancer, Card, and Glass, respectively. On an average, the proposed MLP with PGC (6) isthe best of all for every benchmark task. In the minimum
and maximum results, the proposed MLP with PGC (6) isthe best for Cancer and Card, whereas the MLP with SA
noise (3) has the best minimum and maximum results for
Glass.
Tables 8–10 show results of the Wilcoxon signed-rank
test for each model. In the learning of Cancer, the resulting
evaluation of the proposed MLP with PGC (6) is less than
0.05. We can be sure the our method has better performance
than the others, because average of the proposed MLP with
PGC (6) in Table 7 is less than the others. In contrast, in thelearning of Card, the experimental result of our method isgreater than 0.05. Although we cannot verify the accuracyof the average, the average result of the proposed method isbetter than that of the others. In the evaluation of Glass, thevalue of the sampling probability of the MLP with PGC (6)is less than 0.05 as compared with the conventional MLP(1), the MLP with random noise (2), and the MLP with randomly timed pulses (4); however, these values are greaterthan 0.05 for the MLP with SA noise (3) and the MLP witha glial pulse (5).
Moreover, we show the classification performance of
the unlearning data sets in Proben1. We input the unlearning data sets to the trained MLPs and compare outputs ofWilcoxon signed-rank test of learning performance of Card.

(2)
0.199
0.056
0.630
0.000
0.000

(3)
0.274
0.056
0.106
0.130
0.210

(4)
0.102
0.630
0.106
0.000
0.002

(5)
0.003
0.000
0.130
0.000
0.655

(6)
0.028
0.000
0.211
0.002
0.655
-

Classification performance of unlearning data set of Proben1.

Cancer (1)
(2)
(3)
(4)
(5)
(6)
Card (1)
(2)
(3)
(4)
(5)
(6)
Glass (1)
(2)
(3)
(4)
(5)
(6)

Average Minimum Maximum Std. Dev. Accuracy
0.01686 0.01275
0.018960.0011098.1950.01662 0.01312
0.018750.0011998.2110.01757 0.01325
0.022020.0009498.2200.01671 0.01268
0.01904
0.0011198.2100.01679 0.01358
0.020230.00118
98.2070.01501 0.01193
0.018720.00130
98.2800.08501 0.07414
0.103640.00599
91.4310.08498 0.07453
0.10399
0.00592
91.4410.08502 0.07411
0.103420.00604
91.4370.08499 0.07418
0.10365
0.00599
91.4330.08500 0.07395
0.10367
0.0059391.4350.08284 0.07044
0.10400
0.00595
91.4570.08837 0.08151
0.10872
0.00401
88.2550.08889 0.08235
0.109830.00430
90.9960.08609 0.08284
0.10279
0.00322
91.2120.08817 0.08161
0.10847
0.00394
91.0740.08709 0.08205
0.108140.00405
91.0490.08342 0.08073
0.107700.00409
91.033

the MLPs with the ideal classifications. In this simulation,
the validation accuracies are approximately 98, 91, and 91,
in Cancer, Card, and Glass, respectively. The classification
performance is shown in Table 11, and the trend of the results is similar to that of Table 7. In every simulation, theMLP with PGC (6) obtains the best performance on average and the minimum for Cancer, Card, and Glass. The
MLP with PGC (6) obtains the best maximum result only
for Cancer; however, the MLP with SA noise (3) obtains thebest maximum result for Card and Glass.
Tables 12–14 show the results of the Wilcoxon singedrank test in the classification of Cancer, Card, and Glass,respectively. The value of the sampling probability of theIEICE TRANS. FUNDAMENTALS, VOL.E99–A, NO.3 MARCH 2016

748
Table 12Cancer.
(1)
(2)
(3)
(4)
(5)
(6)

Table 13
Card.
(1)
(2)
(3)
(4)
(5)
(6)

Table 14
Glass.
(1)
(2)
(3)
(4)
(5)
(6)

Wilcoxon signed-rank test of classification performance of
Data
(1)
0.001
0.000
0.000
0.001
0.000

(2)
0.001
0.000
0.149
0.034
0.000

(3)
0.000
0.000
0.000
0.000
0.000

(4)
0.000
0.149
0.000
0.209
0.000

(5)
0.001
0.034
0.000
0.209
0.000

(6)
0.000
0.000
0.000
0.000
0.000
-

98
130

Num. ofinputs2
2

Table 15 Conditions of TSP.
Num. ofTraining
Unlearn.
class.
data sets
data sets
2
98
98
2
130
130

Validation
9
13

Wilcoxon signed-rank test of classification performance of
(1)
0.436
0.667
0.009
0.145
0.004

(2)
0.436
0.293
0.634
0.511
0.008

(3)
0.667
0.293
0.561
0.321
0.017

(4)
0.009
0.634
0.561
0.370
0.010

(5)
0.145
0.511
0.321
0.370
0.026

(6)
0.004
0.008
0.017
0.010
0.026
-

Fig. 6

Supervised points. (a) 98 spiral points. (b) 130 spiral points.

Wilcoxon signed-rank test of classification performance of
(1)
0.000
0.057
0.000
0.045
0.065

(2)
0.000
0.001
0.000
0.411
0.036

(3)
0.057
0.001
0.360
0.411
0.036

(4)
0.000
0.000
0.360
0.862
0.014

(5)
0.045
0.000
0.411
0.862
0.063

(6)
0.065
0.021
0.036
0.014
0.063
-

MLP with PGC (6) is less than 0.05 compared with the others in Cancer and Card. We can be sure that of the MLP
with PGC (6) has better classification performance than theothers. For Glass, the value of the sampling probability of
the MLP with PGC(6) is less than 0.05 compared with the
MLP with random noise (2), the MLP with SA noise (3),and the MLP with randomly timed pulses (4); however, thevalue of the sampling probability of this MLP is greater than
0.05 compared with the conventional MLP (1) and the MLP
with a glial pulse (5).
3.3 Task 3: Two-Spiral Problem
For the next simulation, we use the two-spiral problem(TSP), a well-known highly nonlinear task for ANNs [24],[25]. This task has two sets of diﬀerent spiral points. For
learning, we input the coordinates of the spirals to the neurons in the input layer, and the MLPs learn the classification of two spiral points. Here, we use two diﬀerent spirals comprising 98 and 130 points, respectively, as shownin Fig. 6. In the classification performance, we input coordinates between zero and one after learning. We obtain the
output of the network, and determine which coordinates fits
into which spirals. The simulation conditions of each spiral are as in Table 15. Figure 7 shows the ideal results ofthe classification of coordinates. We change the coordinatesfrom zero to one in increments of 0.01 and input the coordinates to the trained MLP. Thus, the number of generated
test data for the analyses of the classification performance isFig. 7 Ideal classification results of two spirals. (a) 98 spiral points. (b)130 spiral points.

101 × 101. Moreover, we ensure the ideal result by calculating a norm between coordinates and spiral points. Note thatin this simulation, the MLP comprises 2-40-1 neurons.
3.3.1 Spirals Consisting of 98 Points
First, we show the experimental results from learning 98
points in Table 16. In this simulation, the validation accuracies are approximately 67; however, the validation accuracies of the MLPs are more decentralized than in the previous
simulations. From Table 16, the performance of the conventional MLP (1) is the worst among all for the average error,
because the conventional MLP (1) is often trapped in a local
minimum. In the case of the MLP with PGC (6), the average
error is the smallest of all. Energy is provided to the MLPs
from several sources, and the noise providing energy to the
MLP can generally escape from the local minimum. In addition, we can see the diﬀerence in the performance of the
MLPs from this table.
Table 17 shows the evaluation of the Wilcoxon signedrank test in the learning of 98 spiral points. The evaluationof the MLP with PGC (6) is less than 0.05 compared withthe others; thus, we can be sure of the result of the MLP
with PGC (6) for solving this task. The MLP with SA noise(3) has a better performance in Table 16; however, the evaluation of the Wilcoxon signed-rank test is greater than 0.05.
Hence, we cannot say from this simulation that the MLPIKUTA et al.: MULTI-LAYER PERCEPTRON WITH PULSE GLIAL CHAIN

749
Table 16
Average
0.04153
0.037110.02957
0.03666
0.032490.02072

(1)
(2)
(3)
(4)
(5)
(6)

Table 17
points.
(1)
(2)
(3)
(4)
(5)
(6)

(1)
0.085
0.001
0.446
0.008
0.000

Average
0.150290.13966
0.136640.147020.13805
0.12233

Table 19
points.
(1)
(2)
(3)
(4)
(5)
(6)

Maximum
0.183870.17352
0.092130.08208
0.163900.08192

Std. Dev.
0.02637
0.029460.020800.02195
0.021470.01782

Accuracy
73.57366.67467.51366.512
66.94869.345Wilcoxon signed-rank test of learning performance of 98

Table 18
(1)
(2)
(3)
(4)
(5)
(6)

Learning performance for 98 points.

Minimum
0.000170.00006
0.000180.000150.000190.00011

(2)
0.085
0.173
0.375
0.592
0.000

(3)
0.001
0.173
0.010
0.211
0.002

(4)
0.008
0.592
0.211
0.225
0.000

(5)
0.008
0.592
0.211
0.225
0.000

(6)
0.000
0.000
0.002
0.000
0.000
-

Classification performance of 98 spiral points.
Minimum
0.080850.08083
0.076110.07965
0.075290.08140

Maximum
0.211270.20378
0.219630.20083
0.203620.17042

Std. Dev.
0.02434
0.02879
0.02837
0.025530.02468
0.01939

Accuracy
76.81177.57481.97177.17178.74080.434Wilcoxon signed-rank test of classification performance of 98
(1)
0.005
0.001
0.329
0.001
0.000

(2)
0.005
0.370
0.084
0.710
0.000

(3)
0.001
0.370
0.006
0.747
0.000

(4)
0.329
0.084
0.006
0.016
0.000

(5)
0.001
0.710
0.747
0.016
0.000

(6)
0.000
0.000
0.000
0.000
0.000
-

with SA noise (3) has better performance.
We show the classification results in Table 18. The
results show a trend similar to that in Table 16. In general, the MLP has an overlearning problem when the MLP
learns more than required, losing its generalization capability. However, the proposed MLP with PGC (6) can still obtain better approximation and classification performances.
Table 19 shows the evaluation of the Wilcoxon signedrank test as the classification results of the 98 spiral points,
and the trend of the evaluation is similar to that shown in
Table 17. In addition, the evaluation of the MLP with PGC(6) ensures classification performance.
Figure 8 shows examples of the classification results.
We can see from Fig. 8 that the MLP with PGC (6) draws
only the two spirals; in contrast, the others are decoupled insome parts.
3.3.2 Spirals Consisting of 130 Points
Here, we show the results of the MLP learning 130 spiralpoints. For the TSP, simulation diﬃculty increases with in-

Fig. 8Classification results of unlearned coordinates.

creasing number of points. Table 20 shows the approximation results. In this simulation, the validation accuracy becomes approximately 60. The validation accuracies of the
MLPs decrease from the results in Table 16. From this table, the diﬀerences of the performances are greater than for
Table 16. The conventional MLP (1) falls into the local minimum more often than in the learning of 98 spiral points.
Moreover, the MLP with random noise (2), the MLP withSA noise (3), the MLP with randomly timed pulses (4), andthe MLP with glial pulse (5) have performance similar tothe conventional MLP (1). We often hope that noise is efficient for highly nonlinear problems; however, we observe
little improvement in learning performance by the methodsof (2)–(5). The proposed MLP with PGC (6) also obtains
energy from the glias; however, this MLP has a performance
twice as good as the others for the average, the maximum,
and the standard deviation. For the minimum, the MLP withrandom noise (2) obtains the best results. The minimum of
the MLP with the PGC (6) is almost the same as that of the
MLP with random noise (2). From these results, we canconfirm that the proposed PGC is eﬃcient improving MLP
performance.
Table 21 shows the evaluation of the Wilcoxon signed-

IEICE TRANS. FUNDAMENTALS, VOL.E99–A, NO.3 MARCH 2016

750
Table 20
Average
0.12269
0.10847
0.09735
0.11439
0.09393
0.03830

(1)
(2)
(3)
(4)
(5)
(6)

Table 21
points.

Maximum
0.23857
0.242780.24355
0.26349
0.25378
0.12190

Std. Dev.
0.055540.057420.05356
0.057420.055440.02589

Accuracy
61.19462.84764.96059.38659.38662.368Wilcoxon signed-rank test of learning performance of 130

(1)
(2)
(3)
(4)
(5)
(6)

(1)
(2)
(3)
(4)
(5)
(6)

Learning performance of 130 points.
Minimum
0.008310.000470.00107
0.007400.001300.00063

(1)
0.091
0.001
0.541
0.001
0.000

(2)
0.091
0.173
0.514
0.106
0.000

(3)
0.001
0.173
0.071
0.682
0.000

(4)
0.541
0.514
0.071
0.012
0.000

(5)
0.001
0.106
0.682
0.012
0.000

(6)
0.000
0.000
0.000
0.000
0.000
-

Table 22Classification performance of 130 points.

Average0.21782
0.19278
0.196710.20432
0.193970.14731

Minimum
0.10565
0.10460
0.13272
0.12082
0.123030.08792

Table 23
points.
(1)
(2)
(3)
(4)
(5)
(6)

Maximum
0.294770.330650.28846
0.31958
0.29973
0.23723

Std. Dev.
0.03858
0.04434
0.03166
0.03851
0.03730
0.02826

Accuracy
71.96872.70175.66072.45174.18275.870Wilcoxon signed-rank test of classification performance of 130
(1)
0.000
0.000
0.017
0.000
0.000

(2)
0.091
0.403
0.057
0.667
0.000

(3)
0.000
0.403
0.186
0.543
0.000

(4)
0.017
0.057
0.186
0.051
0.000

(5)
0.000
0.667
0.543
0.051
0.000

(6)
0.000
0.000
0.000
0.000
0.000
-

rank test for the learning performance of 130 spiral points.
The evaluation of the MLP with PGC (6) becomes zero;
thus, we can be sure of the result in Table 20.
Table 22 shows the classification results for learning
130 spirals points, and the MLP with PGC (6) is the bestof all by nearly every measure; therefore, we conclude that
pulse propagation is important for MLP performance.
Table 23 shows the evaluation of the Wilcoxon signedrank test of classification performance of 130 spiral points.
The evaluation of the MLP with PGC (6) is less than 0.05
compared to the others; thus we can be sure of the diﬀerences in the average results in Table 22.
Figure 9 shows dependencies of learning and classification performances for the weight of glial eﬀect α in the
proposed MLP with PGC (6). In addition, we show a change
in the performances of the MLP with random noise (2) andin the MLP with SA noise by changing the amplitudes of theuniform random and a SA noises, respectively. Note that wechange the weight of the glial eﬀect α and the amplitudeof the uniform random noise from zero to one, and when
α is equal to zero, the proposed MLP with PGC (6) is theFig. 9 Dependency of the learning and classification performances forthe weight of glial eﬀect α.

same as the conventional MLP (1). Generally, the learning
performance corresponds with changes in the classification
performance in the proposed MLP with PGC (6). The proposed MLP with PGC (6) has the best result for α equal to0.5. In contrast, the MLP with random noise (2) has thebest result with the amplitude of the uniform random noise
equal to 0.2 in learning performance. In the case of the MLP
with SA noise (3), the results are nearly unchanged with theamplitude of the noise at any time. We consider that thenormal distribution noise rarely generates a large value inthe transient state; thus, the dependency of the amplitudeof the noise is lower with SA. From the diﬀerence in the
results between the proposed MLP with PGC (6) and the
MLP with random noise (2), the proposed MLP with PGC(6) can with a larger α than the MLP with random noise(2), because the glia provide energy to the network instantaneously through the generated pulse. Furthermore, we canuse the large noise amplitude for the MLP with SA noise(3); however, the normal distribution noise is minimally effective for MLP performance. Thus, the MLP can reduce
the error even if it receives a pulse of large amplitude fromthe glia. From this result, we conclude that the proposed
PGC is more suitable for diﬃcult tasks. In fact, our model
improves MLP performance more in more diﬃcult tasks.
Next, we show the dependency of learning performance on the number of neurons in the hidden layer of the
MLPs in Fig. 10. We know that the performance of each
MLP improves with the number of neurons, but once thenumber of neurons in the hidden layer is greater than 50,
the learning performance decreases or remains on the samelevel, because the MLP requires long iterations for convergence when it contains many neurons. In the case of the
MLP with PGC (6), the learning performance is similar tothat of the others when the number of neurons is small. In
addition, when the number of neurons increases, the diﬀerence in performance between the MLP with PGC (6) and theothers increases. Thus, we can consider that the eﬀect of theglia increases with the number of neurons. In the PGC, theIKUTA et al.: MULTI-LAYER PERCEPTRON WITH PULSE GLIAL CHAIN

751

Fig. 10 Dependency of learning and classification performance for thenumber of neurons in the hidden layer.

pulse is propagated to the other glia. If the number of neurons is small, the pulse propagation finishes instantaneously;
therefore, PGC has no eﬀect on the MLP.
Finally, we show examples of the classification of unlearned coordinates results in Fig. 11. The conventional
MLP (1) cannot represent the two spirals, and in the MLP
with random noise (2) and the MLP with randomly timed
pulses (4), the spirals are divided into several parts. The spirals of the MLP with SA noise (3) have a false area in thecenter of the image. The MLP with glial pulse (5) draw apart of spirals; however, there exists many errors in the upper area of the figure. The spirals of the MLP with PGC (6)are divided in only one part.

Fig. 11Classification results of unlearned coordinates.

4. Discussions
In this section, we discuss the eﬀects of the PGC on the
MLP.
Firstly, we discuss the updating rule of the weights
based on the BP algorithm. The updating rule of the weights
between the hidden and output layers is as follows.
Δwk j = η(T k − Ok )Ok (1 − Ok )H j ,(7)where T is the target point, O is the output of the neuron
in the output layer, H is the output of the neuron, and η isthe learning coeﬃcient in the hidden layer. The updating
rule of the weights between the input and hidden layers is
expressed by Eq. (8).
Δw ji = ηXi H j (1 − H j )n

wk j (T k − Ok )Ok (1 − Ok ), (8)k=1where X is the output of the neuron in the input layer. Equations (7) and (8) are proportional to H and H(1 − H), respectively. The glial excitation depends on the outputs ofneurons in the hidden layer; hence, we consider the relationships between the two equations and a neuron’s output
in the hidden layer. The weight update between the hidden
and output layers, and the weight update between the input

Fig. 12

Input-output characteristics of H(1 − H).

and hidden layers depend on H and H(1 − H), respectively.
The descriptions of H and H(1−H) are illustrated in Fig. 12.
We can see from this figure that the weight update between
the hidden and output layers increases with H and the weight
update between the input and hidden layers decreases when
H becomes greater than 0.5.
Next, we show total updates of the weights and the ratio of the number of pulse generations in each glial cell to
the total iteration in Figs. 13 and 14. We obtain the ratiosof number of iterations and the number of neurons whose
output is greater than the excitation threshold of the glia toIEICE TRANS. FUNDAMENTALS, VOL.E99–A, NO.3 MARCH 2016

752

Fig. 13 Comparison of the total updated weights between the hidden
layer and the output layer and the generated pulse. (a) The conventional
MLP. (b) The proposed MLP.

Fig. 14 Comparison of the total updated weights between the input layer
and the hidden layer and the generated pulse. (a) The conventional MLP.
(b) The proposed MLP.

the total number of iterations.
Figure 13 compares the characteristics of the hidden
and output layers in the conventional and the proposed MLP.
We can see that there is a positive correlation between the total updates of weights and the number of pulse generationsin Figs. 13 (a) and (b). The characteristics relating to the hidden and output layers are in accordance with Eq. (7); however, the correlation between the total updates of weights
and the number of pulse generations of the proposed MLP
is much weaker than that of the conventional MLP. The correlation coeﬃcient of the conventional MLP is 0.84, while
the correlation coeﬃcient of the proposed MLP is 0.65.
Figure 14 compares the characteristics of the input andhidden layers in the conventional and the proposed MLP.
We can see that there is a negative correlation between the
total updating weights and the number pulse generations in
Figs. 14(a) and (b). The characteristics relating the input andhidden layers are in line with Eq. (8); however, the correlation between the total updates of weights and the number ofpulse generations of the proposed MLP is much weaker thanthat of the conventional MLP. The correlation coeﬃcient of
the conventional MLP is −0.65, whereas the correlation coeﬃcient of the proposed MLP is −0.21.
In the characteristics of the proposed MLP, we can see
the position dependency of the total updates of the weights
at the 20th and 35th neurons. The generated pulse increases

the connected neuron output, and this pulse is propagatedto the neighboring glia. The propagated pulse increases theoutputs of the neighboring neurons irrespective of the inner state of the neighboring neurons, thereby displacing thelearning points of the neighboring neurons. Thus, the updates of the weights in the neighboring neurons are changed,
and the correlation coeﬃcient of the proposed MLP is decreased. Therefore, we conclude that the neurons influence
the neighboring neurons through the glial pulse generation,
thereby improving the learning performance of the MLP.
Finally, we investigate the updated weights in detailwhen the MLPs learn chaotic time series. The updated
weights are placed into five classes according to the stateof the glial pulse. We periodically obtain the average of the
updated weights of the hidden and output layers that meetthe following requirements for a fixed period and unify the
updated weights obtained from the same requirements of theneurons. The five types of updated weights are as follows:(A) dwgp (τgp ) is the updated weight when the pulse input tothe excitation threshold of the neuron and the glia is excitedby the connected neuron, (B) dwgp (τgp − 1) is the updated
weight when the pulse is generated, (C) dwrp (τrp ) is the updated weight when the neuron receives another glial pulse,(D) dwrp (τrp − 1) is the updated weight when another glial
pulse was propagated previously, and (E) dwnp (τnp ) is the

IKUTA et al.: MULTI-LAYER PERCEPTRON WITH PULSE GLIAL CHAIN

753

Fig. 15 Updated weights at pulse generation and received pulse in theconventional MLP (1).

updated weight when the connected glial pulse becomes statistically small. Figures 15 and 16 show the updated weightin the conventional MLP (1) and the MLP with PGC (6),respectively, during the iteration.
In Fig. 15, we assume the pulse generation of glia basedon the proposed method; however, this pulse is not input
to the neuron. dwgp (τgp ) overlaps with dwgp (τgp − 1), anddwrp (τrp ) overlaps with dwrp (τrp − 1) and dwnp (τnp ). According to this characteristic, the updated weight is not influenced for a short time. Moreover, the updated weight increases when the glia generate the pulse for the connected
neuron because dwgp (τgp ) and dwgp (τgp − 1) are larger than
dwrp (τrp ), dwrp (τrp − 1), and dwnp (τnp ). In the error curve
shown as MSE, the error reduction converges earlier.
In contrast, the error curve oscillates in Fig. 16, and every updated weight is diﬀerent in comparison with Fig. 15.
In particular, we can observe three characteristics, as follows.
1. dwgp (τgp ) is smaller overall than dwgp (τgp − 1). Thismeans that the updated weight when the glial cell generates a pulse from the connected neuron is smaller
than that when the neuron receives a pulse from a connected glial cell. As a result of this characteristic, theglial pulse by which the glial cell is excited by the connected neuron decreases the updated weight, because
the glial pulse increases the output of the connected
neuron. The output of the connected neuron is alreadygreater when the neuron receives the pulse of the connected glial cell; thus, the output of the connected neuron becomes closer to one as a result of the pulse. According to Fig. 12, the updated weight decreases when
the neuron output is close to one.
2. dwrp (τrp ) is larger overall than dwrp (τrp − 1). Thismeans that the updated weight when the neuron receives the propagated pulse from another glial cell isgreater than when another glial pulse was previously
propagated. As a result of this characteristic, the otherglial pulse increases the updated weight because thisFig. 16 Updated weights at pulse generation and received pulse in the
MLP with PGC (6).

pulse increases the output neuron irrespective of theprevious output of this neuron.
3. dwnp (τnp ) is similarly small to dwrp (τrp − 1). As a result of this characteristic, the weight is slowly updated
when the glial pulse becomes statistically small.
Overall, we conclude that the MLP with PGC (6) can
find the various solutions because the updated weights arechanged in various ways by the glial pulse. Moreover, theupdated weight becomes small when the glial pulse becomes
statistically small. Then, the MLP with PGC (6) can search
specifically for the solution. In fact, the error curve showsthat the MLP with PGC (6) obtains various solutions during
the iteration and finds a better solution at the end of learning.
5. Conclusions
In this study, we proposed an MLP with PGC. The PGC wasinspired by the biological features of glia, and we connected
the glia to hidden layer neurons. Glial cells generate a pulse
depending on the output of a connected neuron. The pulses
aﬀect the neighboring glial cells and the excitation threshold of the connected neuron. For updating weights, we also
found that the position relationships depend on the generation of the pulses. Finally, we confirmed through three
diﬀerent simulations that the proposed MLP had better approximation and classification performance than the conventional and other MLPs. In our future works, we will verify
the correlation between the position of the neuron in the hidden layer and the pulse generation.
Acknowledgment
This work was partly supported by MEXT/JSPS Grant-inAid for JSPS Fellows (24·10018), and by the Chinese national natural science foundation on under grants 61573272.
References[1] P.G. Haydon, “Glia: Listening and talking to the synapse,” Nat. Rev.

IEICE TRANS. FUNDAMENTALS, VOL.E99–A, NO.3 MARCH 2016754

Neurosci., vol.2, no.3, pp.185–193, 2001.
[2] R.D. Fields and B. Stevens-Graham, “New insights into neuron-Glia
communication,” Science, vol.298, no.5593, pp.556–562, 2002.
[3] G.I Hatton and V. Parpura, Glia ⇔ neuronal signaling, Kluwer Academic Publishers, 2004.
[4] S. Koizumi, K. Fujishita, M. Tsuda, Y. Shigemoto-Mogami, andK. Inoue, “Dynamic inhibition of excitatory synaptic transmission
by astrocyte-derived ATP in hippocampal cultures,” Proc. National
Academy of Sciences, vol.100, no.19, pp.11023–11028, 2011.
[5] S. Ozawa, “Role of glutamate transporters in excitatory synapses
in cerebellar purkinje cells,” Brain and Nerve, vol.59, pp.669–676,
2007.
[6] G. Perea and A. Araque, “Glial calcium signaling and neuron-Glia
communication,” Cell Calcium, vol.38, no.3-4, pp.375–382, 2005.
[7] S. Kriegler and S.Y. Chiu, “Calcium signaling of Glial cells along
mammalian axons,” J. Neurosci., vol.13, pp.4229–4245, 1993.
[8] M.P. Mattson and S.L. Chan, “Neuronal and Glial calcium signalingin Alzheimer’s disease,” Cell Calcium, vol.34, no.4-5, pp.385–397,
2003.
[9] N. Takata, T. Mishima, C. Hisatsune, T. Nagai, E. Ebisui, K.
Mikoshiba, and H. Hirase, “Astrocyte calcium signaling transforms
cholinergic modulation to cortical plasticity in vivo,” J. Neurosci.,vol.31, no.49, pp.18155–18165, 2011.
[10] D.E. Rumelhart, G.E. Hinton, and R.J. Williams, “Learning representations by back-propagating errors,” Nature, vol.323, no.6088,
pp.533–536, 1986.
[11] S. Kirkpatrick, C.D. Gelatt, and M.P. Vecchi, “Optimization by simulated annealing,” Science, vol.220, no.4598, pp.671–680, 1983.
[12] S. Amato, B. Apolloni, G. Caporali, U. Madesani, and A. Zanaboni,“Simulated annealing approach in backpropagation,” Neurocomputing, vol.3, no.5-6, pp.207–220, 1991.
[13] C.B. Owen, A.M. Abunawass, and D.W. Ruck, “Application ofsimulated annealing to the backpropagation model improves convergence,” Proc. SPIE, Science of Artificial Neural Networks II,
pp.269–276, 1993.
[14] A. Porto, A. Pazos, and A. Araque, “Artificial neural networks basedon brain circuits behaviour and genetic algorithms,” Computational
Intelligence and Bioinspired Systems, Lecture Notes in Computer
Science, vol.3512, pp.99–106, Springer Berlin Heidelberg, Berlin,
Heidelberg, 2005.
[15] A.B. Porto-Pazos, N. Veiguela, P. Mesejo, M. Navarrete, A.
Alvarellos, O. Ibanez, A. Pazos and A. Araque, “Artificial astrocytes improve neural network performance,” PLos ONE, vol.6, no.4,
e19109. doi: 10.1371/journal.pone.0019109, 2011.
[16] A. Alvarellos, A. Pazos, and A.B. Porto-Pazos, “Computationalmodel of neuron-astrocyte interactions lead to improved eﬃcacy inthe performance of neural networks,” Computational and Mathematical Methods in Medicine, vol.2012, Article ID 476324, 10 pages,
2012. doi: 10.1155/2012/476324.
[17] L. Precheltt, “PROBEN1 — A set of neural network benchmark
problems and benchmarking rules,” Technical report, Fakultat für
Informatik, Universitat Karlsruhe, Germany, 1994.
[18] C. Ikuta, Y. Uwate, and Y. Nishio, “Multi-layer perceptron with Glial
network for solving two-spiral problem,” IEICE Trans. Fundamentals, vol.E94-A, no.9, pp.1864–1867, Sept. 2011.
[19] M. Morita, F. Yoshiki, and Y. Kudo, “Simultaneous imaging ofphosphatidyl inositol metabolism and Ca2+ levels in PC12h cells,”
Biochem. Biophys. Res. Commun., vol.308, no.4, pp.673–678, Sept.
2003.
[20] A. Bal-Price, Z. Moneer, and G.C. Brown, “Nitric oxide induces
rapid, calcium-dependent release of vesicular glutamate and ATPfrom cultured rat astrocytes,” Glia, vol.40, no.3, pp.312–323, Oct.
2002.
[21] D.L. Elliott, “A better activation function for artificial neural networks,” ISR Technical Report, TR 93-8, Jan. 1993.
[22] D.W. Ruck, S.K. Rogers, and M. Kabrisky, “Feature selection using a multilayer perceptron,” J. Neural Netw. Comput., vol.2 no.2,

pp.40–48, 1990.
[23] S.-I. Amari, H. Park, and K. Fukumizu, “Adaptive method of realizing natural gradient learning for multilayer perceptrons,” NeuralComput., vol.12, no.6, pp.1399–1409, June 2000.
[24] J.R. Álvarez-Sánchez, “Injecting knowledge into the solution of
the two-spiral problem,” Neural Comput. Appl., vol.8, no.3,
pp.265–272, 1999.
[25] H. Sasaki, T. Shiraishi, and S. Morishita, “High precision learningfor neural networks by dynamic modification of their network structure,” Dynamics & Design Conference, pp.411-1–411-6, 2004.

Chihiro Ikuta
was born in Tokushima,
Japan, in 1987. He received the B.E. and M.E.
degrees in electrical and electronic engineeringfrom Tokushima University, Tokushima, Japan,
in 2010 and 2012, respectively. He was a Research Fellow (DC1) of the Japan Society for
the Promotion Science (JSPS) from April 2012
to March 2015. He is currently working towards
the Doctorate degree in the Department of Electrical Engineering, Tokushima University. He
has been currently working at National Instituteof Technology, Anan College as Assistant Professor from April 2015. His
research interests include neural network and image processing. He is a
student member of the IEEE.

Yoko Uwate (S’02-M’07)
was born in Tokushima, Japan, in 1980. She received the B.E.,
M.E., and Ph.D. degrees in electrical and electronic engineering from Tokushima University,
Tokushima, Japan, in 2003, 2005 and 2006, respectively. During October 2006–March 2008,
she was a Postdoctral Resarch Fellow (PD) of
the Japan Society for the Promotion Science(JSPS) at the same university, and she was also
Visiting Post Doctoral Research Fellow, Institute of Neuroinformatics (INI), University andETH Zurich. From April 2008 to March 2010, she worked as a Postdoctoral Fellow for Research Abroad of Japan Society for the Promotion Science (JSPS) at Institute of Neuroinformatics (INI), University and ETH
Zurich. Since April 2010, she has been currently working at Tokushima
University as Assistant Professor. Her research interests include complex
phenomena in chaotic circuits and neural networks. She was an Associate
Editor of the IEEE Transactions on Circuits and Systems–I: Regular Papers
during Dec. 2012–Dec. 2013. She is currently a member of the IEEE CAS
Society Board of Governors (from 2013). She is a member of the IEEE,the IEICE and the RISP.

IKUTA et al.: MULTI-LAYER PERCEPTRON WITH PULSE GLIAL CHAIN

755

Yoshifumi Nishio
received the B.E., M.E.,
and Ph.D. degrees in electrical engineering from
Keio University, Yokohama Japan, in 1988,
1990, and 1993, respectively. In 1993, he joined
the Department of Electrical and Electronic Engineering at Tokushima University, Tokushima
Japan, where he is currently a Professor. FromMay 2000 he spent a year in the Laboratory of
Nonlinear Systems (LANOS) at the Swiss Federal Institute of Technology Lausanne (EPFL) as
a Visiting Professor. His research interests are in
the areas of nonlinear circuits engineering, including analysis and application of chaos in electrical circuits, analysis of synchronization in coupledoscillatory circuits, development of analyzing methods for nonlinear circuits, theory and application of cellular neural networks, neural networkarchitecture, evolutionary computation, and biosignal processing. He was
the Chair of the IEEE CAS Society Technical Committee on Nonlinear
Circuits and Systems (NCAS) during 2004–2005 and the Steering Committee Secretary of the IEICE Research Society of Nonlinear Theory andits Applications (NOLTA) during 2004–2007, and is currently the Chair of
the IEEE CAS Society Shikoku Chapter (from 2011) and a member of theIEEE CAS Society Board of Governors (from 2012). He was an Editorof the IEICE Fundamentals Review during 2007–2012 and an Associate
Editor of the IEEE Transactions on Circuits and Systems–I: Regular Papers during 2004–2005, the IEEE CAS Magazine during 2008–2009, andthe IEEE Transactions on Circuits and Systems–II: Express Briefs during
2012–2013. He is currently serving as a Secretary for the NOLTA, IEICE(from 2009) and an Associate Editor of the IEEE CAS Society Newsletter(from 2007). He is a senior member of the IEEE, and a member of the
IEICE and the RISP.

Guaon Yang
received the B.Eng. degreein automatic control engineering from Jilin University China, and M.Eng. degree from Tokyo Metropolitan University Japan in 1986 and1993, respectively. He was a researcher fellow
with Hertz co. Ltd in Tokyo Japan, from 1993
to 2001. Since May 2001, He joined the institute of artificial intelligence and robotics ofXi’an Jiaotong University China, and received
the Ph.D. degree in the Department of Control
Science and Engineering, in the school of electronic and information of Xi’an Jiaotong University China, in 2007. Since
January 2011, he has been an associate professor of the department of
automation science and technology in the school of electronic and information of Xi’an Jiaotong University. His main research interests include
wavelets, image and multidimensional signal processing, multiscale geometric analysis, and sparse representation theory and applications.