Multi-Layer Perceptron with Pulse Glial Chain
Having Oscillatory Excitation Threshold
Chihiro Ikuta

Yoko Uwate

Yoshifumi Nishio

Dept. of Electrical and Electronic Eng., Dept. of Electrical and Electronic Eng., Dept. of Electrical and Electronic Eng.,
Tokushima University
Tokushima University
Tokushima University
2-1 Minami-Josanjima,
2-1 Minami-Josanjima,
2-1 Minami-Josanjima,
Tokushima Japan
Tokushima Japan
Tokushima Japan
Email: ikuta@ee.tokushima-u.ac.jp
Email: uwate@ee.tokushima-u.ac.jp
Email: nishio@ee.tokushima-u.ac.jp

Abstract—A brain has neuron and glial cells. In the brain,
these cells correlate each other and make a higher brain function.
In this study, we propose a Multi-Layer Perceptron (MLP)
with pulse glial chain having oscillatory excitation threshold. We
connect artificial glia units with the neurons in a hidden-layer.
When the connecting neuron output is larger than an excitationthreshold of the connected glia, the glia excites and generates
the pulse. This pulse transmits to the neighboring glias andthe connecting neuron. The pulse increases a threshold of theconnecting neuron, thus the glia gives energy for solving tasks. In
this model, the excitation threshold is oscillating within a definedvalue. Even if the connecting neuron output does not change,
the pulse generation occurs by the oscillation of the excitationthreshold. The oscillation of the excitation threshold gives moreenergy to the network and improves a learning performance ofthe MLP. By computer simulation, we confirm that the oscillationof the excitation threshold improves a learning performance.

I.

I NTRODUCTION

A glia and a neuron are cells existing in a brain. We
already have many investigations and many applications about
the neuron, because the neuron transmits an electric signal
each other. Thus, the neuron closely relates to brain works. On
the other hand, a main function of the glia were considered
to a support of the neuron. Actually, the glia has many
support functions for the neuron. However, several researchers
discovered that the glia has important functions for a higher
brain function [1][3]. The glia has many receptors to ionssuch as glutamate acid, adenosine triphosphate, calcium [4][6].
Among them, we focus on the calcium ion. The glia transmits asignal by using the change of calcium ion concentration [7][8].
This change of ion concentration also influences a membranepotential of the neuron. Thus, the glia closely relates to theneurons and makes the brain works. We consider that the glia
function can apply to an artificial neural network.
For the application, we use a Multi-Layer Perceptron(MLP). The MLP is a famous artificial neural network. In thismodel, the neurons connect with the neurons in neighboringlayers. Each neuron has weights of connections between theneurons. We can change the output value of the MLP by tuningof the weights of connection. In general, a Back Propagation(BP) algorithm is used for determination of a value of theweights of connections [9]. By this learning, the MLP canbe applied to several tasks such as pattern recognition, data
mining. However, the BP algorithm has a local minimum

978-1-4799-8391-9/15/$31.00 ©2015 IEEE

1330

problem, because the BP algorithm uses the steepest decent
method. If the MLP falls into local minimum, the MLP
cannot escape out from there. For this problem, many methods
have proposed such as a noise inputting, introduction of amomentum coefficient, and so on.
In this study, we propose a Multi-Layer Perceptron (MLP)
with pulse glial chain having oscillatory excitation threshold
which is inspired by the biological glia functions. We one-byone connect the glia with the neuron in the hidden-layer. Eachglia receives the connecting neuron output. When the connecting neuron has a large output than an excitation threshold of theglia, the glia is excited. The excited glia generates a pulse. Thispulse transmits to the neighboring glias and the connecting
neuron. If the neighboring glia receives other glia pulse, thisglia is also excited. Therefore, the glia pulse transmits to all
glias. In the case of the neuron, the glia pulse increases thethreshold of the inner state of the neuron. The neuron output
is changed by the glia pulse and obtains an energy from theglia. In this model, we introduce the oscillatory excitationthreshold of the glia. The excitation threshold oscillates within
defined value during a learning. When the learning of the
network converges, the output of the neuron is fixity. Thereby,
the pattern of the glia pulse generation is also fixed. In the
proposed model, the glia can change between a generation anda stopping generation of the pulse by the oscillatory excitationthreshold of the glia. By the change of the pulse generation,
the pulse propagation is also changed. We consider that theoscillatory excitation threshold influences the convergent of
the MLP learning. We confirm that the glia improves the MLP
learning performance by using Two-Spiral Problem (TSP).
II.

P ROPOSED MLP

The MLP is one of feed forward neural network. The
neurons make layers and have connections with the neurons
in the neighboring layers. In this study, we propose the MLP
with pulse glial chain having oscillatory excitation thresholdshown as Fig. 1.
A. Pulse glial chain having oscillatory excitation threshold
The glias are one-by-one connected with the neurons in thehidden-layer. In this model, the glias and the neurons correlate
each other. The glia receives the connecting neuron output

overcomes the oscillatory excitation threshold of the glia when
the output of this neuron is converged. The connected glia with
the neuron (C) is excited independent from the oscillation of
the excitation threshold of the glia. In the proposed method,
several glias are stochastically excited by the oscillation of theexcitation threshold of the glia. We consider that the oscillatory
excitation threshold breaks the periodic pulse generation of theglias, thereby the glias can effectively give the energy to the
MLP.

Neuron

…

1

Glia

0.8

Fig. 1.

Proposed MLP.
0.6

and is excited by the connecting neuron output. The excitation
conditions of the glia are described by Eq, (1).
𝜓 (𝑡 + 1) =⎡𝑖
1,{𝜃𝑛 (𝑡) < 𝑦𝑖 ∪ 𝜓𝑖+1 (𝑡 − 𝐷) = 1⎢∪ 𝜓𝑖−1 (𝑡 − 𝐷) = 1)} ∩ (𝑡 − 𝜏𝑖 > 𝜃𝑔 ),⎢, (1)⎢ −1, {1 − 𝜃𝑛 (𝑡) > 𝑦𝑖 ∪ 𝜓𝑖+1 (𝑡 − 𝐷) = 1⎣∪ 𝜓𝑖−1 (𝑡 − 𝐷) = 1)} ∩ (𝑡 − 𝜏𝑖 > 𝜃𝑔 )𝛾𝜓𝑖 (𝑡), 𝑒𝑙𝑠𝑒,
where 𝜓 is an output of a glia, 𝑖 is a position number of the glia
in the hidden-layer, 𝛾 is an attenuation parameter (0 < 𝛾 < 1),
𝑦 is an output of a connecting neuron, 𝜃𝑛 is a excitationthreshold of the glia, 𝜏 is a time of a previous pulse generation,
𝜃𝑔 is a period of inactivity, and 𝐷 is a delay time of a glial
effect When the connecting neuron output is larger than the
excitation threshold (𝜃𝑛 (𝑡)) of the glia, the glia is excited andgenerates the positive pulse. And also, when the connecting
neuron output is smaller than 1 − 𝜃𝑛 (𝑡) of the glia, the glia is
excited and generates the negative pulse. The generated pulse
transmits to the neighboring glias and the connecting neuron.
The pulse excites the neighboring glias and increases the innerstate of the connecting neuron. The neighboring glias also
generate the pulse, thereby the pulse transmits to all glias.
When the glia generates the pulse, the glia starts the period
of inactivity. If this glia receives the neuron output and/or theneighboring glial pulse, the glia cannot be excited again during
period of inactivity.
In this study, we introduce the oscillatory excitation threshold of the glia to the pulse glial chain. The oscillatory
excitation threshold of the glia is described by Eq. (2).
𝜃𝑛 (𝑡) = 𝑟(𝑡) (𝑎 < 𝑟 < 𝑏),(2)where 𝑟 is random function, and 𝑎 and 𝑏 is constant value. The
value of the excitation threshold oscillates between 𝑎 and 𝑏.
We use Mersenne Twister pseudo-random number [10] for the
random function. Figure 2 shows example of the relationships
between the oscillatory excitation threshold of the glia andthe output of the neurons. We consider three cases of theoutput of the neuron in the simulation. The output of theneuron (A) does not reach the excitation threshold of the glia,
thereby the connected glia is not excited by this neuron. The
output of the neuron (B) is into the amplitude of the oscillatory
excitation threshold of the glia when the output of this neuron
is converged. In this case, the connected glia with neuron (B) is
stochastically excited. The output of the neuron (C) completely

1331

0.4
Excitation threshold of the glia
Output of neuron (A)Output of neuron (B)Output of neuron (C)

0.2

0

t

Fig. 2. Relationships between oscillatory excitation threshold of the glia andthe output of the neurons.

B. Updating rule of neuron
The neuron has multi-input and single-output, and we canchange the neuron output by tuning the weights of connections
between the neurons. The standard updating rule of the neuron
is defined as shown in Eq. (3).
⎛⎞𝑛∑𝑦𝑖 (𝑡 + 1) = 𝑓 ⎝𝑤𝑖𝑗 (𝑡)𝑥𝑗 (𝑡) − 𝜃𝑖 (𝑡)⎠ ,(3)
𝑗=1

where 𝑦 is an output of the neuron, 𝑤 is a weight of theconnection, 𝑥 is an input of the neuron, and 𝜃 is an excitationthreshold of the neuron. In this equation, the weight of theconnection and the threshold of the neuron are learned basedon the BP algorithm. Next, we show the proposed updating
rule of the neuron. We add the generated pulse of the glia 𝜓to the excitation threshold of the neuron. In this study, thisupdating rule is only used for the neurons in the hidden-layer.
The updating rule is described by Eq. (4).
⎛⎞𝑛∑𝑦𝑖 (𝑡 + 1) = 𝑓 ⎝𝑤𝑖𝑗 (𝑡)𝑥𝑗 (𝑡) − 𝜃𝑖 (𝑡) + 𝛼𝜓𝑖 (𝑡)⎠ ,(4)
𝑗=1

where 𝛼 is a weight of the glial effect. A peek of the generated
pulse is changed according to 𝛼. We choose an optimal valueof 𝛼 for solving a task by a heuristic search. The generated
pulse is independent from the learning of the MLP, thus
this pulse can gives an energy to the network and helps forescaping out from the local minimum. 𝜓 is updated by Eq. (1).
Equations (3) and (4) are used as a sigmoidal function to an
activating function of neuron and is described by Eq. (5).
1𝑓 (𝑎) =,
(5)
1 + 𝑒−𝑎where 𝑎 is an inner state of the neuron.

III.

S IMULATION

We use five kinds of the MLPs for comparison of theperformance.
(1)
(2)
(3)
(4)
(5)

The standard MLP
The MLP with random noise
The MLP with random timing pulses
The MLP with pulse glial chain
The MLP with pulse glial chain having oscillatory
excitation threshold

The standard MLP (1) does not have the external unit, thus
this MLP often falls into the local minimum. The MLP
with random noise (2) has a uniformed random noise in the
excitation threshold of the neurons in the hidden-layer. The
MLP with random timing pulses (3) has a pulse oscillationin the excitation threshold of the neurons in the hidden-layer,
and this pulse is generated at random. The MLP with pulse
glial chain has a glial pulse in the excitation threshold of theneurons in the hidden-layer. The MLP with pulse glial chain
having oscillatory excitation threshold is the proposed MLP. In
this model, the glias generate and transmit pulse, and the glia
excitation threshold oscillates within a defined value. We use a
Two-Spiral Problem (TSP) for a task of the MLPs. The TSP isa famous task for the artificial neural network and has a highnonlinearity [11][12]. In this task, we input the coordinates ofthe spiral points, and the MLP learns the classification of eachspiral coordinate. We obtain ideal classification from norms
between the each coordinates and the learning spiral points.
The learning spiral points and the ideal classification are shownin Fig. 3.
1

1

0.8

0.8

0.6

y

y

0.6
0

0.4

1

0.2

0.2

0

0
0

Fig. 3.

0

0.4

1

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

x

x

(a) Supervised spiral points.

(b) Ideal classification.

1

Target points.

We obtain the experimental results from 100 trials about
each MLP. In each trial, we give the different initial weights
of connections. The MLP has 50000 iterations during one trial
and learns 130 data sets of spiral points. We use the threelayers MLP, and the number of neurons in each layer is 2-401. We use Mean Square Error (MSE) for the error function.
The MSE is described by Eq. (6).

the learning data sets. We compare the output of the MLP
and supervised classification of the spiral points. The learning
performance of each MLP is shown in Table I. The standard
MLP (1) is the worst of all in the average error, the minimum
error and the maximum error. We can say that the standard
MLP (1) falls into local minimum. The MLP with random
noise (2) and the MLP with random timing pulse (3) reduce
the error than the standard MLP (1), however these methods
do not have a large difference from the standard MLP (1).
On the other hand, the MLP with pulse glial chain (4) andthe MLP with pulse glial chain having oscillatory excitationthreshold (5) reduce the error more than the others. In the caseof the MLP with pulse glial chain having oscillatory excitationthreshold (5), the maximum error is smaller than the MLP withpulse glial chain (4). From this result, we consider that theoscillatory excitation threshold of the glia helps escaping out
from the local minimum. Thereby, the maximum error of the
proposed MLP (5) is reduced than that of the MLP with pulse
glial chain.
TABLE I.

Minimum
0.008310.00047
0.007400.000670.00069

Maximum
0.23857
0.24278
0.26349
0.116640.08139

Std. Dev.
0.055540.057420.057420.02226
0.01688

Next, we show the difference of the learning curves between the MLP with pulse glial chain (4) and the MLP withpulse glial chain having oscillatory excitation threshold (5)as shown in Fig. 4. The learning curves are obtained fromthe average error in each iteration. The error reductions are
similar from start of the learning to 10000 iterations. After
that, the error reduction of the MLP with pulse glial chain
having oscillatory excitation threshold (5) is faster than the
MLP with pulse glial chain (4). We consider that the pulse
generations both methods are similar to each other at start ofthe learning, because the neurons’ outputs are often changedby the learning. When the neuron output becomes stable ineach neuron, the oscillatory excitation threshold influences thepulse generation. Hence, the MLP with pulse glial chain having
oscillatory excitation threshold (5) can reduce the error thanthe MLP with pulse glial chain (4).
1
MLP with pulse glial chain (4)MLP with pulse glial chain
having oscillatory excitationthreshold (5)

MSE

𝑁

1 ∑
𝑀 𝑆𝐸 =(𝑇𝑛 − 𝑂𝑛 )2 ,𝑁 𝑛=1

Average
0.12269
0.108470.114390.019900.01436

(1)
(2)
(3)
(4)
(5)

L EARNING PERFORMANCE .

(6)

0.1

where 𝑁 is the number of learning datum, 𝑇 is a target value,
and 𝑂 is an output of MLP. We obtain results which are
an average error, a minimum error, a maximum error, and astandard deviation of the results.
0.01
0A. Learning performance

10000

20000

30000

IterationsFirstly, we obtain a learning performance. The learning
performance means that the accuracy of the classification for

1332

Fig. 4.

Learning curves.

40000

50000

B. Classification performance
We show the classification performance of the MLPs. The
classification performance shows a generalization capability.
We give the unlearning coordinates to the learned MLPs from0 to 1 in x-y plane. And also, we compare the output of the
MLP and the ideal classification in each coordinate. The classification performance is shown in Table II. The standard MLP(1), the MLP with random noise (2) and the MLP with randomtiming pulse (3) have similar classification performance. The
MLP with pules glial chain (4) and the MLP with pulse glial
chain having oscillatory excitation threshold (5) have a betterclassification performance than the others. Especially, the MLP
with pulse glial chain having oscillatory excitation threshold(5) obtains the smallest of all in the minimum error. In general,
the MLP becomes over fitting when the MLP is excessively
learned. In the MLP with pulse glial chain having oscillatory
excitation threshold (5) has a high learning performance and ahigh classification performance. From this result, we can saythat this MLP has a high searching ability of the solutions.

1

1

0.8

0.8

0

0.4

y

0.6

y

0.6

0.2
0

1

0.2

0

0.2

0.4

0.6

0.8

0

1

0

0.2

0.4

0.6

0.8

1

x

x

(a) Standard MLP.

(b) MLP with random noise.

1

1

0.8

0.8
0.6
0

0.4

y

y

0.6

0

0.4

1

1

0.2

0.2
0

0

0.4

1

0
0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

x

x

(c) MLP with random timing (d) MLP with pulse glial chain.
pulses.
1
0.8
0.6

C LASSIFICATION PERFORMANCE .

y

TABLE II.

0

0.4

(1)
(2)
(3)
(4)
(5)

Average0.21782
0.19278
0.20432
0.12538
0.11884

Minimum
0.10565
0.10460
0.120820.08027
0.06693

Maximum
0.294770.33065
0.31958
0.196390.19198

Std. Dev.
0.03858
0.04434
0.03851
0.02625
0.02344

0
0

0.2

0.4

0.6

0.8

1

x

(e) MLP with pulse glial chain
having oscillatory excitationthreshold.

Finally, we show examples of the classification of theunlearning coordinates of the MLPs in Fig. 5. We obtain theclassification image about average results in the MLPs. The
standard MLP (1), the MLP with random noise (2) and the
MLP with random timing pulses (3) cannot draw the two
spirals. On the other hand, the MLP with pulse glial chain(4) and the MLP with pulse glial chain having oscillatory
excitation threshold (5) can draw the two spirals. From thisresult, we consider that the MLP with pulse glial chain having
oscillatory excitation threshold can solve the TSP.

IV.

1

0.2

C ONCLUSION

In this study, we have proposed the MLP with pulse
glial chain having oscillatory excitation threshold which isinspired from the glia functions. The glias are one-by-one
connected with the neurons in the hidden-layer. The glia
generates the pulse according to the connecting neuron output.
This pulse transmits to the neighboring glias and the neurons.
In the proposed model, we introduce the oscillatory excitationthreshold of the glia. By the oscillatory excitation threshold,the excited glias are sometimes randomly chosen. Thereby,
the pulse glial chain having oscillatory excitation thresholdgives more energy to the MLP performance than the previous
method. We confirmed that the proposed MLP has the better
performance than the conventional MLPs.

ACKNOWLEDGMENT
This work was partly supported by MEXT/JSPS Grant-inAid for JSPS Fellows (24⋅10018).

1333

Fig. 5.

Examples of classification results.

R EFERENCES[1] P.G. Haydon, “Glia: Listening and Talking to the Synapse,” Nature
Reviews Neuroscience, vol.2, pp.844-847, 2001.
[2] R.D. Fields, B. Stevens-Graham, “New Insights into Neuron-Glia Communication,” Science, vol.298, pp.556-562, 2002.
[3] G.I Hatton, V. Parpura, “Glia Neuronal Signaling,” 2004, Kluwer
Academic Publishers.
[4] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami and K. Inoue, “Dynamic
Inhibition of Excitatory Synaptic Transmission by Astrocyte-Derived
ATP in Hippocampal Cultures,” Proc. National Academy of Science of
U.S.A, vol.100, pp.11023-11028, Mar. 2003.
[5] S. Ozawa, “Role of Glutamate Transporters in Excitatory Synapses inCerebellar Purkinje Cells,” Brain and Nerve, vol.59, pp.669-676, 2007.
[6] G. Perea and A. Araque, “Glial Calcium Signaling and Neuro-Glia
Communication,” Cell Calcium, vol.38, pp.375-382, 2005.
[7] S. Kriegler and S.Y. Chiu, “Calcium Signaling of Glial Cells along
Mammalian Axons,” The Journal of Neuroscience, vol.13, pp.42294245, 1993.
[8] M.P. Mattoson and S.L. Chan, “Neuronal and Glial Calcium Signalingin Alzheimer’s Disease,” Cell Calcium, vol.34, pp.385-397, 2003.
[9] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol.323-9, pp.533-536,
1986.
[10] M. Matsumoto and T. Nishimura, “Mersenne twister: a 623dimensionally equidistributed uniform pseudo-random number generator,” ACM Transactions on Modeling and Computer Simulation, vol.8(1), pp.3-30, 1998.
[11] J.R. Alvarez-Sanchez, “Injecting Knowledge into the Solution of the
Two-Spiral Problem,” Neural Computing & Applications, vol.8, pp.265272, 1999.
[12] H. Sasaki, T. Shiraishi and S. Morishita, “High Precision Learning
for Neural Networks by Dynamic Modification of Their Network
Structure,” Dynamics & Design Conference, pp.411-1–411-6, 2004.