2013 International Workshop on Nonlinear Circuits,
Communications and Signal Processing
NCSP'13, Isrand of Hawaii, Hawaii, USA. March 4-7, 2013

Investigation of Glia-Neuron Network with Group Learning of Hidden-Layer Neurons
Chihiro Ikuta† , Yoko Uwate† , Yoshifumi Nisho† , and Guoan Yang‡†

Dept. of Electrical and Electronic Eng., Tokushima University, 2-1 Minami-Josanjima, Tokushima Japan
Email: {ikuta, uwate, nishio}@ee.tokushima-u.ac.jp‡
Dept. of Automation Science and Technology, School of Electronic and Information Eng.,
Xi’an Jiaotong University, No. 28 Xianning West Road, Xi’an City, Shaanxi Province, China
Email: gayang@mail.xjtu.deu.cn

abstract
A glia is an important cell for a higher brain function. This
cell can transmit signals to neurons and glias by using ion
concentrations. From this feature, we propose a glia-neuron
network with group learning of hidden-layer neurons. In thismodel, the neurons in a hidden-layer are separated to somegroups. We connect glias to each group. These neurons are
switched between a learning term and a non-learning term
according to firing of the glias. The time length of learningterm is controlled by the glia firing. The glia firing depends
on the integration of the connecting neuron outputs. Thus, theneurons and the glias are correlated to each other. By simulations, we confirm that the performance and characteristics ofthe proposed model.
1. Introduction
The glia is an important nervous cell for a higher brain
function. However, this cell had not been investigated about
detail, because this cell was considered to a support cell of aneuron. Recently, some researchers discovered novel glial
functions [1]-[3]. The glia uses an ion concentration to atransporter of signals. The ions are a Ca2+ , a glutamate acid,an adenosine triphosphate, and so on [4][5]. Among them,
we notice the Ca2+ . The glia release the Ca2+ when glia isreceived stimulus from the neurons [6]. Moreover, the Ca2+transmits to wide range in the brain and composes the brain
function.
N. Takatas reported that the Ca2+ affects the synaptic
Long-Term Potentiation (LTP) by an experiment on living animals [7]. In that research, they used two mice. One mouse
has normal glia. On the other hand, the mouse has the additional genetic defect glia which cannot become detached toCa2+ . In the genetic defect mouse, they could not observe
the LTP. In the normal mouse, they could observe the LTP
by the increase of the response. Moreover, the increase of the
D-serine was observed in the increase of the Ca2+ concentration. The D-serine is the important ion for LTP. Thereby, we
can say that the glia corresponds to the LTP of the synapse.

In this study, we propose a glia-neuron network with grouplearning of hidden-layer neurons. The glia closely relate to
the neuron works in the biological system. The proposedmodel is inspired from relationships between the glias andthe neurons. In this model, we connect the glias to the neurons in the hidden-layer. These neurons are separated to somegroups. The groups are switched between a learning term andnon-learning term according to firing of the glia. The glias
receive the output of connecting neurons in the same groupand integrate the outputs of neurons with respect to the time.
Firing time length of the glia is decided by amount of the integrated outputs. The glia does not have a learning method,
however the neurons are learned by Back Propagation (BP)algorithm [8]. Thus, the responses of glias are dynamicallychanged during the iterations. We consider that the artificial
neural network is improved by the relationships between theneuron and the glia. By the computer simulations, we confirm that the performance and characteristics of the proposed
network.

2. Proposed method
The Multi-Layer Perceptron (MLP) is a famous feed forward neural network. It is applied to various nonlinear tasksuch as an approximation of function, a classification task, adata mining, and so on. In general, the MLP is learned by BP
algorithm [8] which uses the steepest decent method. However, the BP algorithm often falls into local minima. We givethe noise to the MLP for escaping out from the local minimum. The noise is efficient to difficult tasks, because a difficult task has many local minima. However, the noise encumbers the learning when the MLP solves the easy task. In previous study, we proposed the glia-neuron network model [9].
The neurons in the hidden-layer have a switching between thelearning and non-learning term. The learning term and nonlearning term are periodically switched. We confirmed thatthe switching of two terms improves the MLP learning performance. However, the learning of this network is converged
earlier, because the parameters are fixed.

- 440 -

In this study, we propose the glia-neuron network with again. The neurons’ outputs are changed by learning, thereby
group learning of hidden-layer neurons. The construction of the time length of learning term is different from the previous
the proposed network is presented in Fig. 1. In this model, time length.
we connect glias with the neurons in the hidden-layer. They
Group 1
influence each other during the iterations. The neurons are
separated to some groups. Each group is switched between
the learning term and non-learning term according to firingof the connected glia. Firing of the glias periodically change.
Delay
In this model, each glia has different time length of the firing.
The time length of firing is decided by the connecting neuron
outputs. The glia integrates the neurons’ outputs in the connecting group with respect to iterations. The time length oflearning term is decided by Eq. (1).
Lg =

T
N
Lmax ∑ ∑Ognt ,
T N t=1 n=1(1)

where L is a time length of learning term, g is a group number, T is a iteration length from a start time of learning termof first group and the end time of learning term of last group,
N is the number of neurons in the same group, and O is an
output of one neuron. The glia is not learned, however theneurons are learned by BP algorithm. The neuron output dynamically changes, thus the time length of learning term isdynamically changed with the iterations.
Learning term

The number of neurons

Firing glia

Figure 2: Time flow of the learning term and non-learning
term.

Static glia
Non-learning term

Figure 1: Proposed MLP.

3. Simulations
In this section, we show the simulation results of the proposed network. We use the Two-Spiral Problem (TSP) for
learning task. The TSP is a famous task for the artificial neural network [10]. It is linearly-inseparable problem, thereby
it has a high nonlinearity [11]. The MLP receives the coordinates of the spirals for the inputs and learns the correspondingclassifications. In this simulation, the MLP is composed of
the neurons (constructed 2-40-1). The iterations are 1000000
times. Figure 3 shows the two spirals for the MLP. In thisfigure, the spirals are composed of 130 points. For the index
of the error, we use the Mean Square Error (MSE) for both
simulation results. The MSE is described by Eq. (2).

Next, we show the flow of the learning of the proposed
MLP in Fig. 2. In this example, one group is composed of
two neurons. Every group is connected with one glia. By
this glia, the time length of learning term is decided, thereby
the time length of learning term is different each other. The
learning term and the non-learning term are changed with iterations. During the learning term, the neurons are learned
by BP algorithm. On the other hand, the weights of connections between the hidden-layer neuron and the input-layerP
1 ∑neurons are not updated. First group goes into the learning(Tn − On )2 ,(2)
M SE =
P n=1
term. Other groups are in the non-learning term. Second
group goes into the learning term, then the first group remain
the learning term. Every group repeats changing the learning where P is the number of the pairs of the input and the suterm and the non-learning term. When the final group fin- pervised value, T is a supervised value, O is an output of theishes the learning term, the first group starts the learning term MLP.

- 441 -

1

Table 1: Learning performance.

0.8

Input 2

0.6

Proposed
PreviousStandard

0.4
0.2
0
0

0.2

0.4

0.6

0.8

Ave.
0.02390.03120.0854Min.
0.00030.00020.0006Max.
0.2426
0.16200.2463

Std. Dev.
0.0387
0.03640.05581

Input 1Figure 3: Two-spiral problem.

3.2 Parameter characteristics

We compare three different MLPs which are the proposed
MLP, the previous proposed MLP [9], and the standard MLP.
The previous proposed MLP is similar to the proposed MLP.
The neurons in the hidden-layer are periodically switched between the learning term and non-learning term. However, thetime length of the learning term is constant value. Thus, this
MLP becomes the learning in a higher rate of periodic. The
standard MLP does not have the external unit, thus this MLP
often falls into local minimum.
Figure 4 shows an example of the learning curves of three
MLPs. The learning curve of the standard MLP is converged
earlier because this MLP is trapped into the local minimum.
The learning curve of the previous proposed MLP oscillates
to about 50000 times. By this oscillation, this MLP can finda better solution than the standard MLP. However, the learning curve converges over 60000 times. The proposed MLP
can reduce the error the best of all. Moreover, the oscillation of the curve is observed all iterations. From these result,
we can say that the proposed MLP obtains the energy for escaping out from the local minimum by the glial network all
iterations.

Next, we show the parameter characteristics of the proposed MLP. We define the some parameters of the proposed
MLP. The performance of the proposed MLP is changed according to change of the parameters. We change three kindsof parameters which are the number of neurons in a samegroup, the delay of start time of learning term, and the maximum time length of learning term. Figures 5-7 show theparameter characteristics. Each figure is result for different
number of neurons in the same group. In Fig. 5, the MLP
performance is better when the delay of start time of learningterm is smaller. The dependency of the delay is changed according to the increase of the number of neurons in the samegroup. When the time length of learning terms is increased,
the error is almost reduced. However, the errors increase overa point. From these results, we consider that amount of theoverlap of learning term is important for the MLP learning
performance.
0.25
DELAY=1
DELAY=5
DELAY=10
DELAY=20

0.2

MSE

3.1 Learning performance

0.15

0.1

0.05

1

0

Mean square error

20

30

40

50

60

Length of learning term
0.1

0.01

Figure 5: Parameter characteristics. (The number of neurons
in the same group is 2.)

Proposed
PreviousStandard

0.25

0.001
0

20000

40000

60000

80000

100000Iteration

0.2

DELAY=1
DELAY=5
DELAY=10
DELAY=20

0.15

MSE

Figure 4: Learning curves.

0.1

Table 1 shows the statistic result of the MLPs. From this
0.05
result, the proposed MLP has the best of all for the average of0
error. However, the maximum error is a worse than the pre20
30
40
50
60
Length of learning term
vious MLP. For this reason, we consider that the glias change
the time length of the learning term. The time length of learning term become too small, thus the overlap of the learning Figure 6: Parameter characteristics. (The number of neurons
in the same group is 5.)
term is not happened.

- 442 -

0.25

prove the MLP performance. By the solving TSP, we confirmed the glial-neuron network improves the MLP learning
performance, moreover the MLP has high dependency for thetime length of overlap of learning term.

DELAY=1
DELAY=5
DELAY=10
DELAY=20

0.2

MSE

0.15

0.1

Acknowledgment

0.05This work was partly supported by MEXT/JSPS Grant-in-Aid for
JSPS Fellows (24·10018).

0
20

30

40

50

60

Length of learning term

Figure 7: Parameter characteristics. (The number of neurons
in the same group is 10.)
Finally, we show the performance change for a ratio of thetime length of learning term to all iterations in Fig. 8. Thisvalue means the length of overlap of learning term. We divide
five different range of value and calculate the average of theerror. From this figure, the performance improves to 0.4-0.6,
after that the performance decreases over 0.4-0.6. We cansee that the proposed MLP has high dependency for the time
length of overlap of learning term. Thus, when we use theproposed MLP, we need to consider the time length of overlapof learning term.

0.160
0.140

[2] S. Kriegler and S.Y. Chiu, “Calcium Signaling of Glial Cells
along Mammalian Axons,” The Journal of Neuroscience,
vol. 13, pp. 4229-4245, 1993.
[3] S. Ozawa, “Role of Glutamate Transporters in Excitatory
Synapses in Cerebellar Purkinje Cells,” Brain and Nerve,
vol. 59, pp. 669-676, 2007.
[4] M.P. Mattoson and S.L. Chan, “Neuronal and Glial Calcium
Signaling in Alzheimer’s Disease,” Cell Calcium, vol. 34,
pp. 385-397, 2003. pp. 11023-11028, Mar. 2003.

[6] G. Perea and A. Araque, “Glial Calcium Signaling and NeuroGlia Communication,” Cell Calcium, vol. 38, pp. 375-382,
2005.

0.120

MSE

[1] P.G. Haydon, “Glia: Listening and Talking to the Synapse,”
Nature Reviews Neuroscience, vol. 2, pp. 844-847, 2001.

[5] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami and K. Inoue,“Dynamic Inhibition of Excitatory Synaptic Transmission by
Astrocyte-Derived ATP in Hippocampal Cultures,” Proc. National Academy of Science of U.S.A, vol. 100,

0.180

0.100
0.080

[7] N. Takata, T. Mishima, C. Hisatsune, T. Nagai, E. Ebisui,
K. Mikoshiba, and H. Hirase, “Astrocyte Calcium Signaling
Transforms Cholinergic Modulation to Cortical Plasticity In
Vivo,” The Journal of Neurocience, vol. 31, pp. 18155-18165,
Dec. 2011.

0.060
0.040
0.020
0.000
0.0-0.2

0.2-0.4

0.4-0.6

0.6-0.8

0.8-0.9

Ratio of Length of learning term to iterations

Figure 8: Learning performance for time length of overlap oflearning term.

4. Conclusions

References[8] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature,
vol. 323-9, pp. 533-536, 1986.
[9] C. Ikuta, Y. Uwate, Y. Nishio, and G. Yang, “Multi-Layer Perceptron Decided Learning Neurons by Regular Output Glias,”
Proc. NOLTA’12, pp. 719-722, Oct. 2012.
[10] J.R. Alvarez-Sanchez, “Injecting knowledge into the Solutionof the Two-Spiral Problem,” Neural Computing & Applications, vol. 8, pp. 265-272, 1999.

In this study, we have proposed the glia-neuron network [11] H. Sasaki, T. Shiraishi and S. Morishita, “High precision learnwith group learning of hidden-layer learning neurons. In
ing for neural networks by dynamic modification of their netthis model, the neurons in the hidden-layer is decomposed
work structure,” Dynamics & Design Conference, pp. 411-1–
to some learning group. Each group is connected with the411-6, 2004.
glia. Each group has the learning term and non-learningterm which are switched according to the connecting glias.
The time length of learning term is changed by the glial response and the glial responded is decided by integration of
the neuron output in the connecting neurons. We consider
that the relationships between the glia and the neuron im-- 443 -