2011 International Workshop on Nonlinear Circuits,Communication and Signal Processing
NCSP'11, Tianjin, China, March 1-3, 2011

Multi-Layer Perceptron with Glial Network
Influenced by Local External Stimulus
Chihiro Ikuta, Yoko Uwate and Yoshifumi Nishio
Tokushima University
2–1 Minami-Josanjima, Tokushima, Japan
Phone:+81-88-656-7470
Email: {ikuta, uwate, nishio}@ee.tokushima-u.ac.jp

Abstract
We have proposed the glial network which was inspired fromthe feature of glias. The glias are nervous cell existing in thebrain and transmit signal each other like neurons. In the glialnetwork, the glias connect to the neurons and other glias. Theglias trade information each other by this network.
In this article, we investigate the glial network when only
one glia is stimulated by an external noise. The external noisepropagates the glial network and influences to the MultiLayer Perceptron (MLP). By the eﬀect of the one-way influence via the glial network, a kind of position-dependingfeature appears in the MLP. The simulation results show thatthe proposed network possesses better learning performance,
more biased anti-damaging property, and better generalization capability than the conventional networks.

of the neural networks. In [5], we proposed the MLP with theglial network whose glias generated independent oscillations
and these oscillations propagated neurons and other glias. We
confirmed by computer simulations that the glial network improved the learning performance of the MLP by connecting
the neurons more eﬀectively than the conventional networks.
In this paper, we research a learning result of the MLP
when it is influenced by local stimulus. In the biological system, a neural network is not always aﬀected by uniformed
external stimulus. We assume that only one glia is aﬀectedby external stimulus and that this influence propagates in theglial network. We give random oscillation to one glia and theMLP learns data having diﬀerent amplitude oscillations by
the glial network. We confirm that the learning result of MLP
is changed by the biased oscillations.
2. MLP with Glial Network Influenced by Local ExternalStimulus

1. Introduction
Human’s brain is made by nervous cells which are almost
neurons or glias. In particular, the neurons were investigatedby many researchers. Because the neurons were known thatthey make thinking of human by connected to each other. TheMulti-Layer Perceptron (MLP) is one of feed forward neuralnetworks and is useful to perform several tasks, for example,
pattern recognition, pattern classification, data mining and soon. Back Propagation (BP) algorithm is a learning algorithm
for the MLP using the steepest decent method [1].
We have proposed the glial networks to improve the performance of MLP. The glial network was inspired from thefeature of the glias which are existing in the brain. The glias
did not attract researchers’ attentions for a long time, because
the glias have been believed not to use electrical signals. Recently, some researchers discovered that the glias transmit
signals by using ions [2]. In the brain, the glias influence eachother and the ions aﬀect neurons’ thresholds [3][4]. We tried
to exploit the glial network’s behavior such that another network located close to the neural network helps the functions

The MLP is the most famous feed forward neural network.
Several methods using the MLP have been proposed for solving many kinds of tasks. This network has some neuron layers and the weights between the layers are learned by the BPalgorithm. In this study, we use the MLP with three layers(2-10-1) and a glial network connected to the hidden layer.
Figure 1 shows an example of the MLP with the glial network.
2.1. Neuron Updating Rule
The standard neuron updating rule is given by Eq. (1).
 n
∑xi (t + 1) = f  wi j (t)x j (t) − θi (t) ,(1)j=1where x : input or output, w : weight parameter, θ : threshold
and f : output function. The parameters w and θ are learned
by using the BP algorithm.

- 199 -…

We use a = 4.0 which makes the time series as fully developed chaotic oscillation. We compare the performances
External stimulusof five diﬀerent MLPs, which are the conventional MLP, theMLP with the glial network (α = 0.15, β = 0.9), the MLP
with the glial network (α = 0.2, β = 0.8), the MLP with theglial network (α = 0.4, β = 0.6) and the MLP with inputted
random noise (noise amplitude is 1.0). We decided the parameter combinations of the MLP with the glial networks asNeuron
the oscillation amplitudes become around 1.0.
Figure 2 is an example of the chaotic time series (100Gliapoints) which is obtained by giving a certain initial value tothe logistic map. The BP learning for the MLPs are carried
Figure 1: MLP with glial network influenced by local external out by giving two successive points of the chaotic time series
as an input and the following one point as an output. Thestimulus.
learning is repeated for 100 diﬀerent sets like Fig. 3.

…

The updating rule of the hidden layer’s neurons of the proposed neural network with the glial network is modified asEq. (2).
 n
∑xi (t + 1) = f  wi j (t)x j (t) − θi (t) + αΨi (t) ,(2)j=1where Ψ : output of the glias, α : weight of glia outputs. We
use the sigmoid function for the output f as Eq. (3).
f (a) =1.
1 + e−a(3)Figure 2: Chaotic time series (a = 4.0).

2.2. Glial Network
In the biological neural network, it is known that the glias
aﬀect to the neighboring neurons over a wide range by making their outputs to propagate in the network.
In this study, we assume that the glias do not generate oscillations by themselves but only one glia located on an edgeof the network is influenced by a local external stimulus. The
external noise propagates the glial network with decaying.
The output of the glias can be given as Eq. (4).
Ψi (t) = βi−1 ψ(t − i − 1) (i = 1, 2, 3, · · · 10),

Input 1

Input 2
11

0.8

0.8
0.6
0.4

Input 1

0.6

Learning

0.4

0.2

0.2

00

Learning
10.8Learning
Input 2Input 2

0.60.4

Input 10.20

1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 10

1st learning step

2nd learning step

3rd learning step

Figure 3: Learning procedure.

(4)

We evaluate three kinds of performances with this task;
where ψ is the local external stimulus given to the first glia learning performance, anti-damaging property and generaland β denotes attenuation parameter. In this study, we input ization capability. We use the following Mean Square Error
uniform random noise as external stimulus ψ. The noise ef- (MSE) for evaluate these performances.
fect is propagating the glial network, however, the noise isattenuated by β and delayed.
N1 ∑MS E =(6)(ti − Oi )2 .
N i=1
3. Simulation Results
In this section, we show the performance of the proposed 3.1. Learning Performance
MLP with the glial network by learning chaotic time series.
First, we investigate the learning performance of the MLPs
We use the logistic map to generate chaotic time series. Thebycalculating the time evolution of the errors. The number
logistic map is formulated by Eq. (5).
of the trials is 200 and the MLPs learn 500000 times duringx(t + 1) = ax(t){1 − x(t)}.
(5) one trial. We calculate the average of error (Avg. Err.), the- 200 -…

Noise–
0.121
0.058
0.163
0.024

…

Table 1: Learning performance (10−3 ).
Conv.
Glia
Glia
Glia
–
β = 0.9 β = 0.8 β = 0.6
Avg. Err. 0.327
0.162
0.194
0.248
Min.
0.075
0.054
0.054
0.062
Max.
1.467
1.720
1.564
1.499
St. Dev. 0.478
0.2040.304
0.382

…

minimum error (Min.), the maximum error (Max.) and thestandard deviation (St. Dev.).
Table 1 shows the obtained results. We can see that theconventional MLP is the worst. In this result, the MLP withrandom noise performs the best. However, the results of theMLPs with the glial networks are similar to that of the MLP
with random noise. Further, the maximum error of the MLPs
with the glial networks are larger than that of the MLP withrandom noise. We consider that the averaged values of theMLPs with the glial networks are influenced by a few very
bad results.

Figure 5: Examples of damaging MLPs.
Figure 6 shows the position dependence of the average error for 200 trials after cutting one neuron. We can say thata neuron is very important for the corresponding MLP if theerror becomes large after cutting the neuron. On the contrary,
if the performance is similar even cutting the neuron, the neuron is not useful for the MLP. In Fig. 6, the horizontal labels
denote the positions of the neurons, hence, the eﬀect of theexternal noise is stronger for smaller label of the positions forthe MLPs with the glial networks.
0.2
0.16

ErrorFigure 4 shows an example of the learning curves. Theerror of the conventional MLP becomes constant early and is
not improved any more. This is because of the local minimum
problem of the optimizations. The other MLPs seem to beable to escape out from local minima. We can see that theMLPs with the glial networks can find good states earlier thanthe MLP with random noise in this example.

0.18
0.14

Convntional

0.12

Glial network (β=0.9)

0.1

Glial network (β=0.8)

0.08
Glial network (β=0.6)

0.06

Noise

0.04
0.02
0123456789
The number of neuron position

10Figure 6: Performance after damaging one neuron.
From this figure, the MLP with random noise is similar tothe conventional MLP. As we can expect, for the conventional
MLP and the MLP with random noise, every neurons have thesimilar importance for the performance. However, we can see
clearly from the figure that the importances of the neurons of
the MLPs with the glial networks are diﬀerent depending on
their positions.
3.3. Generalization Capability
Figure 4: Learning curve of five MLPs.

Finally, in order to evaluate the generalization capabilitiesof the MLPs, we give the MLPs unknown time series generated by the same logistic map after the learning and evalu3.2. Anti-Damaging Property
ate the errors between the outputs and the correct following
Next, in order to investigate the anti-damaging property of points. We use ten diﬀerent time series including 100 points
the MLPs, we cut one neuron in the hidden layer after the for the test data. The learning data and one example of thelearning as shown in Fig. 5. This means that the MLPs must test data are shown in Fig. 7.
The error performance are summarized in Table 2. The
operate by only 9 neurons in their hidden layers. We change
performance of the conventional MLP is the worst. The erthe position of the cut neuron and evaluate the errors.

- 201 -

1

glial networks possessed better learning performance, morebiased anti-damaging property, and better generalization capability than the conventional networks.
At the moment, we could not maintain that the proposed
MLP performs much better for a certain task than the conventional one in the sense of learning performances, but theexistence of the glial network produces novel features whichhave not realized in the conventional artificial neural networks; such as individuality of neurons. Real brains mustbe, of course, non-uniform and hence we believe that a kindof individuality will give wider chance to realize more intelligent artificial neural networks.

0.9
0.8
0.7
0.6
Learning data

0.5

Test data

0.4
0.3
0.2
0.1
0020

40

60

80

100TimeFigure 7: Learning data and test data.

Acknowledgment
ror of the MLP with random noise has the smallest standard
deviation. The MLPs with the glial networks has the large This work was partly supported by JSPS Grant-in-Aid for
diﬀerence between the maximum error and the minimum er- Scientific Research 22500203.
ror.
References
Table 2: Generalization capability (10−3 )
Conv.
Glia
Glia
Glia
–
β = 0.9 β = 0.8 β = 0.6
Avg. Err. 0.386
0.353
0.343
0.342
Min
0.057
0.076
0.083
0.068
Max
2.131
2.522
2.823
2.845
St. Dev. 0.550
0.230
0.280
0.418

Noise–
0.355
0.120
0.532
0.271

[1] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol. 323-9, pp. 533-536, 1986.
[2] P.G. Haydon, “Glia: Listening and Talking to theSynapse,” Nature Reviews Neuroscience, vol. 2, pp. 844847, 2001.

[3] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami and K. Inoue, “Dynamic Inhibition of Excitatory Synaptic Transmission by Astrocyte-Derived ATP in Hippocampal CulNext, we add noise (5%) to the test patterns and investigatetures,” Proc. National Academy of Science of U.S.A,
the denoising ability. The results are summarized in Table 3.
vol. 100, pp. 11023-11028, 2003.
We can see that the performance of the conventional MLP isthe worst. The MLPs with the random noise has the better [4] S. Ozawa, “Role of Glutamate Transporters in Excitatory
performance than the MLPs with the glial networks.
Synapses in Cerebellar Purkinje Cells,” Brain and Nerve,
vol. 59, pp. 669-676, 2007.
Table 3: Generalization capability for noisy data (10−3 )
Conv.
Glia
Glia
Glia
Noise–β = 0.9 β = 0.8 β = 0.6–Avg. Err. 8.748
6.243
6.562
7.7729 6.000
Min
3.635
3.195
3.428
3.316
3.222
Max
58.99
20.97
17.39
42.99
17.84
St. Dev. 4.779
1.340
1.457
3.404
1.254

[5] C. Ikuta, Y. Uwate and Y. Nishio, “Chaos Glial Network
Connected to Multi-Layer Perceptron for Solving TwoSpiral Problem,” Proc. ISCAS’10, pp. 1360-1363, May
2010.

4. Conclusion
In this article, we have researched the MLP with the glialnetwork when the glias do not generate any oscillations butpropagate the eﬀect of the external stimulus. By computer
simulations, we confirmed that the proposed MLPs with the- 202 -