IEEE Workshop on Nonlinear Circuit Networks
December 15-16, 2017

Learning Method Using Acetylcholine
in Feed Forward Neural Networks
with Back Propagation Learning
Kazuki NAGAO, Shinsaburo KITTAKA, Yoko UWATE and Yoshifumi NISHIO
Dept. of Electrical and Electronic Engineering of Tokushima University
2-1 Minami-Josanjima, Tokushima-shi, Tokushima-ken 770-8506, Japan
Email: {k-nagao, kittaka, uwate, nishio}@ee.tokushima-u.ac.jp
Abstract—In this study, we propose back propagation learningin feed forward Neural Networks using the characteristic ofacetylcholine. Acetylcholine is a major neurotransmitter in thehuman brain. The main work of acetylcholine is to excite thenerves and enhance motivation for learning and memory. In
the neural networks, this function is used to approximate theinformation processing performed by the human brain. Our
purpose is to reduce learning error rate.

I. I NTRODUCTION
Neural Network (NN) and Deep Learning are made toimitate the human nervous system. Such systems learn to dotasks by considering examples, generally without task-specificprogramming. For example, in 2012, in image recognition,
they might learn to identify images that contain cats byanalyzing example images that have been manually labeled
as “cat” or “no cat” and using the analytic results to identify
cats in other images (Fig. 1). They have found most use
in applications difficult to express in a traditional computer
algorithm using rule-based programming [1].

Fig. 1.

Images of cat recognized by Artificial Intelligence (AI).

The human brain processes many and various kinds ofinformation got on the basis of the five senses (sight, hearing,smell, taste and touch). And we can judge things and put us
into action. Thereby, the human brain has high bandwidth,
high speed parallel processing and flexibility. So, artificial
intelligence was created to do the same work as the humanbrain on a computer. Typical examples are NN and Deep
Learning [2]. In recent years, they have been applied in various
fields and used for applications. For example, smartphone, car,stock price forecast, robot and so on.
There are the following mainly substances constituting
the human neural circuit. First, there are many nerve cells(neuron) in the human brain, and information is passed on next
neurons through the connection, and we can store and judge
things. Second, each connection (synapse) between neurons
can transmit a signal to another neuron. The receiving neuron

can process the signals and then signal downstream neurons
connected to it. In machine learning, neurons may have stated,
generally represented by real numbers, typically between 0 and
1. Neurons and synapses may also have a weight that varies aslearning proceeds, which can increase or decrease the strengthof the signal that it sends downstream. Third, neurotransmitteris substances generated from synapse. It excites and suppresses
neurons. It has a role to promote learning. Other substances include glial cells. Glial cells include several types of cells suchas astrocytes, and oligodendrocytes apart from the neurons inthe brain. In particular, astrocytes are known to be importantin higher function and are therefore sometimes simply called
glial cells. Now, neuron, synapse and glia are used in NN [3]and Deep Learning. However, neurotransmitter is not used at
all in NN and Deep Learning.
The original goal of NN approach was to solve problems
in the same way that the human brain would. Over time,attention focused on matching specific mental abilities. In thisstudy, we propose a system of Feed Forward NN (FFNN)
having artificial neurotransmitter. We use characteristic ofacetylcholine among neurotransmitter. Acetylcholine has beenreported to be related to motivation, diseases and memory [4],[5]. We use this system. We decrease learning error rate. In thehuman brain, information processing in the opposite direction
has not been carried out. In NN and Deep Learning, backpropagation is the algorithm used in the mainstream. 1II. ACETYLCHOLINE AMONG N EUROTANSMITTER
Neurotransmitter, also known as chemical messenger, isendogenous chemicals that enable neurotransmission. Theytransmit signals across a chemical synapse, such as a neuromuscular junction, from one neuron to another neuron.
Neurotransmitter is released from synaptic vesicles in synapses
into the synaptic cleft, where they are received by receptors on
the target cells. Many neurotransmitters are synthesized from
simple and plentiful precursors such as amino acids, which
are readily available from the diet and only require a small
number of biosynthetic steps for conversion. Neurotransmitter
plays a major role in shaping everyday life and functions.
1 This research revised International Symposium on Circuits and Systems
2018 (ISCAS’18) [8].

- 34 -Their exact numbers are unknown, but more than 100 chemical
messengers have been uniquely identified. For example, there
are acetylcholine, adrenaline, glutamate, dopamine and so on.
In this study, we focused on acetylcholine among neurotransmitter. Acetylcholine was discovered by Henry Halletdale
in 1914 and revealed to be a neurotransmitter by Otto Levy.
It is early detection among neurotransmitters. It has manyfeatures. First, it has a relationship with illness. It is known
that too much acetylcholine secretion leads to Parkinson’sdisease. It is also known that too little acetylcholine secretion
leads to Alzheimer’s disease. Second, it has relationship withmemory. In the human brain, there is a part that plays acentral role in memory formation called hippocampus. It is
studied as acetylcholine as a substance that promotes memoryof the hippocampus. Third, it is secretion amount changes
within 1 day. Secretion tends to decrease throughout the day.
However, you can restore the secretion of acetylcholine byhaving a meal. In our day’s life, my motivation for studying
and working may diminish at noon. This is attributed to the
reduced amount of acetylcholine’s secretion. However, if wehave lunch it increases motivation for studying and working.
This is due to the increased amount of acetylcholine secretedby having meals. However, because there are individual differences, specific amount of secretion is unknown.

Fig. 2. Schematic diagram of method using acetylcholine for input information of middle layer in FFNN.

III. P ROPOSED METHOD
We propose a system using artificial characteristics ofacetylcholine for the only middle layer of FFNN. This isbecause the middle layer plays a very large role in FFNN.
In this study, we propose a method adding acetylcholine.
The one is “Method Using Acetylcholine for Input Informationof Middle Layer”. As a characteristic of acetylcholine, it
excites and inhibits the next neuron during signal transduction.
This is modeled between input layer and middle layer to makeit the one.
Characteristics of acetylcholine are not fully understood yet.
We use two reported characteristics of acetylcholine in thismethod. First, the amount of acetylcholine secretion decreases
in a daily life. Second, the amount of secretion recovers byhaving three meals (breakfast, lunch and dinner). For using
these characteristics, we propose three patterns which artificial
characteristics of acetylcholine as shown in Fig. 3. We consider
the number of trials (= 50000) to be one day. So we decideto recover the secretion amount three times within learning
loops. The range of acetylcholine is changed from -0.1 to 0.1
in all patterns.
Pattern 1 shows that the amount of secretion decreases
linearly at each learning loop. Pattern 2 shows that the amountof secretion decreases curvilinearly at each learning loop.
The angle of the tangent becomes sharply. Pattern 3 also
shows that the amount of secretion decreases curvilinearly ateach learning loop. The angle of the tangent becomes gentle.
Patterns 2 and 3 are represented by quadratic functions.
Equation (1) shows the propagating equation of the conventional neuron system at input layer to hidden layer. It does nothave the acetylcholine’s system.

(a) pattern 1.

(b) pattern 2.

(c) pattern 3.
Fig. 3.

Three patterns which artificial characteristics of acetylcholine.

hij (t + 1) = f{∑}
wij (t)x(t) − θ(t)(1)

x is input data. w is connection weight of neurons. θ isthreshold. h is output value.
Equation (2) shows the propagating equation of the neuron
having the acetylcholine’s system at input layer to hidden
layer.

hij (t + 1) = f{∑}
wij (t)(x(t) + χ(t)) − θ(t)(2)

χ(t) denotes the acetylcholine’s value. It also includes thecharacteristics of Fig. 3.

- 35 -And Eqs. (1) and (2) using sigmoid function Eq. (3) foractivation function [6].
1(3)f (a) =
1 + e−aIV. S IMULATION RESULT
We compared with the minimum value in the average oflearning error rate of 10 consecutive times. Learning error rateis determined by using root mean squared error. Root meansquare error is expressed as Eq. (4).
vu nu1 ∑(4)A(y) = t(ylearn − y)2
n i=1
ylearn is derived based on learning. y is the answer we haveprepared. The weight correction constant is set to 0.1 in allsystems.
We verify the proposed method using three data sets. First,
iris data set [7]. It has 150 data and uses the network of
the number of neurons in the input layer is 4, the numberof neurons in the middle layer is 10, the number of neurons
in the output layer is 3. Second, we use a data set with 1728
data. It is one that evaluates six to four features of cars (cardata set) [7]. The network of the number of neurons in theinput layer is 6, the number of neurons in the middle layer is10, the number of neurons in the output layer is 4. Third, we
use a data set with 201 data. It lets you learn the point of [-1,1] of the function of y = x2 .
The results when using three data sets are shown in Tables I,II and III below. In Tables I, II and III, “Conventional” shows theconventional system that does not have acetylcholine’s system.
Patterns 1, 2 and 3 shows proposed system that have featuresof acetylcholine. These features are shown in Fig. 3.
TABLE IL EARNING ERROR RATE OF USING IRIS DATA SET.
SystemConventional
Pattern 1
Pattern 2
Pattern 3

Learning error rate
0.050110.04246
0.044960.04659

In Table I, all proposed patterns obtain higher learning
accuracy than the conventional system. In pattern 1, learning
accuracy is improved by about 18%.

conventional system, so the proposed method of this time
seems to have failed.
TABLE IIILEARNING ERROR RATE OF USING y = x2 DATA SET.

SystemConventional
Pattern 1
Pattern 2
Pattern 3

In Table III, patterns 2 and 3 were able to reduce learningerror rate compared with the conventional system. In pattern
3, learning error rate is reduced by about 7%.
V. C ONCLUSION
A new method was proposed for FFNN using the propertyof acetylcholine, which is a neurotransmitter. Then, we used
the amount of acetylcholine secretion decreases in a daily
life and the amount of secretion recovers by having meals(breakfast, lunch and dinner). For using these characteristics,
we made three patterns. As the results of verification in threedata sets, it was possible to reduce the error rate as comparedwith the conventional system. However, it was difficult to saythat it was suitable for all patterns. It will be necessary toverify using more data sets and would like to challenge more
realistic problems in the future.
R EFERENCES[1] Hinton G, Vinyals O and Dean J, “Distilling knowledge in a neuralnetwork,” In Deep Learning and Representation Learning Workshop,NIPS, 2014.
[2] Gurney and Kevin, “An introduction to neural networks,” UCL Press,
ISBN 1857286731, OCLC 37875698, 1997.
[3] Chihiro IKUTA, Yoko UWATE, Yoshifumi NISHIO and Guoan YANG,”Multi-Layer Perceptron with Pulse Glial Chain,” IEICE Transactions
on Fundamentals, vol. E99-A, no. 3, pp. 742-755, Mar. 2016.
[4] Lombardo S and Maskos U, “Role of the nicotinic acetylcholine receptor
in Alzheimer’s disease pathology and treatment,” Neuropharmacology
96(ptB), 255-62, 2015.
[5] Dai Mitsushima, Akane Sano and Takuya Takahashi, “A cholinergic
trigger drives learning-induced plasticity at hippocampal synapses,
Nature Communications 4, 2760, 2013.
[6] Cybenko adn G.V, “Approximation by Superpositions of a Sigmoidalfunction,” In van Schuppen, Jan H. Mathematics of Control, Signals,and Systems, Springer International, pp. 303314, 2006.
[7] http://archive.ics.uci.edu/ml/index.php, “Center for Machine Learning
and Intelligent Systems”[8] Kazuki Nagao, Yoko Uwate and Yoshifumi Nishio, “Back Propagation
Learning with Acetylcholine in Feed Forward Neural Network,” IEEE
International Symposium on Circuits and Systems (ISCAS), 2018.
(being submitted)

TABLE IILEARNING ERROR RATE OF USING CAR DATA SET.

SystemConventional
Pattern 1
Pattern 2
Pattern 3

Learning error rate
0.00559
0.005620.005310.00520

Learning error rate
0.00001
0.00001
0.000020.00002

In Table II, pattern 1 has the same learning accuracy as
the conventional system, and patterns 2 and 3 have poor error
rate. As a cause of this, because the number of data sets islarge. Very high learning accuracy is obtained also in the- 36 -