# An Extension Network of Dendritic Neurons

Authors: Qianyi Peng; Shangce Gao; Yirui Wang; Junyan Yi; Gang Yang; Yuki Todo
Journal: Computational Intelligence and Neuroscience
 (2023)

---

## Abstract

Deep learning (DL) has achieved breakthrough successes in various tasks, owing to its layer-by-layer information processing and sufficient model complexity. However, DL suffers from the issues of both redundant model complexity and low interpretability, which are mainly because of its oversimplified basic McCulloch–Pitts neuron unit. A widely recognized biologically plausible dendritic neuron model (DNM) has demonstrated its effectiveness in alleviating the aforementioned issues, but it can only solve binary classification tasks, which significantly limits its applicability. In this study, a novel extended network based on the dendritic structure is innovatively proposed, thereby enabling it to solve multiple-class classification problems. Also, for the first time, an efficient error-back-propagation learning algorithm is derived. In the extensive experimental results, the effectiveness and superiority of the proposed method in comparison with other nine state-of-the-art classifiers on ten datasets are demonstrated, including a real-world quality of web service application. The experimental results suggest that the proposed learning algorithm is competent and reliable in terms of classification performance and stability and has a notable advantage in small-scale disequilibrium data. Additionally, aspects of network structure constrained by scale are examined.


---

## Body

## 1. Introduction

In recent years, deep learning (DL) has dominated the research field of artificial intelligence (AI) and achieved dramatic successes in terms of speech recognition, protein structure prediction, drug discovery, image and video processing, and others. [1]. At present, the mainstream deep learning models are mostly constructed based upon neural networks, referring to multiple-layered parameterized McCulloch–Pitts neurons inspired by the biological neuron [2]. Neural networks as black boxes are not only extensively studied in the field of artificial intelligence but also highly applied to the industry of information technology [3, 4]. The appearance of deep neural networks pushes the development of neural networks to a new peak. However, given the numerous difficulties and problems they face, including the lack of a theoretical foundation for explanation [5, 6], fairness [7, 8], and causal discovery [9, 10], neural networks tend to become stuck in a relatively inert state. In pace with the increasing attention of interpretative theory, it is essential to urgently require newer and better discoveries and more valuable research orientations to avoid blindness for the promotion of scientific and technological progress.From the perspective of understanding the mechanics of artificial neural networks, various methods and contributions are introduced [11, 12]. Through the application of the Monte Carlo simulation for quantifying the variable’s importance, Olden et al. justified the rightness of the connection weight calculation method in neural networks [13]. Statistics pointed out that Sarle et al. proclaimed the relations between neural networks and the generalized linear model, maximum redundancy analysis, projection tracking, clustering analysis, and other statistical models and also transformed terms in neural networks into statistical terms [14]. Similarly, certain relations between artificial neural networks and statistical methods were proposed in [15].Despite putting forward the abovementioned studies, the bottleneck of neural networks is not addressed. Originating from the simulation of neurons in biological concepts, the artificial neural network takes artificial neurons as nodes to construct a complete conduction structure. As a basic information processing unit, the neuron is formed by a dendrite, cell body (soma), axon, and synapse, as shown in Figure1(a). To be specific, the dendrite receives information from the outside world, the cell body processes the information, and the axon and synapse transmit the signal and pass it on to other neurons, respectively [16]. The structure of biological neurons can be traced back to the 1940s, when McCulloch and Pitts jointly published the abstract neuron model McCulloch–Pitts (MP) [17] for the first time, as illustrated in Figure 1(b). Then, in 1949, the Hebb rule [18] was proposed based on the theory of the variability of synapse connections of neurons within the human brain. The adjusting weight method was introduced into machine learning, thereby laying the foundation for the learning algorithms.Figure 1
(a) The function of a biological neuron is completely different depending on the shape of its dendrites and the location of its synapses. (b) McCulloch–Pitts neuron model: no interaction in dendrite morphology and dendrites. (c) Single dendritic neural model: faithful representation of dendrite morphology and dendrites fixated on binary classification.
(a)(b)(c)Inspired by the structure of biological neurons and MP, Todo et al. proposed the dendritic neural model (DNM) [19]. Different from the traditional neural network model, the dendritic neural model is designed based on neuron conduction and single neuron. DNM obtains the support of the biological theory to simulate the biological neuron. Furthermore, DNM compensates for some defects of the homologous perceptron model, such as the inability to solve the XOR problem. At the same time, the novel study of the human brain [20] is also brought about.As a classifier, shown in Figure1(c), DNM has been applied to various classification problems. For example, Sha et al. classified the breast cancer dataset, and Jiang et al. detected the liver disorder [21] for assisted disease diagnosis. Apart from the development of medical aid applications, an unconventional method was also applied to the financial field. To improve the classification performance, metaheuristic algorithms were introduced to train the hyperparameters of DNM [22, 23]. Through the use of the decision tree, Luo et al. initialized the model to realize better effectiveness [24]. For solving a generalized large-scale classification problem, Jia et al. suggested a reconciliation method with DNM by using a particle antagonism mechanism, and Ji et al. proposed a DNM-based multiobjective evolutionary algorithm [25]. In terms of feature selection, Song et al. addressed the high-dimensional challenge [26], and Gao et al. also showed the expansibility and flexibility of DNM for diverse applications [27]. Utilizing the multiplication operation that is useful to the information processing for a single neuron, the computing in synapses is imaginatively described using sigmoid functions. It is advantageous to establish the morphology of a neuron by determining the values of the parameters in synapses since the output of synapses can effectively represent signals. Nevertheless, it is noted that the single neuron is limited in partial application scenarios. In [28], the binary classification results of DNM were incorporated to undertake multiple classification tasks and thus recognize the multiclassification datasets.By adopting the quality of service (QoS) as the evaluation dataset, this study implements the multiclassification of web service selection. QoS is defined as the fact that a network utilizes a range of basic technologies to provide superior service capabilities for the designated network communication [29]. As a security mechanism of the network and a technology, QoS is carried out to deal with network delay [30], blocking [31], and other problems. For a general situation, the common network bandwidth as a significant metric is instanced in order to illustrate QoS. When the standard of service quality has not appeared, the network environment treats all services and applications in an equal way, resulting in a disordered situation, as shown in Figure 2(a) where the colored area stands for different web services and applications. In other words, when a network device does not have the capacity of QoS, the network environment will be threatened, and a bottleneck will be created [32]. As shown in Figure 2(b), prioritization from the perspective of QoS provides a more orderly, efficient, and stable network environment. QoS contains a set of nonfunctional attributes, which is the measure and criteria of such characteristics of the web services, such as reliability and response time, to effectively classify and sort different services.Figure 2
(a) The network is disordered without QoS rules. (b) The network is in order with QoS rules.
(a)(b)Web services refer to some software modules running on the network, which are service-oriented and based on distributed programs. Due to the fact that the web service employs general Internet standards, such as HTTP and XML (a subset of the standard generalized markup language) [33], human beings then have access to data on the web via various terminal devices in various places. In this article, the described web service is different from the common network application. It generally refers to some application modules, such as the network protocol and method, which is the basis of network applications. With the development of the Internet, many candidate services have implemented the same task, and most of them have the same functions but different nonfunctional characteristics. As a result, these services are divided into different service quality levels. Overlapping is seen to be inevitable because of the existence of a wide range of web services on the network. Based on the QoS, web service selection is considered an effective solution [34]. As network technology and operation concepts develop rapidly, web services are becoming the latest technology and development trend for constructing distributed, modular, and service-oriented applications.Based on DNM, this article proposes a multiple dendritic neural network (MDNN) with multiple single neurons to achieve the multiclassification of web service selection based on QoS. To adjust the multiclassification mechanism, the structure of DNM introduced in Figure3 is reconstructed. For the purpose of accelerating the gradient descent and improving the multiclassification accuracy, the backpropagating algorithm and adaptive moment estimation optimization are derived for the first time. Experiments are carried out on the Quality of Web Service dataset and nine UCI multiclassification datasets [35]. In the comparison between MDNN and nine state-of-the-art classifiers, the superiority of the proposed method is demonstrated.Figure 3
Multiple dendritic neural networks: applied on comprehensive applications based on the dendritic structure.The contributions are majorly classified into the aspects as follows: 1) a novel multiple single-neuron neural network for multiclassification tasks is developed. 2) The potential and application scenarios of the dendritic neural network are explored. 3) A new approach for QoS-driven web service selection is proposed.Given as follows is the organization of the remaining parts of this article: Section2 presents the structure of the multiple dendritic neural networks. Section 3 elaborates on the learning processes of the proposed method and expounds on the optimization strategies. The comparison with other algorithms and experimental results are shown in Section 4. At last, Section 5 concludes the paper and formulates future work.


## 2. The Dendritic Neuron Network-Based Multiclassifier Approach

The proposed multiclassifier is constructed by multiple single neurons. The general architecture is shown in Figure4. As for each neuron, xi, the input of the model is preprocessed by using a nonlinear sigmoid filter. To differentiate neurons, the function introduces the subscript j, which is defined as follows:(1)Yj,i,m=11+e−wj,i,mxi−qj,i,m,where i is the number of attributes of the sample, m is the number of nodes within the hidden layer, and j is the number of classifications of output results. In addition, the weight wj,i,m and threshold qj,i,m denote the neural network parameters in the training stage and are randomly initialized within (0, 0.01) and 0, respectively.Figure 4
The structure of the proposed MDNN. The framed rectangle represents the structure of a single neuron.In contrast to the perceptron model, a quadrature method is adopted for the hidden layer to not only rule out the inhibited neuronal excitation but also enhance the activated neuronal excitation. The formula is described as follows:(2)Zj,m=∏i=1Yj,i,m,(3)Vj=∑m=1Zj,m.Eq. (2) means that all of the hidden layers are activated, which is equivalent to a logical AND. Eq. (3) is equivalent to a logical OR where all inhibited neuronal excitations from the former layer are suppressed exclusively and the rest are reactivated. As a result, the multiclassification structure of multiple neurons is formed.Apart from the dendritic mechanism, MDNN utilizes the normalized exponential function to output final results. For ease of consistency in representation, the illustrated style of the normalized exponential function is followed. To be noted, the output of multiple neurons is processed by all information from the previous layer instead of being directly conveyed, expressed as follows:(4)Oj=eVj∑j=1eVj,where Oj is the possibility of the prediction for each class. The normalized exponential function converts the output value of the upper layer, Vj, to the probability distribution with the range of [0, 1], and the sum of the probability values of each neuron being 1. The formula first converts the results Vj into an exponential function, ensuring the nonnegative probability, and then normalizes the probability values into 1.Since the prediction result follows the rules of the probability distribution, the cross-entropy function as the loss function is considered a proper substitute for mean square error, which is defined as follows:(5)Ej=−∑j=1Tj∗logOj,where Ej represents the similarity of probability distribution between the prediction of the model and the actual classification and Tj is the actual classification label.


## 3. Learning Mechanism and Optimization Strategies

The existing learning algorithms cannot be directly applied since MDNN is a new dendritic neuron model containing multiplication operators in its calculation. Accordingly, in this section, we for the first time derive the learning algorithms for our proposed MDNN, specifically one is the traditional error backpropagation, and the other is an Adam-like learning algorithm.


### 3.1. Backpropagation

In the course of learning samples, the model is promoted by the stochastic gradient descent of parameterswj,i,m and qj,i,m, which is described as follows:(6)wj,i,mt=wj,i,mt−1−η∆wj,i,m,qj,i,mt=qj,i,mt−1−η∆qj,i,m,where η as the learning rate is a positive constant. t and t−1 denote the current iteration and the previous iteration in the training stage, respectively.The error of the proposed MDNN is calculated by the cross-entropy function. According to the calculated error, the error backpropagation algorithm is introduced as the learning scheme. In backpropagation, all the samples or a batch of samples are involved. To better realize intuition, the relation among layers is shown in Figure5.Figure 5
The relation among layers.∆wj,i,m and ∆qj,i,m are expressed by the partial differential form as follows:(7)∆wj,i,m=∂Ej∂Oj∂Oj∂Vj∂Vj∂Zj,m∂Zj,m∂Yj,i,m∂Yj,i,m∂wj,i,m,∆qj,i,m=∂Ej∂Oj∂Oj∂Vj∂Vj∂Zj,m∂Zj,m∂Yj,i,m∂Yj,i,m∂qj,i,m.Since the model is trained by batches,∆wj,i,m and ∆qj,i,m obtained by the gradient descent are finally calculated as follows:(8)∆wj,i,m¯=∆wj,i,mN,∆qj,i,m¯=∆qj,i,mN,where N denotes the size of input data within the current iteration.Following the chain rule, the derivation procedures and results are presented according to the backpropagation. Firstly, the partial differential of errorE is calculated. By the empirical evidence of normalized exponential function, ∂Ej/∂Oj and ∂Oj/∂Vj are computed collectively instead of computing them separately.The forward propagation for the multiclassification is not directly corresponding. Thus, the derivation of errorE is discussed in Cases (1) and (2). To avoid confusion, the subscripts of V and O are redefined as m and n, respectively. Their relation is simplified as follows:(9)Vm⟵On⟵En.On the basis of Equations. (5) and (4), when n=m, there is Case (1):(10)∂En∂Vm=∂En∂On∂On∂Vm,=−Tn1OnOn1−On,=−Tn1−On.Whenn≠m, there is Case (2):(11)∂En∂Vm=−∑n≠mTm1Om−OnOm=∑n≠mTmOn.We incorporate Cases (1) and (2) into the following formula:(12)∂En∂Vm=−Tn1−On+∑n≠mTmOn,=−Tn+TnOn+∑n≠mTmOn,=OnTn+∑n≠mTm−Tn,=On−Tn.Thus,∂Ej/∂Vj is expressed as follows:(13)∂Ej∂Vj=∂Ej∂Oj∂Oj∂Vj=Oj−Tj.For the rest layers of MDNN, they are derived according to Equations. (3) and (2) as follows:(14)∂Vj∂Zj,m=1,(15)∂Zj,m∂Yj,i,m=Zj,mYj,i,m.Taking the derivative of Equation. (1) with the sigmoid function, ∂Yj,i,m/∂wj,i,m and ∂Yj,i,m/∂qj,i,m are obtained as follows:(16)∂Yj,i,m∂wj,i,m=xi⋅Yj,i,m1−Yj,i,m,∂Yj,i,m∂qj,i,m=−Yj,i,m1−Yj,i,m.


### 3.2. Adam-Like Optimization

For improving the convergence and classification ability of the proposed model, inspired by the well-known adaptive moment estimation (Adam) [36], an Adam-like learning algorithm for MDNN is also introduced to accelerate the gradient descent without diverging. The way of updating weights in each iteration is optional. The traditional way mentioned in Section 3.1 or Adam can be altered according to the user’s setting.As an extended optimization strategy of stochastic gradient descent (SGD) [37], momentum [38, 39] is introduced to reduce the oscillation and accelerate the gradient descent. The fundamental concept of gradient descent with momentum lies in updating the weight by calculating the exponentially weighted average of the gradient as follows:(17)v∆wj,i,m=αv∆wj,i,m+1−α∆wj,i,m,(18)v∆qj,i,m=αv∆qj,i,m+1−α∆qj,i,m,where α is a positive constant to smooth out the gradient descent process. Intuitively, ∆wj,i,m and ∆qj,i,m are interpreted as the acceleration in physics. v∆wj,i,m and v∆qj,i,m are regarded as the velocity, and α is seen as the friction. In addition, ∆wj,i,m and ∆qj,i,m accelerate the gradient descent and gain the velocity ∆wj,i,m and ∆qj,i,m, and the friction α prevents the acceleration.In this case, the updates of parameterswj,i,m and qj,i,m are modified as follows:(19)wj,i,mt=wj,i,mt−1−ηv∆wj,i,m,(20)qj,i,mt=qj,i,mt−1−ηv∆qj,i,m.Serving as a crucial part of Adam, the root mean square prop (RMSprop) [40] auxiliary accelerates the gradient descent as follows:(21)u∆wj,i,m=βv∆wj,i,m+1−β∆wj,i,m2,where β is a positive constant similar to α. wj,i,m is calculated as follows:(22)wj,i,mt=wj,i,mt−1−η∆wj,i,mv∆wj,i,m.Similarly,u∆qj,i,m is obtained by(23)u∆qj,i,m=βv∆qj,i,m+1−β∆qj,i,m2,(24)qj,i,mt=qj,i,mt−1−η∆qj,i,mv∆qj,i,m.In order to avoid the bias of exponentially weighted average in the initial learning stage, Equations. (17), (18), (21), and (23) are modified to obtain more accurate results as follows:(25)vcorr∆wj,i,m=v∆wj,i,m1−αt,vcorr∆qj,i,m=v∆qj,i,m1−αt,ucorr∆wj,i,m=u∆wj,i,m1−βt,ucorr∆qj,i,m=u∆qj,i,m1−βt.For the acceleration of the gradient descent, Adam combines RMSprop with momentum. Thus, based on Equations. (23) and (24), the parameter updating equations optimized by Adam are expressed as follows:(26)wj,i,mt=wj,i,mt−1−ηvcorr∆wj,i,mucorr∆wj,i,m+ε,qj,i,mt=qj,i,mt−1−ηvcorr∆qj,i,mucorr∆qj,i,m+ε,where ε is an infinitesimal so as to prevent the computation overflow.


## 3.1. Backpropagation

In the course of learning samples, the model is promoted by the stochastic gradient descent of parameterswj,i,m and qj,i,m, which is described as follows:(6)wj,i,mt=wj,i,mt−1−η∆wj,i,m,qj,i,mt=qj,i,mt−1−η∆qj,i,m,where η as the learning rate is a positive constant. t and t−1 denote the current iteration and the previous iteration in the training stage, respectively.The error of the proposed MDNN is calculated by the cross-entropy function. According to the calculated error, the error backpropagation algorithm is introduced as the learning scheme. In backpropagation, all the samples or a batch of samples are involved. To better realize intuition, the relation among layers is shown in Figure5.Figure 5
The relation among layers.∆wj,i,m and ∆qj,i,m are expressed by the partial differential form as follows:(7)∆wj,i,m=∂Ej∂Oj∂Oj∂Vj∂Vj∂Zj,m∂Zj,m∂Yj,i,m∂Yj,i,m∂wj,i,m,∆qj,i,m=∂Ej∂Oj∂Oj∂Vj∂Vj∂Zj,m∂Zj,m∂Yj,i,m∂Yj,i,m∂qj,i,m.Since the model is trained by batches,∆wj,i,m and ∆qj,i,m obtained by the gradient descent are finally calculated as follows:(8)∆wj,i,m¯=∆wj,i,mN,∆qj,i,m¯=∆qj,i,mN,where N denotes the size of input data within the current iteration.Following the chain rule, the derivation procedures and results are presented according to the backpropagation. Firstly, the partial differential of errorE is calculated. By the empirical evidence of normalized exponential function, ∂Ej/∂Oj and ∂Oj/∂Vj are computed collectively instead of computing them separately.The forward propagation for the multiclassification is not directly corresponding. Thus, the derivation of errorE is discussed in Cases (1) and (2). To avoid confusion, the subscripts of V and O are redefined as m and n, respectively. Their relation is simplified as follows:(9)Vm⟵On⟵En.On the basis of Equations. (5) and (4), when n=m, there is Case (1):(10)∂En∂Vm=∂En∂On∂On∂Vm,=−Tn1OnOn1−On,=−Tn1−On.Whenn≠m, there is Case (2):(11)∂En∂Vm=−∑n≠mTm1Om−OnOm=∑n≠mTmOn.We incorporate Cases (1) and (2) into the following formula:(12)∂En∂Vm=−Tn1−On+∑n≠mTmOn,=−Tn+TnOn+∑n≠mTmOn,=OnTn+∑n≠mTm−Tn,=On−Tn.Thus,∂Ej/∂Vj is expressed as follows:(13)∂Ej∂Vj=∂Ej∂Oj∂Oj∂Vj=Oj−Tj.For the rest layers of MDNN, they are derived according to Equations. (3) and (2) as follows:(14)∂Vj∂Zj,m=1,(15)∂Zj,m∂Yj,i,m=Zj,mYj,i,m.Taking the derivative of Equation. (1) with the sigmoid function, ∂Yj,i,m/∂wj,i,m and ∂Yj,i,m/∂qj,i,m are obtained as follows:(16)∂Yj,i,m∂wj,i,m=xi⋅Yj,i,m1−Yj,i,m,∂Yj,i,m∂qj,i,m=−Yj,i,m1−Yj,i,m.


## 3.2. Adam-Like Optimization

For improving the convergence and classification ability of the proposed model, inspired by the well-known adaptive moment estimation (Adam) [36], an Adam-like learning algorithm for MDNN is also introduced to accelerate the gradient descent without diverging. The way of updating weights in each iteration is optional. The traditional way mentioned in Section 3.1 or Adam can be altered according to the user’s setting.As an extended optimization strategy of stochastic gradient descent (SGD) [37], momentum [38, 39] is introduced to reduce the oscillation and accelerate the gradient descent. The fundamental concept of gradient descent with momentum lies in updating the weight by calculating the exponentially weighted average of the gradient as follows:(17)v∆wj,i,m=αv∆wj,i,m+1−α∆wj,i,m,(18)v∆qj,i,m=αv∆qj,i,m+1−α∆qj,i,m,where α is a positive constant to smooth out the gradient descent process. Intuitively, ∆wj,i,m and ∆qj,i,m are interpreted as the acceleration in physics. v∆wj,i,m and v∆qj,i,m are regarded as the velocity, and α is seen as the friction. In addition, ∆wj,i,m and ∆qj,i,m accelerate the gradient descent and gain the velocity ∆wj,i,m and ∆qj,i,m, and the friction α prevents the acceleration.In this case, the updates of parameterswj,i,m and qj,i,m are modified as follows:(19)wj,i,mt=wj,i,mt−1−ηv∆wj,i,m,(20)qj,i,mt=qj,i,mt−1−ηv∆qj,i,m.Serving as a crucial part of Adam, the root mean square prop (RMSprop) [40] auxiliary accelerates the gradient descent as follows:(21)u∆wj,i,m=βv∆wj,i,m+1−β∆wj,i,m2,where β is a positive constant similar to α. wj,i,m is calculated as follows:(22)wj,i,mt=wj,i,mt−1−η∆wj,i,mv∆wj,i,m.Similarly,u∆qj,i,m is obtained by(23)u∆qj,i,m=βv∆qj,i,m+1−β∆qj,i,m2,(24)qj,i,mt=qj,i,mt−1−η∆qj,i,mv∆qj,i,m.In order to avoid the bias of exponentially weighted average in the initial learning stage, Equations. (17), (18), (21), and (23) are modified to obtain more accurate results as follows:(25)vcorr∆wj,i,m=v∆wj,i,m1−αt,vcorr∆qj,i,m=v∆qj,i,m1−αt,ucorr∆wj,i,m=u∆wj,i,m1−βt,ucorr∆qj,i,m=u∆qj,i,m1−βt.For the acceleration of the gradient descent, Adam combines RMSprop with momentum. Thus, based on Equations. (23) and (24), the parameter updating equations optimized by Adam are expressed as follows:(26)wj,i,mt=wj,i,mt−1−ηvcorr∆wj,i,mucorr∆wj,i,m+ε,qj,i,mt=qj,i,mt−1−ηvcorr∆qj,i,mucorr∆qj,i,m+ε,where ε is an infinitesimal so as to prevent the computation overflow.


## 4. Experimental Evaluation

### 4.1. Experimental Setup

The quality of web service (QWS) dataset [41, 42] is a real-world dataset based on the quality of service. Several versions of QWS are available. In this study, experiments use the original version, which consists of 364 web services. Its quality is described by a total of 10 nonfunctional attribute indexes. The QWS dataset divides the web service into 4 levels from the highest to the lowest, which are platinum, gold, silver, and bronze.To avoid overfitting while improving the accuracy of results, data preprocessing strategies are adopted in the experiments. The raw data are normalized by using the rule of standardization:(27)x=x−x¯σ.Moreover, the normalized data are randomly divided into three parts: 70 percent for the training process, 15 percent for the testing process, and the remaining data for the validation, to reduce unnecessary time consumption.


### 4.2. Optimal Parameter Settings

All of the hyperparameters are illustrated in Table1, which presents their description and values. Sigmoid, tanh, Rectified Linear Unit (ReLU), and Leaky ReLU are available to freely choose the suitable active function. For large datasets, the samples are divided into mini-batch and shuffled for the gradient descent during the training stage. If batchsize is equal to the number of samples during the iteration, then the batch gradient descent will be executed.Table 1
Hyperparameters in the experiments.
IDParametersDescriptionDefault1activationActivation functionSigmoid2kParameter for Leaky ReLU0.013timeOperation times304ηLearning rate0.015MNumber of nodes for the hidden layer56epochTraining epoch1007batchsizeMini-batch size for training1008precisionEnd the training if reaching the precision0.19AdamActive or inactive the optimization110αParameter for Adam0.911βParameter for Adam0.99912epsilonParameter for Adam1e-8However, it is tricky to determine the epoch size. To enable the model to reach optimal performance, a self-adaptive appending training epoch is arranged in the training stage according to the convergence of the validation process. Beginning with the default configuration, theepoch then adaptively increases pivoting on whether the gradient descent is approaching stagnation. At the same time, the initial value is set to 100 to avoid a higher epoch causing the time consumption. For general neural networks, experimental results are highly affected by the combination of parameters. Therefore, parameters, which are batchsize, M, η, and precision, are adjusted by the orthogonal experiment with 4 factors and 3 levels. The specific design is listed in Table 2. The optimal parameters of MDNN for QWS are finally shown in Table 3. The other parameters comply with the setting in Table 1.Table 2
Orthogonal experimental design of hyperparameters withL934 for QWS.
No.batchsizeMprecisionη112040.010.005212050.020.008312060.050.01424040.020.01524050.050.005624060.010.008736440.050.008836450.010.01936460.020.005Table 3
Optimal parameter settings for QWS.
IDParametersOptimalAdaptive/Not adaptive1batchsize364Not adaptive2precision0.01Not adaptive3M5Not adaptive4η0.01Not adaptive5epoch100Adaptive


### 4.3. Performance and Discussion

This section is divided into two subsections: experimental results of QWS analyzed by a variety of evaluation indicators are elaborated in Section 4.3.1, and Section 4.3.2 compares the proposed model with other multiclassification methods to demonstrate the prominent superiority of the proposed model.


#### 4.3.1. Experimental Results

For the comprehensive performance evaluation of MDNN, the following statistical indicators are used: precision, recall,F1 score, accuracy, and area under the curve (AUC) [43]. It is worth noting that the precision refers to the classification precision. For not only the convenience of the intuitive evaluation but also the justification of the following comparison, the macroaverage, which is the arithmetic average of performance indicators of all categories instead of instances, is adopted to statistically process classification results.Table4 shows the classification results of MDNN on the 4-class QWS dataset. The average values and optimum values represent the classification performance of MDNN. Although the dataset is technically unbalanced, such as the number of classes of QWS in Table 5, the stability and generalization ability of MDNN are considered to be effectively validated.Table 4
Average and optimum values of classification results of MDNN for QWS.
PlatinumGoldSilverBronzeMean valuesPrecision(Average)99.4498.4898.9299.5599.10(Optimum)100.00100.00100.00100.00100.00Recall(Average)96.5299.7299.4698.8198.63(Optimum)100.00100.00100.00100.00100.00F1(Average)97.7399.0599.1699.1598.77(Optimum)100.00100.00100.00100.00100.00Accuracy(Average)99.5899.5299.4599.5299.52(Optimum)100.00100.00100.00100.00100.00AUC(Average)99.1199.7199.9299.9799.68(Optimum)100.00100.0099.85100.0099.96Table 5
Description of datasets and parameter settings of MDNN.
DatasetsInstancesNumber of featuresNumber of classesSize of classesMηIris1504350, 50, 5040.01Wine17813359, 71, 4850.01Vehicle846184199, 217, 218, 212100.02Balance scale6254349, 288, 288120.01CMC147393629, 333, 511200.03Seed2107370, 70, 7050.01Vowel8713672, 89, 172, 151, 207, 18060.01Thyroid21553150, 35, 3040.01Robot navigation545624482, 620, 972, 205, 329300.02QWS36410441, 100, 120, 10350.01The five statistical indicators suggest that the classification achieved by MDNN for each class is effective, stable, and reliable. In their mean values, it is indicated that MDNN has good classification performance. The gradient descent optimization strategy effectively reduces the errors. Moreover, MDNN accelerates the gradient descent to maintain a continuous downward trend, thereby finally guaranteeing the generalization and robustness of MDNN.


#### 4.3.2. Comparison of Methods

To further verify the efficiency of MDNN, nine classifiers in total are used to compare with MDNN on nine multiclassification datasets from the UCI machine learning repository and the QWS dataset. The information of datasets and parameter settings of MDNN are listed in Table5. The nine classifiers consist of BP, SVM, KNN, CART, naïve Bayes, LDA, QDA, J48, and random forest [44]. In addition, the ten datasets include Iris, Wine, Vehicle, Balance scale, CMC, Seed, Vowel, Thyroid, Robot navigation, and QWS.Experimental results are shown in Table6 where the best result for each dataset among all compared methods is highlighted in bold. According to five statistical indicators, it can be found that MDNN has the most optimal values in comparison with other classifiers. On the Iris, Wine, Seed, and Thyroid datasets, MDNN gives the best performance, with perfect outcomes of 100 percent correctness. Also, MDNN performs well with unbalanced data such as Vowel, Thyroid, and QWS. As a result, MDNN’s classification performance is more constant than that of other methods. Nevertheless, MDNN appears to have a minor disadvantage on the large datasets, such as Balance scale, Vehicle, CMC, and Robot navigation, which seem to be constrained by the distribution of the network structure. In the comparison between MDNN and other classifiers, the superiority and effectiveness of the multiple dendritic neuron structure are verified.Table 6
Comparison between MDNN and nine classifiers.
PrecisionRecallF1AccuracyAUCPrecisionRecallF1AccuracyAUCPrecisionRecallF1AccuracyAUCBPSVMKNNIris96.3096.1395.9197.3999.6896.4396.1196.0697.3999.7696.4696.5296.2997.6898.78Wine27.7443.3031.6565.6062.2943.6240.2531.3164.2872.2971.0271.0269.1080.7487.78Vehicle12.2628.4614.9163.5754.0312.1124.6510.2961.7738.1262.8664.2262.1982.1886.18Balance scale67.5067.8466.3793.3390.6565.7066.6065.3691.3594.5959.7056.5257.2483.4365.29CMC37.2342.8538.2665.6964.1554.4551.6451.6970.3772.0647.2246.0145.0666.0665.75Seed90.7990.4389.8993.5497.5690.3990.2489.9893.4798.8289.0588.7288.0292.4396.36Vowel3.8216.676.2174.3141.9983.7853.2157.5486.2180.7483.6283.2782.8894.8295.55Thyroid73.8471.6170.8690.5687.9768.7274.5767.0389.7996.4794.3684.2587.6995.2896.81Robot navigation73.7663.3965.3288.1891.3890.1887.5288.7394.7698.1785.0384.8684.7392.6895.79QWS17.6430.7119.4268.9451.3485.0384.8684.7392.6832.6962.1660.4959.3978.6181.92CARTNaive BayesLDAIris96.2696.1696.0297.2097.8295.9795.8795.5297.4999.5298.0798.1397.9498.6599.07Wine91.5492.5491.3694.5795.1596.3596.9596.3497.7099.8997.8998.2097.8998.6898.55Vehicle68.9169.1868.7384.4684.2651.6746.1141.6872.6677.4177.6378.5377.7189.2387.98Balance scale55.7557.2756.2585.7276.3560.9965.6262.8893.2289.6673.6880.3267.5882.6582.20CMC49.7949.2349.1567.6367.3749.4549.8847.5665.1267.5649.8850.5249.0666.4463.64Seed88.8188.5687.9992.2992.9891.4291.4191.1494.2498.6796.7796.7696.6597.9297.59Vowel96.7796.7696.6597.9297.5977.7277.5076.8993.2296.7575.3676.8575.0692.6284.00Thyroid87.4688.2286.3194.5893.0097.3495.7796.1598.2699.6595.8188.1590.5395.6993.08Robot navigation99.1799.2299.1999.6799.5954.9766.4454.2076.3683.6656.8066.7358.5380.8275.68QWS98.1898.7198.4099.7398.6481.6184.1281.7190.6495.7886.3586.3585.7793.4289.55QDAJ48Random ForestIris97.6097.6197.4498.3698.82100.0097.7898.8394.0798.89100.00100.00100.0094.81100.00Wine98.9598.6098.6799.1899.1892.5493.1992.6391.5794.6897.1598.1297.5696.2699.82Vehicle85.2085.4985.0292.5792.1986.6789.8988.1571.6094.0687.4795.2191.1074.8899.09Balance scale83.5791.0485.6294.4791.713.822.713.0377.9253.652.753.403.0280.3353.38CMC62.2860.5760.8674.9678.1261.3661.2561.2051.9469.3159.3264.5961.7750.6771.65Seed94.9494.4094.4896.3995.2286.9886.8386.5991.0689.5987.9190.3288.9292.4996.11Vowel75.4777.5075.5092.8084.4263.1757.6459.3083.5286.9464.8164.9964.2385.1189.00Thyroid94.2095.2994.1797.2997.1794.3995.1194.6792.5191.3195.4398.3796.8495.5098.61Robot navigation64.4275.5966.7383.4482.7498.9099.3799.1399.4099.6798.6199.4299.0199.4199.98QWS83.6585.1883.7091.5289.3197.99100.0098.9599.7699.8697.0496.7596.7898.9999.93MDNNIris100.00100.00100.00100.00100.00Wine100.00100.00100.00100.00100.00Vehicle85.3485.4885.3393.3196.12Balance scale80.5077.7678.9295.7493.26CMC62.2860.5760.8674.9678.12Seed100.00100.00100.00100.00100.00Vowel91.2984.2686.4795.6798.03Thyroid100.00100.00100.00100.00100.00Robot navigation94.8992.1193.4197.1999.30QWS99.1098.6398.7799.5299.68The receiver operating characteristic (ROC) curves of ten multiclassification methods show the correct classification coverage of each class of the QWS dataset in Figure6. It can be found that MDNN not only has a consistent performance on each class of QWS but also outperforms the other classifiers, thus indicating the effectiveness and stability of MDNN for the QWS classification and unbalanced multiclassification applications. Besides, experiments also demonstrate the efficiency and superiority of MDNN in terms of classification performance and stability.Figure 6
(a) The comparison of receiver operating characteristic curves for the platinum class of the QWS dataset. (b) The comparison of receiver operating characteristic curves for the gold class of the QWS dataset. (c) The comparison of receiver operating characteristic curves for the silver class of the QWS dataset. (d) The comparison of receiver operating characteristic curves for the bronze class of the QWS dataset.
(a)(b)(c)(d)


### 4.4. Morphology and Logical Circle Realization

For the display of a data sample, the shuffle operation set in the pretraining period is at disposal. According to the initialization ofwj,i,m and qj,i,m within Section 2, the synapses were calculated around 0.5 in the previous state. Through the training stage, the weights wj,i,m and qj,i,m were gradually stabilized. As shown in Figure 7, synaptic changes, thus, yield to accomplish the pruning of the redundant network structure.Figure 7
Dendritic changes in the structure of randomly selected samples before and after network training were applied to the QWS dataset.Z-axis represents the neuron, and X-axis and Y-axis are attributes and hidden layers, respectively.It can be easily observed that neuron 1, neuron 2, and neuron 4 are fully inhibited in accordance with the rule of Equation. (2) and Equation. (3). Consequently, the structure of neuron 1, neuron 3, and neuron 4 is ruled out. To specify the states of dendrites, a total of four scenarios are listed as follows:(28)dendriticstate=constant−1connection,whenqj,i,m<wj,i,m<0orqj,i,m<0<wj,i,m,constant−0connection,when0<wj,i,m<qj,i,morwj,i,m<0<qj,i,m,excitatoryconnection,when0<qj,i,m<wj,i,m,inhibitoryconnection.whenwj,i,m<qj,i,m<0.For the remainder neuron, the residual dendrite morphology is formed in Figure8(a), which places the line of dashes indicating pruning. As mentioned previously in Section 2, logic-based inherent relations have existed within dendritic structures. Finally, since constant-1 holds no substantive impact on the attributes, the connections among dendrites are equivalent to logic OR. Thus, the hardware realization is transformed as illustrated in Figure 8(b), where the multiplexer is a 1 : 2 numerical compactor. In addition to showing the extendibility of MDNN, the overhead also indicates that MDNN can avoid overfitting by increasing the dendritic matrix sparsity.Figure 8
(a) The dendritic states of MDNN are fixated on the QWS dataset. Circle 0 and Circle 1 stand for constant-0 and constant-1 connection, respectively. In addition, black-filled square and circle stand for inhibitory and excitatory states, respectively. (b) Logical circle realization of MDNN on the QWS dataset, whereλ = qj,i,m/wj,i,m.
(a)(b)


## 4.1. Experimental Setup

The quality of web service (QWS) dataset [41, 42] is a real-world dataset based on the quality of service. Several versions of QWS are available. In this study, experiments use the original version, which consists of 364 web services. Its quality is described by a total of 10 nonfunctional attribute indexes. The QWS dataset divides the web service into 4 levels from the highest to the lowest, which are platinum, gold, silver, and bronze.To avoid overfitting while improving the accuracy of results, data preprocessing strategies are adopted in the experiments. The raw data are normalized by using the rule of standardization:(27)x=x−x¯σ.Moreover, the normalized data are randomly divided into three parts: 70 percent for the training process, 15 percent for the testing process, and the remaining data for the validation, to reduce unnecessary time consumption.


## 4.2. Optimal Parameter Settings

All of the hyperparameters are illustrated in Table1, which presents their description and values. Sigmoid, tanh, Rectified Linear Unit (ReLU), and Leaky ReLU are available to freely choose the suitable active function. For large datasets, the samples are divided into mini-batch and shuffled for the gradient descent during the training stage. If batchsize is equal to the number of samples during the iteration, then the batch gradient descent will be executed.Table 1
Hyperparameters in the experiments.
IDParametersDescriptionDefault1activationActivation functionSigmoid2kParameter for Leaky ReLU0.013timeOperation times304ηLearning rate0.015MNumber of nodes for the hidden layer56epochTraining epoch1007batchsizeMini-batch size for training1008precisionEnd the training if reaching the precision0.19AdamActive or inactive the optimization110αParameter for Adam0.911βParameter for Adam0.99912epsilonParameter for Adam1e-8However, it is tricky to determine the epoch size. To enable the model to reach optimal performance, a self-adaptive appending training epoch is arranged in the training stage according to the convergence of the validation process. Beginning with the default configuration, theepoch then adaptively increases pivoting on whether the gradient descent is approaching stagnation. At the same time, the initial value is set to 100 to avoid a higher epoch causing the time consumption. For general neural networks, experimental results are highly affected by the combination of parameters. Therefore, parameters, which are batchsize, M, η, and precision, are adjusted by the orthogonal experiment with 4 factors and 3 levels. The specific design is listed in Table 2. The optimal parameters of MDNN for QWS are finally shown in Table 3. The other parameters comply with the setting in Table 1.Table 2
Orthogonal experimental design of hyperparameters withL934 for QWS.
No.batchsizeMprecisionη112040.010.005212050.020.008312060.050.01424040.020.01524050.050.005624060.010.008736440.050.008836450.010.01936460.020.005Table 3
Optimal parameter settings for QWS.
IDParametersOptimalAdaptive/Not adaptive1batchsize364Not adaptive2precision0.01Not adaptive3M5Not adaptive4η0.01Not adaptive5epoch100Adaptive


## 4.3. Performance and Discussion

This section is divided into two subsections: experimental results of QWS analyzed by a variety of evaluation indicators are elaborated in Section 4.3.1, and Section 4.3.2 compares the proposed model with other multiclassification methods to demonstrate the prominent superiority of the proposed model.


### 4.3.1. Experimental Results

For the comprehensive performance evaluation of MDNN, the following statistical indicators are used: precision, recall,F1 score, accuracy, and area under the curve (AUC) [43]. It is worth noting that the precision refers to the classification precision. For not only the convenience of the intuitive evaluation but also the justification of the following comparison, the macroaverage, which is the arithmetic average of performance indicators of all categories instead of instances, is adopted to statistically process classification results.Table4 shows the classification results of MDNN on the 4-class QWS dataset. The average values and optimum values represent the classification performance of MDNN. Although the dataset is technically unbalanced, such as the number of classes of QWS in Table 5, the stability and generalization ability of MDNN are considered to be effectively validated.Table 4
Average and optimum values of classification results of MDNN for QWS.
PlatinumGoldSilverBronzeMean valuesPrecision(Average)99.4498.4898.9299.5599.10(Optimum)100.00100.00100.00100.00100.00Recall(Average)96.5299.7299.4698.8198.63(Optimum)100.00100.00100.00100.00100.00F1(Average)97.7399.0599.1699.1598.77(Optimum)100.00100.00100.00100.00100.00Accuracy(Average)99.5899.5299.4599.5299.52(Optimum)100.00100.00100.00100.00100.00AUC(Average)99.1199.7199.9299.9799.68(Optimum)100.00100.0099.85100.0099.96Table 5
Description of datasets and parameter settings of MDNN.
DatasetsInstancesNumber of featuresNumber of classesSize of classesMηIris1504350, 50, 5040.01Wine17813359, 71, 4850.01Vehicle846184199, 217, 218, 212100.02Balance scale6254349, 288, 288120.01CMC147393629, 333, 511200.03Seed2107370, 70, 7050.01Vowel8713672, 89, 172, 151, 207, 18060.01Thyroid21553150, 35, 3040.01Robot navigation545624482, 620, 972, 205, 329300.02QWS36410441, 100, 120, 10350.01The five statistical indicators suggest that the classification achieved by MDNN for each class is effective, stable, and reliable. In their mean values, it is indicated that MDNN has good classification performance. The gradient descent optimization strategy effectively reduces the errors. Moreover, MDNN accelerates the gradient descent to maintain a continuous downward trend, thereby finally guaranteeing the generalization and robustness of MDNN.


### 4.3.2. Comparison of Methods

To further verify the efficiency of MDNN, nine classifiers in total are used to compare with MDNN on nine multiclassification datasets from the UCI machine learning repository and the QWS dataset. The information of datasets and parameter settings of MDNN are listed in Table5. The nine classifiers consist of BP, SVM, KNN, CART, naïve Bayes, LDA, QDA, J48, and random forest [44]. In addition, the ten datasets include Iris, Wine, Vehicle, Balance scale, CMC, Seed, Vowel, Thyroid, Robot navigation, and QWS.Experimental results are shown in Table6 where the best result for each dataset among all compared methods is highlighted in bold. According to five statistical indicators, it can be found that MDNN has the most optimal values in comparison with other classifiers. On the Iris, Wine, Seed, and Thyroid datasets, MDNN gives the best performance, with perfect outcomes of 100 percent correctness. Also, MDNN performs well with unbalanced data such as Vowel, Thyroid, and QWS. As a result, MDNN’s classification performance is more constant than that of other methods. Nevertheless, MDNN appears to have a minor disadvantage on the large datasets, such as Balance scale, Vehicle, CMC, and Robot navigation, which seem to be constrained by the distribution of the network structure. In the comparison between MDNN and other classifiers, the superiority and effectiveness of the multiple dendritic neuron structure are verified.Table 6
Comparison between MDNN and nine classifiers.
PrecisionRecallF1AccuracyAUCPrecisionRecallF1AccuracyAUCPrecisionRecallF1AccuracyAUCBPSVMKNNIris96.3096.1395.9197.3999.6896.4396.1196.0697.3999.7696.4696.5296.2997.6898.78Wine27.7443.3031.6565.6062.2943.6240.2531.3164.2872.2971.0271.0269.1080.7487.78Vehicle12.2628.4614.9163.5754.0312.1124.6510.2961.7738.1262.8664.2262.1982.1886.18Balance scale67.5067.8466.3793.3390.6565.7066.6065.3691.3594.5959.7056.5257.2483.4365.29CMC37.2342.8538.2665.6964.1554.4551.6451.6970.3772.0647.2246.0145.0666.0665.75Seed90.7990.4389.8993.5497.5690.3990.2489.9893.4798.8289.0588.7288.0292.4396.36Vowel3.8216.676.2174.3141.9983.7853.2157.5486.2180.7483.6283.2782.8894.8295.55Thyroid73.8471.6170.8690.5687.9768.7274.5767.0389.7996.4794.3684.2587.6995.2896.81Robot navigation73.7663.3965.3288.1891.3890.1887.5288.7394.7698.1785.0384.8684.7392.6895.79QWS17.6430.7119.4268.9451.3485.0384.8684.7392.6832.6962.1660.4959.3978.6181.92CARTNaive BayesLDAIris96.2696.1696.0297.2097.8295.9795.8795.5297.4999.5298.0798.1397.9498.6599.07Wine91.5492.5491.3694.5795.1596.3596.9596.3497.7099.8997.8998.2097.8998.6898.55Vehicle68.9169.1868.7384.4684.2651.6746.1141.6872.6677.4177.6378.5377.7189.2387.98Balance scale55.7557.2756.2585.7276.3560.9965.6262.8893.2289.6673.6880.3267.5882.6582.20CMC49.7949.2349.1567.6367.3749.4549.8847.5665.1267.5649.8850.5249.0666.4463.64Seed88.8188.5687.9992.2992.9891.4291.4191.1494.2498.6796.7796.7696.6597.9297.59Vowel96.7796.7696.6597.9297.5977.7277.5076.8993.2296.7575.3676.8575.0692.6284.00Thyroid87.4688.2286.3194.5893.0097.3495.7796.1598.2699.6595.8188.1590.5395.6993.08Robot navigation99.1799.2299.1999.6799.5954.9766.4454.2076.3683.6656.8066.7358.5380.8275.68QWS98.1898.7198.4099.7398.6481.6184.1281.7190.6495.7886.3586.3585.7793.4289.55QDAJ48Random ForestIris97.6097.6197.4498.3698.82100.0097.7898.8394.0798.89100.00100.00100.0094.81100.00Wine98.9598.6098.6799.1899.1892.5493.1992.6391.5794.6897.1598.1297.5696.2699.82Vehicle85.2085.4985.0292.5792.1986.6789.8988.1571.6094.0687.4795.2191.1074.8899.09Balance scale83.5791.0485.6294.4791.713.822.713.0377.9253.652.753.403.0280.3353.38CMC62.2860.5760.8674.9678.1261.3661.2561.2051.9469.3159.3264.5961.7750.6771.65Seed94.9494.4094.4896.3995.2286.9886.8386.5991.0689.5987.9190.3288.9292.4996.11Vowel75.4777.5075.5092.8084.4263.1757.6459.3083.5286.9464.8164.9964.2385.1189.00Thyroid94.2095.2994.1797.2997.1794.3995.1194.6792.5191.3195.4398.3796.8495.5098.61Robot navigation64.4275.5966.7383.4482.7498.9099.3799.1399.4099.6798.6199.4299.0199.4199.98QWS83.6585.1883.7091.5289.3197.99100.0098.9599.7699.8697.0496.7596.7898.9999.93MDNNIris100.00100.00100.00100.00100.00Wine100.00100.00100.00100.00100.00Vehicle85.3485.4885.3393.3196.12Balance scale80.5077.7678.9295.7493.26CMC62.2860.5760.8674.9678.12Seed100.00100.00100.00100.00100.00Vowel91.2984.2686.4795.6798.03Thyroid100.00100.00100.00100.00100.00Robot navigation94.8992.1193.4197.1999.30QWS99.1098.6398.7799.5299.68The receiver operating characteristic (ROC) curves of ten multiclassification methods show the correct classification coverage of each class of the QWS dataset in Figure6. It can be found that MDNN not only has a consistent performance on each class of QWS but also outperforms the other classifiers, thus indicating the effectiveness and stability of MDNN for the QWS classification and unbalanced multiclassification applications. Besides, experiments also demonstrate the efficiency and superiority of MDNN in terms of classification performance and stability.Figure 6
(a) The comparison of receiver operating characteristic curves for the platinum class of the QWS dataset. (b) The comparison of receiver operating characteristic curves for the gold class of the QWS dataset. (c) The comparison of receiver operating characteristic curves for the silver class of the QWS dataset. (d) The comparison of receiver operating characteristic curves for the bronze class of the QWS dataset.
(a)(b)(c)(d)


## 4.3.1. Experimental Results

For the comprehensive performance evaluation of MDNN, the following statistical indicators are used: precision, recall,F1 score, accuracy, and area under the curve (AUC) [43]. It is worth noting that the precision refers to the classification precision. For not only the convenience of the intuitive evaluation but also the justification of the following comparison, the macroaverage, which is the arithmetic average of performance indicators of all categories instead of instances, is adopted to statistically process classification results.Table4 shows the classification results of MDNN on the 4-class QWS dataset. The average values and optimum values represent the classification performance of MDNN. Although the dataset is technically unbalanced, such as the number of classes of QWS in Table 5, the stability and generalization ability of MDNN are considered to be effectively validated.Table 4
Average and optimum values of classification results of MDNN for QWS.
PlatinumGoldSilverBronzeMean valuesPrecision(Average)99.4498.4898.9299.5599.10(Optimum)100.00100.00100.00100.00100.00Recall(Average)96.5299.7299.4698.8198.63(Optimum)100.00100.00100.00100.00100.00F1(Average)97.7399.0599.1699.1598.77(Optimum)100.00100.00100.00100.00100.00Accuracy(Average)99.5899.5299.4599.5299.52(Optimum)100.00100.00100.00100.00100.00AUC(Average)99.1199.7199.9299.9799.68(Optimum)100.00100.0099.85100.0099.96Table 5
Description of datasets and parameter settings of MDNN.
DatasetsInstancesNumber of featuresNumber of classesSize of classesMηIris1504350, 50, 5040.01Wine17813359, 71, 4850.01Vehicle846184199, 217, 218, 212100.02Balance scale6254349, 288, 288120.01CMC147393629, 333, 511200.03Seed2107370, 70, 7050.01Vowel8713672, 89, 172, 151, 207, 18060.01Thyroid21553150, 35, 3040.01Robot navigation545624482, 620, 972, 205, 329300.02QWS36410441, 100, 120, 10350.01The five statistical indicators suggest that the classification achieved by MDNN for each class is effective, stable, and reliable. In their mean values, it is indicated that MDNN has good classification performance. The gradient descent optimization strategy effectively reduces the errors. Moreover, MDNN accelerates the gradient descent to maintain a continuous downward trend, thereby finally guaranteeing the generalization and robustness of MDNN.


## 4.3.2. Comparison of Methods

To further verify the efficiency of MDNN, nine classifiers in total are used to compare with MDNN on nine multiclassification datasets from the UCI machine learning repository and the QWS dataset. The information of datasets and parameter settings of MDNN are listed in Table5. The nine classifiers consist of BP, SVM, KNN, CART, naïve Bayes, LDA, QDA, J48, and random forest [44]. In addition, the ten datasets include Iris, Wine, Vehicle, Balance scale, CMC, Seed, Vowel, Thyroid, Robot navigation, and QWS.Experimental results are shown in Table6 where the best result for each dataset among all compared methods is highlighted in bold. According to five statistical indicators, it can be found that MDNN has the most optimal values in comparison with other classifiers. On the Iris, Wine, Seed, and Thyroid datasets, MDNN gives the best performance, with perfect outcomes of 100 percent correctness. Also, MDNN performs well with unbalanced data such as Vowel, Thyroid, and QWS. As a result, MDNN’s classification performance is more constant than that of other methods. Nevertheless, MDNN appears to have a minor disadvantage on the large datasets, such as Balance scale, Vehicle, CMC, and Robot navigation, which seem to be constrained by the distribution of the network structure. In the comparison between MDNN and other classifiers, the superiority and effectiveness of the multiple dendritic neuron structure are verified.Table 6
Comparison between MDNN and nine classifiers.
PrecisionRecallF1AccuracyAUCPrecisionRecallF1AccuracyAUCPrecisionRecallF1AccuracyAUCBPSVMKNNIris96.3096.1395.9197.3999.6896.4396.1196.0697.3999.7696.4696.5296.2997.6898.78Wine27.7443.3031.6565.6062.2943.6240.2531.3164.2872.2971.0271.0269.1080.7487.78Vehicle12.2628.4614.9163.5754.0312.1124.6510.2961.7738.1262.8664.2262.1982.1886.18Balance scale67.5067.8466.3793.3390.6565.7066.6065.3691.3594.5959.7056.5257.2483.4365.29CMC37.2342.8538.2665.6964.1554.4551.6451.6970.3772.0647.2246.0145.0666.0665.75Seed90.7990.4389.8993.5497.5690.3990.2489.9893.4798.8289.0588.7288.0292.4396.36Vowel3.8216.676.2174.3141.9983.7853.2157.5486.2180.7483.6283.2782.8894.8295.55Thyroid73.8471.6170.8690.5687.9768.7274.5767.0389.7996.4794.3684.2587.6995.2896.81Robot navigation73.7663.3965.3288.1891.3890.1887.5288.7394.7698.1785.0384.8684.7392.6895.79QWS17.6430.7119.4268.9451.3485.0384.8684.7392.6832.6962.1660.4959.3978.6181.92CARTNaive BayesLDAIris96.2696.1696.0297.2097.8295.9795.8795.5297.4999.5298.0798.1397.9498.6599.07Wine91.5492.5491.3694.5795.1596.3596.9596.3497.7099.8997.8998.2097.8998.6898.55Vehicle68.9169.1868.7384.4684.2651.6746.1141.6872.6677.4177.6378.5377.7189.2387.98Balance scale55.7557.2756.2585.7276.3560.9965.6262.8893.2289.6673.6880.3267.5882.6582.20CMC49.7949.2349.1567.6367.3749.4549.8847.5665.1267.5649.8850.5249.0666.4463.64Seed88.8188.5687.9992.2992.9891.4291.4191.1494.2498.6796.7796.7696.6597.9297.59Vowel96.7796.7696.6597.9297.5977.7277.5076.8993.2296.7575.3676.8575.0692.6284.00Thyroid87.4688.2286.3194.5893.0097.3495.7796.1598.2699.6595.8188.1590.5395.6993.08Robot navigation99.1799.2299.1999.6799.5954.9766.4454.2076.3683.6656.8066.7358.5380.8275.68QWS98.1898.7198.4099.7398.6481.6184.1281.7190.6495.7886.3586.3585.7793.4289.55QDAJ48Random ForestIris97.6097.6197.4498.3698.82100.0097.7898.8394.0798.89100.00100.00100.0094.81100.00Wine98.9598.6098.6799.1899.1892.5493.1992.6391.5794.6897.1598.1297.5696.2699.82Vehicle85.2085.4985.0292.5792.1986.6789.8988.1571.6094.0687.4795.2191.1074.8899.09Balance scale83.5791.0485.6294.4791.713.822.713.0377.9253.652.753.403.0280.3353.38CMC62.2860.5760.8674.9678.1261.3661.2561.2051.9469.3159.3264.5961.7750.6771.65Seed94.9494.4094.4896.3995.2286.9886.8386.5991.0689.5987.9190.3288.9292.4996.11Vowel75.4777.5075.5092.8084.4263.1757.6459.3083.5286.9464.8164.9964.2385.1189.00Thyroid94.2095.2994.1797.2997.1794.3995.1194.6792.5191.3195.4398.3796.8495.5098.61Robot navigation64.4275.5966.7383.4482.7498.9099.3799.1399.4099.6798.6199.4299.0199.4199.98QWS83.6585.1883.7091.5289.3197.99100.0098.9599.7699.8697.0496.7596.7898.9999.93MDNNIris100.00100.00100.00100.00100.00Wine100.00100.00100.00100.00100.00Vehicle85.3485.4885.3393.3196.12Balance scale80.5077.7678.9295.7493.26CMC62.2860.5760.8674.9678.12Seed100.00100.00100.00100.00100.00Vowel91.2984.2686.4795.6798.03Thyroid100.00100.00100.00100.00100.00Robot navigation94.8992.1193.4197.1999.30QWS99.1098.6398.7799.5299.68The receiver operating characteristic (ROC) curves of ten multiclassification methods show the correct classification coverage of each class of the QWS dataset in Figure6. It can be found that MDNN not only has a consistent performance on each class of QWS but also outperforms the other classifiers, thus indicating the effectiveness and stability of MDNN for the QWS classification and unbalanced multiclassification applications. Besides, experiments also demonstrate the efficiency and superiority of MDNN in terms of classification performance and stability.Figure 6
(a) The comparison of receiver operating characteristic curves for the platinum class of the QWS dataset. (b) The comparison of receiver operating characteristic curves for the gold class of the QWS dataset. (c) The comparison of receiver operating characteristic curves for the silver class of the QWS dataset. (d) The comparison of receiver operating characteristic curves for the bronze class of the QWS dataset.
(a)(b)(c)(d)


## 4.4. Morphology and Logical Circle Realization

For the display of a data sample, the shuffle operation set in the pretraining period is at disposal. According to the initialization ofwj,i,m and qj,i,m within Section 2, the synapses were calculated around 0.5 in the previous state. Through the training stage, the weights wj,i,m and qj,i,m were gradually stabilized. As shown in Figure 7, synaptic changes, thus, yield to accomplish the pruning of the redundant network structure.Figure 7
Dendritic changes in the structure of randomly selected samples before and after network training were applied to the QWS dataset.Z-axis represents the neuron, and X-axis and Y-axis are attributes and hidden layers, respectively.It can be easily observed that neuron 1, neuron 2, and neuron 4 are fully inhibited in accordance with the rule of Equation. (2) and Equation. (3). Consequently, the structure of neuron 1, neuron 3, and neuron 4 is ruled out. To specify the states of dendrites, a total of four scenarios are listed as follows:(28)dendriticstate=constant−1connection,whenqj,i,m<wj,i,m<0orqj,i,m<0<wj,i,m,constant−0connection,when0<wj,i,m<qj,i,morwj,i,m<0<qj,i,m,excitatoryconnection,when0<qj,i,m<wj,i,m,inhibitoryconnection.whenwj,i,m<qj,i,m<0.For the remainder neuron, the residual dendrite morphology is formed in Figure8(a), which places the line of dashes indicating pruning. As mentioned previously in Section 2, logic-based inherent relations have existed within dendritic structures. Finally, since constant-1 holds no substantive impact on the attributes, the connections among dendrites are equivalent to logic OR. Thus, the hardware realization is transformed as illustrated in Figure 8(b), where the multiplexer is a 1 : 2 numerical compactor. In addition to showing the extendibility of MDNN, the overhead also indicates that MDNN can avoid overfitting by increasing the dendritic matrix sparsity.Figure 8
(a) The dendritic states of MDNN are fixated on the QWS dataset. Circle 0 and Circle 1 stand for constant-0 and constant-1 connection, respectively. In addition, black-filled square and circle stand for inhibitory and excitatory states, respectively. (b) Logical circle realization of MDNN on the QWS dataset, whereλ = qj,i,m/wj,i,m.
(a)(b)


## 5. Conclusion and Future Directions

This article puts forward a novel extended network of dendritic neurons, namely, the multiple dendritic neural network (MDNN). The architecture of MDNN is completely different from the previous DL models which are based on MP neuron models. By deriving its new learning algorithms, MDNN is for the first time able to resolve the multiclassification problems in comparison with previous single dendritic neuron models. Besides, we propose an approach to improve the interpretability of artificial neural networks with the theoretical support of neuroscience. Experiments are mainly carried out on a QoS-related application. In the comparison between MDNN and other classifiers, the superior performance of the proposed model is shown, and MDNN is also highly advantageous to small-scale unbalanced data. In view of this, the performance and efficiency of the proposed neural network are limited by scale. In the follow-up work, the deficiency of this experiment will be made up to improve the generalization ability [45, 46] and study the capabilities and limitations. Meanwhile, the exploration of applicable domains for MDNN will be conducted in the following aspects: 1) expanding research on more computer-related data mining to solve practical engineering problems, such as quality of service of mobile networks [47] and security bug report [48, 49]; 2) practicing in other forms of data structures, e.g., semantic [50]; 3) focusing on the unbalanced data [51] and simplifying the network structure adequately [52] with the practice.



---

*Source: 7037124-2023-01-23.xml*